Aug 09 02:12:44 user nova-compute[72351]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code.
Aug 09 02:12:46 user nova-compute[72351]: DEBUG os_vif [-] Loaded VIF plugin class '<class 'vif_plug_linux_bridge.linux_bridge.LinuxBridgePlugin'>' with name 'linux_bridge' {{(pid=72351) initialize /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:44}}
Aug 09 02:12:46 user nova-compute[72351]: DEBUG os_vif [-] Loaded VIF plugin class '<class 'vif_plug_noop.noop.NoOpPlugin'>' with name 'noop' {{(pid=72351) initialize /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:44}}
Aug 09 02:12:46 user nova-compute[72351]: DEBUG os_vif [-] Loaded VIF plugin class '<class 'vif_plug_ovs.ovs.OvsPlugin'>' with name 'ovs' {{(pid=72351) initialize /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:44}}
Aug 09 02:12:46 user nova-compute[72351]: INFO os_vif [-] Loaded VIF plugins: linux_bridge, noop, ovs
Aug 09 02:12:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): grep -F node.session.scan /sbin/iscsiadm {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:12:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "grep -F node.session.scan /sbin/iscsiadm" returned: 0 in 0.018s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:12:47 user nova-compute[72351]: INFO nova.virt.driver [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] Loading compute driver 'libvirt.LibvirtDriver'
Aug 09 02:12:47 user nova-compute[72351]: INFO nova.compute.provider_config [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] No provider configs found in /etc/nova/provider_config/. If files are present, ensure the Nova process has access.
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] Acquiring lock "singleton_lock" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] Acquired lock "singleton_lock" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] Releasing lock "singleton_lock" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] Full set of CONF: {{(pid=72351) _wait_for_exit_or_signal /usr/local/lib/python3.10/dist-packages/oslo_service/service.py:362}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] ******************************************************************************** {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2589}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] Configuration options gathered from: {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2590}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] command line args: ['--config-file', '/etc/nova/nova-cpu.conf'] {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2591}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] config files: ['/etc/nova/nova-cpu.conf'] {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2592}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] ================================================================================ {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2594}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] allow_resize_to_same_host      = True {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] arq_binding_timeout            = 300 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] backdoor_port                  = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] backdoor_socket                = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] block_device_allocate_retries  = 300 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] block_device_allocate_retries_interval = 5 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cert                           = self.pem {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] compute_driver                 = libvirt.LibvirtDriver {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] compute_monitors               = [] {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] config_dir                     = [] {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] config_drive_format            = iso9660 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] config_file                    = ['/etc/nova/nova-cpu.conf'] {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] config_source                  = [] {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] console_host                   = user {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] control_exchange               = nova {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cpu_allocation_ratio           = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] daemon                         = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] debug                          = True {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] default_access_ip_network_name = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] default_availability_zone      = nova {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] default_ephemeral_format       = ext4 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] default_log_levels             = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] default_schedule_zone          = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] disk_allocation_ratio          = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] enable_new_services            = True {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] enabled_apis                   = ['osapi_compute'] {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] enabled_ssl_apis               = [] {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] flat_injected                  = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] force_config_drive             = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] force_raw_images               = True {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] graceful_shutdown_timeout      = 5 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] heal_instance_info_cache_interval = 60 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] host                           = user {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] initial_cpu_allocation_ratio   = 4.0 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] initial_disk_allocation_ratio  = 1.0 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] initial_ram_allocation_ratio   = 1.0 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] injected_network_template      = /opt/stack/nova/nova/virt/interfaces.template {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] instance_build_timeout         = 0 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] instance_delete_interval       = 300 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] instance_format                = [instance: %(uuid)s]  {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] instance_name_template         = instance-%08x {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] instance_usage_audit           = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] instance_usage_audit_period    = month {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] instance_uuid_format           = [instance: %(uuid)s]  {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] instances_path                 = /opt/stack/data/nova/instances {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] internal_service_availability_zone = internal {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] key                            = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] live_migration_retry_count     = 30 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] log_config_append              = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] log_date_format                = %Y-%m-%d %H:%M:%S {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] log_dir                        = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] log_file                       = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] log_options                    = True {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] log_rotate_interval            = 1 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] log_rotate_interval_type       = days {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] log_rotation_type              = none {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] logging_context_format_string  = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] logging_debug_format_suffix    = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] logging_default_format_string  = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] logging_exception_prefix       = ERROR %(name)s %(instance)s {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] logging_user_identity_format   = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] long_rpc_timeout               = 1800 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] max_concurrent_builds          = 10 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] max_concurrent_live_migrations = 1 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] max_concurrent_snapshots       = 5 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] max_local_block_devices        = 3 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] max_logfile_count              = 30 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] max_logfile_size_mb            = 200 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] maximum_instance_delete_attempts = 5 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] metadata_listen                = 0.0.0.0 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] metadata_listen_port           = 8775 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] metadata_workers               = 3 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] migrate_max_retries            = -1 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] mkisofs_cmd                    = genisoimage {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] my_block_storage_ip            = 10.0.0.210 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] my_ip                          = 10.0.0.210 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] network_allocate_retries       = 0 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] osapi_compute_listen           = 0.0.0.0 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] osapi_compute_listen_port      = 8774 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] osapi_compute_unique_server_name_scope =  {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] osapi_compute_workers          = 3 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] password_length                = 12 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] periodic_enable                = True {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] periodic_fuzzy_delay           = 60 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] pointer_model                  = ps2mouse {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] preallocate_images             = none {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] publish_errors                 = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] pybasedir                      = /opt/stack/nova {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] ram_allocation_ratio           = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] rate_limit_burst               = 0 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] rate_limit_except_level        = CRITICAL {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] rate_limit_interval            = 0 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] reboot_timeout                 = 0 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] reclaim_instance_interval      = 0 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] record                         = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] reimage_timeout_per_gb         = 60 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] report_interval                = 120 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] rescue_timeout                 = 0 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] reserved_host_cpus             = 0 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] reserved_host_disk_mb          = 0 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] reserved_host_memory_mb        = 512 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] reserved_huge_pages            = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] resize_confirm_window          = 0 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] resize_fs_using_block_device   = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] resume_guests_state_on_host_boot = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] rootwrap_config                = /etc/nova/rootwrap.conf {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] rpc_response_timeout           = 60 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] run_external_periodic_tasks    = True {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] running_deleted_instance_action = reap {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] running_deleted_instance_poll_interval = 1800 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] running_deleted_instance_timeout = 0 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] scheduler_instance_sync_interval = 120 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] service_down_time              = 720 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] servicegroup_driver            = db {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] shelved_offload_time           = 0 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] shelved_poll_interval          = 3600 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] shutdown_timeout               = 0 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] source_is_ipv6                 = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] ssl_only                       = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] state_path                     = /opt/stack/data/nova {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] sync_power_state_interval      = 600 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] sync_power_state_pool_size     = 1000 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] syslog_log_facility            = LOG_USER {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] tempdir                        = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] timeout_nbd                    = 10 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] transport_url                  = **** {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] update_resources_interval      = 0 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] use_cow_images                 = True {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] use_eventlog                   = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] use_journal                    = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] use_json                       = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] use_rootwrap_daemon            = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] use_stderr                     = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] use_syslog                     = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vcpu_pin_set                   = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vif_plugging_is_fatal          = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vif_plugging_timeout           = 0 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] virt_mkfs                      = [] {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] volume_usage_poll_interval     = 0 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] watch_log_file                 = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] web                            = /usr/share/spice-html5 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_concurrency.disable_process_locking = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_concurrency.lock_path     = /opt/stack/data/nova {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_messaging_metrics.metrics_enabled = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_messaging_metrics.metrics_process_name =  {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] api.auth_strategy              = keystone {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] api.compute_link_prefix        = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] api.dhcp_domain                = novalocal {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] api.enable_instance_password   = True {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] api.glance_link_prefix         = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] api.instance_list_cells_batch_fixed_size = 100 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] api.instance_list_cells_batch_strategy = distributed {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] api.instance_list_per_project_cells = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] api.list_records_by_skipping_down_cells = True {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] api.local_metadata_per_cell    = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] api.max_limit                  = 1000 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] api.metadata_cache_expiration  = 15 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] api.neutron_default_tenant_id  = default {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] api.use_forwarded_for          = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] api.use_neutron_default_nets   = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] api.vendordata_dynamic_connect_timeout = 5 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] api.vendordata_dynamic_failure_fatal = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] api.vendordata_dynamic_read_timeout = 5 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] api.vendordata_dynamic_ssl_certfile =  {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] api.vendordata_dynamic_targets = [] {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] api.vendordata_jsonfile_path   = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] api.vendordata_providers       = ['StaticJSON'] {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cache.backend                  = dogpile.cache.memcached {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cache.backend_argument         = **** {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cache.config_prefix            = cache.oslo {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cache.dead_timeout             = 60.0 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cache.debug_cache_backend      = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cache.enable_retry_client      = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cache.enable_socket_keepalive  = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cache.enabled                  = True {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cache.expiration_time          = 600 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cache.hashclient_retry_attempts = 2 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cache.hashclient_retry_delay   = 1.0 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cache.memcache_dead_retry      = 300 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cache.memcache_password        =  {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cache.memcache_pool_connection_get_timeout = 10 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cache.memcache_pool_flush_on_reconnect = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cache.memcache_pool_maxsize    = 10 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cache.memcache_pool_unused_timeout = 60 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cache.memcache_sasl_enabled    = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cache.memcache_servers         = ['localhost:11211'] {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cache.memcache_socket_timeout  = 1.0 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cache.memcache_username        =  {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cache.proxies                  = [] {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cache.retry_attempts           = 2 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cache.retry_delay              = 0.0 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cache.socket_keepalive_count   = 1 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cache.socket_keepalive_idle    = 1 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cache.socket_keepalive_interval = 1 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cache.tls_allowed_ciphers      = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cache.tls_cafile               = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cache.tls_certfile             = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cache.tls_enabled              = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cache.tls_keyfile              = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cinder.auth_section            = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cinder.auth_type               = password {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cinder.cafile                  = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cinder.catalog_info            = volumev3::publicURL {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cinder.certfile                = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cinder.collect_timing          = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cinder.cross_az_attach         = True {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cinder.debug                   = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cinder.endpoint_template       = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cinder.http_retries            = 3 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cinder.insecure                = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cinder.keyfile                 = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cinder.os_region_name          = RegionOne {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cinder.split_loggers           = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cinder.timeout                 = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] compute.consecutive_build_service_disable_threshold = 10 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] compute.cpu_dedicated_set      = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] compute.cpu_shared_set         = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] compute.image_type_exclude_list = [] {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] compute.live_migration_wait_for_vif_plug = True {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] compute.max_concurrent_disk_ops = 0 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] compute.max_disk_devices_to_attach = -1 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] compute.resource_provider_association_refresh = 300 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] compute.shutdown_retry_interval = 10 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] compute.vmdk_allowed_types     = ['streamOptimized', 'monolithicSparse'] {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] conductor.workers              = 3 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] console.allowed_origins        = [] {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] console.ssl_ciphers            = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] console.ssl_minimum_version    = default {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] consoleauth.token_ttl          = 600 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cyborg.cafile                  = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cyborg.certfile                = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cyborg.collect_timing          = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cyborg.connect_retries         = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cyborg.connect_retry_delay     = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cyborg.endpoint_override       = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cyborg.insecure                = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cyborg.keyfile                 = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cyborg.max_version             = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cyborg.min_version             = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cyborg.region_name             = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cyborg.service_name            = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cyborg.service_type            = accelerator {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cyborg.split_loggers           = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cyborg.status_code_retries     = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cyborg.status_code_retry_delay = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cyborg.timeout                 = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cyborg.valid_interfaces        = ['internal', 'public'] {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] cyborg.version                 = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] database.backend               = sqlalchemy {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] database.connection            = **** {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] database.connection_debug      = 0 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] database.connection_parameters =  {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] database.connection_recycle_time = 3600 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] database.connection_trace      = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] database.db_inc_retry_interval = True {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] database.db_max_retries        = 20 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] database.db_max_retry_interval = 10 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] database.db_retry_interval     = 1 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] database.max_overflow          = 50 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] database.max_pool_size         = 5 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] database.max_retries           = 10 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] database.mysql_enable_ndb      = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] database.mysql_sql_mode        = TRADITIONAL {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] database.mysql_wsrep_sync_wait = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] database.pool_timeout          = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] database.retry_interval        = 10 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] database.slave_connection      = **** {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] database.sqlite_synchronous    = True {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] api_database.backend           = sqlalchemy {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] api_database.connection        = **** {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] api_database.connection_debug  = 0 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] api_database.connection_parameters =  {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] api_database.connection_recycle_time = 3600 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] api_database.connection_trace  = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] api_database.db_inc_retry_interval = True {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] api_database.db_max_retries    = 20 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] api_database.db_max_retry_interval = 10 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] api_database.db_retry_interval = 1 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] api_database.max_overflow      = 50 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] api_database.max_pool_size     = 5 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] api_database.max_retries       = 10 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] api_database.mysql_enable_ndb  = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] api_database.mysql_sql_mode    = TRADITIONAL {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] api_database.mysql_wsrep_sync_wait = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] api_database.pool_timeout      = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] api_database.retry_interval    = 10 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] api_database.slave_connection  = **** {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] api_database.sqlite_synchronous = True {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] devices.enabled_mdev_types     = [] {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] ephemeral_storage_encryption.enabled = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] ephemeral_storage_encryption.key_size = 512 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] glance.api_servers             = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] glance.cafile                  = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] glance.certfile                = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] glance.collect_timing          = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] glance.connect_retries         = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] glance.connect_retry_delay     = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] glance.debug                   = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] glance.default_trusted_certificate_ids = [] {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] glance.enable_certificate_validation = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] glance.enable_rbd_download     = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] glance.endpoint_override       = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] glance.insecure                = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] glance.keyfile                 = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] glance.max_version             = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] glance.min_version             = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] glance.num_retries             = 3 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] glance.rbd_ceph_conf           =  {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] glance.rbd_connect_timeout     = 5 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] glance.rbd_pool                =  {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] glance.rbd_user                =  {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] glance.region_name             = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] glance.service_name            = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] glance.service_type            = image {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] glance.split_loggers           = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] glance.status_code_retries     = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] glance.status_code_retry_delay = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] glance.timeout                 = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] glance.valid_interfaces        = ['internal', 'public'] {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] glance.verify_glance_signatures = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] glance.version                 = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] guestfs.debug                  = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] hyperv.config_drive_cdrom      = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] hyperv.config_drive_inject_password = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] hyperv.dynamic_memory_ratio    = 1.0 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] hyperv.enable_instance_metrics_collection = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] hyperv.enable_remotefx         = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] hyperv.instances_path_share    =  {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] hyperv.iscsi_initiator_list    = [] {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] hyperv.limit_cpu_features      = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] hyperv.mounted_disk_query_retry_count = 10 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] hyperv.power_state_check_timeframe = 60 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] hyperv.power_state_event_polling_interval = 2 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] hyperv.qemu_img_cmd            = qemu-img.exe {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] hyperv.use_multipath_io        = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] hyperv.volume_attach_retry_count = 10 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] hyperv.volume_attach_retry_interval = 5 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] hyperv.vswitch_name            = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] hyperv.wait_soft_reboot_seconds = 60 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] mks.enabled                    = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] mks.mksproxy_base_url          = http://127.0.0.1:6090/ {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] image_cache.manager_interval   = 2400 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] image_cache.precache_concurrency = 1 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] image_cache.remove_unused_base_images = True {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] image_cache.subdirectory_name  = _base {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] ironic.api_max_retries         = 60 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] ironic.api_retry_interval      = 2 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] ironic.auth_section            = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] ironic.auth_type               = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] ironic.cafile                  = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] ironic.certfile                = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] ironic.collect_timing          = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] ironic.connect_retries         = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] ironic.connect_retry_delay     = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] ironic.endpoint_override       = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] ironic.insecure                = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] ironic.keyfile                 = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] ironic.max_version             = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] ironic.min_version             = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] ironic.partition_key           = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] ironic.peer_list               = [] {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] ironic.region_name             = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] ironic.serial_console_state_timeout = 10 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] ironic.service_name            = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] ironic.service_type            = baremetal {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] ironic.split_loggers           = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] ironic.status_code_retries     = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] ironic.status_code_retry_delay = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] ironic.timeout                 = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] ironic.valid_interfaces        = ['internal', 'public'] {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] ironic.version                 = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] key_manager.backend            = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] key_manager.fixed_key          = **** {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] barbican.auth_endpoint         = http://localhost/identity/v3 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] barbican.barbican_api_version  = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] barbican.barbican_endpoint     = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] barbican.barbican_endpoint_type = public {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] barbican.barbican_region_name  = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] barbican.cafile                = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] barbican.certfile              = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] barbican.collect_timing        = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] barbican.insecure              = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] barbican.keyfile               = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] barbican.number_of_retries     = 60 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] barbican.retry_delay           = 1 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] barbican.send_service_user_token = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] barbican.split_loggers         = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] barbican.timeout               = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] barbican.verify_ssl            = True {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] barbican.verify_ssl_path       = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] barbican_service_user.auth_section = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] barbican_service_user.auth_type = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] barbican_service_user.cafile   = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] barbican_service_user.certfile = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] barbican_service_user.collect_timing = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] barbican_service_user.insecure = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] barbican_service_user.keyfile  = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] barbican_service_user.split_loggers = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] barbican_service_user.timeout  = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vault.approle_role_id          = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vault.approle_secret_id        = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vault.cafile                   = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vault.certfile                 = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vault.collect_timing           = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vault.insecure                 = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vault.keyfile                  = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vault.kv_mountpoint            = secret {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vault.kv_version               = 2 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vault.namespace                = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vault.root_token_id            = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vault.split_loggers            = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vault.ssl_ca_crt_file          = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vault.timeout                  = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vault.use_ssl                  = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vault.vault_url                = http://127.0.0.1:8200 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] keystone.cafile                = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] keystone.certfile              = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] keystone.collect_timing        = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] keystone.connect_retries       = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] keystone.connect_retry_delay   = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] keystone.endpoint_override     = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] keystone.insecure              = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] keystone.keyfile               = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] keystone.max_version           = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] keystone.min_version           = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] keystone.region_name           = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] keystone.service_name          = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] keystone.service_type          = identity {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] keystone.split_loggers         = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] keystone.status_code_retries   = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] keystone.status_code_retry_delay = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] keystone.timeout               = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] keystone.valid_interfaces      = ['internal', 'public'] {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] keystone.version               = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.connection_uri         =  {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.cpu_mode               = custom {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.cpu_model_extra_flags  = [] {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: WARNING oslo_config.cfg [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] Deprecated: Option "cpu_model" from group "libvirt" is deprecated. Use option "cpu_models" from group "libvirt".
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.cpu_models             = ['Nehalem'] {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.cpu_power_governor_high = performance {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.cpu_power_governor_low = powersave {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.cpu_power_management   = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.cpu_power_management_strategy = cpu_state {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.device_detach_attempts = 8 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.device_detach_timeout  = 20 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.disk_cachemodes        = [] {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.disk_prefix            = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.enabled_perf_events    = [] {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.file_backed_memory     = 0 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.gid_maps               = [] {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.hw_disk_discard        = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.hw_machine_type        = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.images_rbd_ceph_conf   =  {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.images_rbd_glance_store_name =  {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.images_rbd_pool        = rbd {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.images_type            = default {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.images_volume_group    = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.inject_key             = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.inject_partition       = -2 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.inject_password        = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.iscsi_iface            = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.iser_use_multipath     = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.live_migration_bandwidth = 0 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.live_migration_completion_timeout = 800 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.live_migration_downtime = 500 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.live_migration_downtime_delay = 75 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.live_migration_downtime_steps = 10 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.live_migration_inbound_addr = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.live_migration_permit_auto_converge = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.live_migration_permit_post_copy = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.live_migration_scheme  = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.live_migration_timeout_action = abort {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.live_migration_tunnelled = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: WARNING oslo_config.cfg [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] Deprecated: Option "live_migration_uri" from group "libvirt" is deprecated for removal (
Aug 09 02:12:47 user nova-compute[72351]: live_migration_uri is deprecated for removal in favor of two other options that
Aug 09 02:12:47 user nova-compute[72351]: allow to change live migration scheme and target URI: ``live_migration_scheme``
Aug 09 02:12:47 user nova-compute[72351]: and ``live_migration_inbound_addr`` respectively.
Aug 09 02:12:47 user nova-compute[72351]: ).  Its value may be silently ignored in the future.
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.live_migration_uri     = qemu+ssh://stack@%s/system {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.live_migration_with_native_tls = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.max_queues             = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.mem_stats_period_seconds = 10 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.nfs_mount_options      = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.nfs_mount_point_base   = /opt/stack/data/nova/mnt {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.num_aoe_discover_tries = 3 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.num_iser_scan_tries    = 5 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.num_memory_encrypted_guests = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.num_nvme_discover_tries = 5 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.num_pcie_ports         = 0 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.num_volume_scan_tries  = 5 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.pmem_namespaces        = [] {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.quobyte_client_cfg     = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.rbd_connect_timeout    = 5 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.rbd_destroy_volume_retries = 12 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.rbd_secret_uuid        = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.rbd_user               = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.realtime_scheduler_priority = 1 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.remote_filesystem_transport = ssh {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.rescue_image_id        = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.rescue_kernel_id       = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.rescue_ramdisk_id      = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.rng_dev_path           = /dev/urandom {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.rx_queue_size          = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.smbfs_mount_options    =  {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.snapshot_compression   = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.snapshot_image_format  = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.snapshots_directory    = /opt/stack/data/nova/instances/snapshots {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.sparse_logical_volumes = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.swtpm_enabled          = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.swtpm_group            = tss {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.swtpm_user             = tss {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.sysinfo_serial         = unique {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.tb_cache_size          = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.tx_queue_size          = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.uid_maps               = [] {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.use_virtio_for_bridges = True {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.virt_type              = kvm {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.volume_clear           = zero {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.volume_clear_size      = 0 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.volume_use_multipath   = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.vzstorage_cache_path   = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.vzstorage_log_path     = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.vzstorage_mount_group  = qemu {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.vzstorage_mount_opts   = [] {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.vzstorage_mount_perms  = 0770 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.vzstorage_mount_user   = stack {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] libvirt.wait_soft_reboot_seconds = 120 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] neutron.auth_section           = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] neutron.auth_type              = password {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] neutron.cafile                 = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] neutron.certfile               = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] neutron.collect_timing         = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] neutron.connect_retries        = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] neutron.connect_retry_delay    = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] neutron.default_floating_pool  = public {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] neutron.endpoint_override      = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] neutron.extension_sync_interval = 600 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] neutron.http_retries           = 3 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] neutron.insecure               = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] neutron.keyfile                = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] neutron.max_version            = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] neutron.metadata_proxy_shared_secret = **** {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] neutron.min_version            = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] neutron.ovs_bridge             = br-int {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] neutron.physnets               = [] {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] neutron.region_name            = RegionOne {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] neutron.service_metadata_proxy = True {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] neutron.service_name           = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] neutron.service_type           = network {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] neutron.split_loggers          = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] neutron.status_code_retries    = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] neutron.status_code_retry_delay = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] neutron.timeout                = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] neutron.valid_interfaces       = ['internal', 'public'] {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] neutron.version                = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] notifications.bdms_in_notifications = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] notifications.default_level    = INFO {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] notifications.notification_format = unversioned {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] notifications.notify_on_state_change = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] pci.alias                      = [] {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] pci.device_spec                = [] {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] pci.report_in_placement        = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] placement.auth_section         = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] placement.auth_type            = password {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] placement.auth_url             = http://10.0.0.210/identity {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] placement.cafile               = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] placement.certfile             = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] placement.collect_timing       = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] placement.connect_retries      = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] placement.connect_retry_delay  = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] placement.default_domain_id    = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] placement.default_domain_name  = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] placement.domain_id            = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] placement.domain_name          = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] placement.endpoint_override    = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] placement.insecure             = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] placement.keyfile              = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] placement.max_version          = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] placement.min_version          = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] placement.password             = **** {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] placement.project_domain_id    = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] placement.project_domain_name  = Default {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] placement.project_id           = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] placement.project_name         = service {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] placement.region_name          = RegionOne {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] placement.service_name         = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] placement.service_type         = placement {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] placement.split_loggers        = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] placement.status_code_retries  = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] placement.status_code_retry_delay = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] placement.system_scope         = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] placement.timeout              = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] placement.trust_id             = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] placement.user_domain_id       = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] placement.user_domain_name     = Default {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] placement.user_id              = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] placement.username             = placement {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] placement.valid_interfaces     = ['internal', 'public'] {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] placement.version              = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] quota.cores                    = 20 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] quota.count_usage_from_placement = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] quota.driver                   = nova.quota.DbQuotaDriver {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] quota.injected_file_content_bytes = 10240 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] quota.injected_file_path_length = 255 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] quota.injected_files           = 5 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] quota.instances                = 10 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] quota.key_pairs                = 100 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] quota.metadata_items           = 128 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] quota.ram                      = 51200 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] quota.recheck_quota            = True {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] quota.server_group_members     = 10 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] quota.server_groups            = 10 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] rdp.enabled                    = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] rdp.html5_proxy_base_url       = http://127.0.0.1:6083/ {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] scheduler.enable_isolated_aggregate_filtering = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] scheduler.image_metadata_prefilter = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] scheduler.max_attempts         = 3 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] scheduler.max_placement_results = 1000 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] scheduler.placement_aggregate_required_for_tenants = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] scheduler.query_placement_for_availability_zone = True {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] scheduler.query_placement_for_image_type_support = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] scheduler.workers              = 3 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] filter_scheduler.enabled_filters = ['ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] filter_scheduler.host_subset_size = 1 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] filter_scheduler.hypervisor_version_weight_multiplier = 1.0 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] filter_scheduler.image_properties_default_architecture = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] filter_scheduler.isolated_hosts = [] {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] filter_scheduler.isolated_images = [] {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] filter_scheduler.max_instances_per_host = 50 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] filter_scheduler.max_io_ops_per_host = 8 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] filter_scheduler.pci_in_placement = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] filter_scheduler.track_instance_changes = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] metrics.required               = True {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] metrics.weight_multiplier      = 1.0 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] metrics.weight_of_unavailable  = -10000.0 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] metrics.weight_setting         = [] {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] serial_console.base_url        = ws://127.0.0.1:6083/ {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] serial_console.enabled         = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] serial_console.port_range      = 10000:20000 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] serial_console.proxyclient_address = 127.0.0.1 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] serial_console.serialproxy_host = 0.0.0.0 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] serial_console.serialproxy_port = 6083 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] service_user.auth_section      = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] service_user.auth_type         = password {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] service_user.cafile            = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] service_user.certfile          = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] service_user.collect_timing    = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] service_user.insecure          = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] service_user.keyfile           = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] service_user.send_service_user_token = True {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] service_user.split_loggers     = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] service_user.timeout           = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] spice.agent_enabled            = True {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] spice.enabled                  = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] spice.html5proxy_base_url      = http://10.0.0.210:6081/spice_auto.html {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] spice.html5proxy_host          = 0.0.0.0 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] spice.html5proxy_port          = 6082 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] spice.image_compression        = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] spice.jpeg_compression         = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] spice.playback_compression     = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] spice.server_listen            = 127.0.0.1 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] spice.server_proxyclient_address = 127.0.0.1 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] spice.streaming_mode           = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] spice.zlib_compression         = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] upgrade_levels.baseapi         = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] upgrade_levels.cert            = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] upgrade_levels.compute         = auto {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] upgrade_levels.conductor       = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] upgrade_levels.scheduler       = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vendordata_dynamic_auth.auth_section = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vendordata_dynamic_auth.auth_type = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vendordata_dynamic_auth.cafile = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vendordata_dynamic_auth.certfile = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vendordata_dynamic_auth.collect_timing = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vendordata_dynamic_auth.insecure = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vendordata_dynamic_auth.keyfile = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vendordata_dynamic_auth.split_loggers = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vendordata_dynamic_auth.timeout = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vmware.api_retry_count         = 10 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vmware.ca_file                 = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vmware.cache_prefix            = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vmware.cluster_name            = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vmware.connection_pool_size    = 10 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vmware.console_delay_seconds   = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vmware.datastore_regex         = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vmware.host_ip                 = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vmware.host_password           = **** {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vmware.host_port               = 443 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vmware.host_username           = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vmware.insecure                = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vmware.integration_bridge      = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vmware.maximum_objects         = 100 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vmware.pbm_default_policy      = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vmware.pbm_enabled             = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vmware.pbm_wsdl_location       = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vmware.serial_log_dir          = /opt/vmware/vspc {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vmware.serial_port_proxy_uri   = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vmware.serial_port_service_uri = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vmware.task_poll_interval      = 0.5 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vmware.use_linked_clone        = True {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vmware.vnc_keymap              = en-us {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vmware.vnc_port                = 5900 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vmware.vnc_port_total          = 10000 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vnc.auth_schemes               = ['none'] {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vnc.enabled                    = True {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vnc.novncproxy_base_url        = http://10.0.0.210:6080/vnc_lite.html {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vnc.novncproxy_host            = 0.0.0.0 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vnc.novncproxy_port            = 6080 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vnc.server_listen              = 0.0.0.0 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vnc.server_proxyclient_address = 10.0.0.210 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vnc.vencrypt_ca_certs          = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vnc.vencrypt_client_cert       = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vnc.vencrypt_client_key        = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] workarounds.disable_compute_service_check_for_ffu = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] workarounds.disable_fallback_pcpu_query = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] workarounds.disable_group_policy_check_upcall = True {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] workarounds.disable_libvirt_livesnapshot = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] workarounds.disable_rootwrap   = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] workarounds.enable_numa_live_migration = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] workarounds.enable_qemu_monitor_announce_self = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] workarounds.handle_virt_lifecycle_events = True {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] workarounds.libvirt_disable_apic = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] workarounds.never_download_image_if_on_rbd = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] workarounds.qemu_monitor_announce_self_count = 3 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] workarounds.qemu_monitor_announce_self_interval = 1 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] workarounds.skip_cpu_compare_at_startup = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] workarounds.skip_cpu_compare_on_dest = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] workarounds.skip_reserve_in_use_ironic_nodes = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] wsgi.api_paste_config          = /etc/nova/api-paste.ini {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] wsgi.client_socket_timeout     = 900 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] wsgi.default_pool_size         = 1000 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] wsgi.keep_alive                = True {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] wsgi.max_header_line           = 16384 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] wsgi.secure_proxy_ssl_header   = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] wsgi.ssl_ca_file               = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] wsgi.ssl_cert_file             = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] wsgi.ssl_key_file              = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] wsgi.tcp_keepidle              = 600 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] wsgi.wsgi_log_format           = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] zvm.ca_file                    = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] zvm.cloud_connector_url        = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] zvm.image_tmp_path             = /opt/stack/data/nova/images {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] zvm.reachable_timeout          = 300 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_policy.enforce_new_defaults = True {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_policy.enforce_scope      = True {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_policy.policy_default_rule = default {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_policy.policy_dirs        = ['policy.d'] {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_policy.policy_file        = policy.yaml {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_policy.remote_ssl_client_crt_file = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_policy.remote_ssl_client_key_file = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] profiler.connection_string     = messaging:// {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] profiler.enabled               = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] profiler.es_doc_type           = notification {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] profiler.es_scroll_size        = 10000 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] profiler.es_scroll_time        = 2m {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] profiler.filter_error_trace    = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] profiler.hmac_keys             = **** {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] profiler.sentinel_service_name = mymaster {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] profiler.socket_timeout        = 0.1 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] profiler.trace_requests        = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] profiler.trace_sqlalchemy      = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] profiler_jaeger.process_tags   = {} {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] profiler_jaeger.service_name_prefix = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] profiler_otlp.service_name_prefix = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] remote_debug.host              = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] remote_debug.port              = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_messaging_rabbit.heartbeat_rate = 2 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_messaging_rabbit.kombu_compression = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_messaging_rabbit.rabbit_quorum_max_memory_bytes = 0 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_messaging_rabbit.rabbit_quorum_max_memory_length = 0 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_messaging_rabbit.ssl      = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_messaging_rabbit.ssl_ca_file =  {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_messaging_rabbit.ssl_cert_file =  {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_messaging_rabbit.ssl_key_file =  {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_messaging_rabbit.ssl_version =  {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_messaging_notifications.retry = -1 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_messaging_notifications.topics = ['notifications'] {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_messaging_notifications.transport_url = **** {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_limit.auth_section        = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_limit.auth_type           = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_limit.cafile              = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_limit.certfile            = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_limit.collect_timing      = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_limit.connect_retries     = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_limit.connect_retry_delay = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_limit.endpoint_id         = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_limit.endpoint_override   = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_limit.insecure            = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_limit.keyfile             = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_limit.max_version         = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_limit.min_version         = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_limit.region_name         = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_limit.service_name        = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_limit.service_type        = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_limit.split_loggers       = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_limit.status_code_retries = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_limit.status_code_retry_delay = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_limit.timeout             = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_limit.valid_interfaces    = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_limit.version             = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_reports.file_event_handler = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_reports.file_event_handler_interval = 1 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] oslo_reports.log_dir           = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vif_plug_linux_bridge_privileged.capabilities = [12] {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vif_plug_linux_bridge_privileged.group = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vif_plug_linux_bridge_privileged.helper_command = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vif_plug_linux_bridge_privileged.logger_name = oslo_privsep.daemon {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vif_plug_linux_bridge_privileged.thread_pool_size = 12 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vif_plug_linux_bridge_privileged.user = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vif_plug_ovs_privileged.capabilities = [12, 1] {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vif_plug_ovs_privileged.group  = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vif_plug_ovs_privileged.helper_command = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vif_plug_ovs_privileged.logger_name = oslo_privsep.daemon {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vif_plug_ovs_privileged.thread_pool_size = 12 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] vif_plug_ovs_privileged.user   = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] os_vif_linux_bridge.flat_interface = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] os_vif_linux_bridge.forward_bridge_interface = ['all'] {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] os_vif_linux_bridge.iptables_bottom_regex =  {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] os_vif_linux_bridge.iptables_drop_action = DROP {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] os_vif_linux_bridge.iptables_top_regex =  {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] os_vif_linux_bridge.network_device_mtu = 1500 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] os_vif_linux_bridge.use_ipv6   = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] os_vif_linux_bridge.vlan_interface = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] os_vif_ovs.isolate_vif         = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] os_vif_ovs.network_device_mtu  = 1500 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] os_vif_ovs.ovs_vsctl_timeout   = 120 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] os_vif_ovs.ovsdb_connection    = tcp:127.0.0.1:6640 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] os_vif_ovs.ovsdb_interface     = native {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] os_vif_ovs.per_port_bridge     = False {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] os_brick.lock_path             = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] privsep_osbrick.capabilities   = [21] {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] privsep_osbrick.group          = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] privsep_osbrick.helper_command = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] privsep_osbrick.logger_name    = os_brick.privileged {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] privsep_osbrick.thread_pool_size = 12 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] privsep_osbrick.user           = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] nova_sys_admin.capabilities    = [0, 1, 2, 3, 12, 21] {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] nova_sys_admin.group           = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] nova_sys_admin.helper_command  = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] nova_sys_admin.logger_name     = oslo_privsep.daemon {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] nova_sys_admin.thread_pool_size = 12 {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] nova_sys_admin.user            = None {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG oslo_service.service [None req-77f187b3-8db8-46f1-8d36-4bb7eeb55c35 None None] ******************************************************************************** {{(pid=72351) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2613}}
Aug 09 02:12:47 user nova-compute[72351]: INFO nova.service [-] Starting compute node (version 0.0.0)
Aug 09 02:12:47 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Starting native event thread {{(pid=72351) _init_events /opt/stack/nova/nova/virt/libvirt/host.py:492}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Starting green dispatch thread {{(pid=72351) _init_events /opt/stack/nova/nova/virt/libvirt/host.py:498}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Starting connection event dispatch thread {{(pid=72351) initialize /opt/stack/nova/nova/virt/libvirt/host.py:620}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Connecting to libvirt: qemu:///system {{(pid=72351) _get_new_connection /opt/stack/nova/nova/virt/libvirt/host.py:503}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Registering for lifecycle events <nova.virt.libvirt.host.Host object at 0x7fd0032f4b20> {{(pid=72351) _get_new_connection /opt/stack/nova/nova/virt/libvirt/host.py:509}}
Aug 09 02:12:47 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Registering for connection events: <nova.virt.libvirt.host.Host object at 0x7fd0032f4b20> {{(pid=72351) _get_new_connection /opt/stack/nova/nova/virt/libvirt/host.py:530}}
Aug 09 02:12:47 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Connection event '1' reason 'None'
Aug 09 02:12:47 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Cannot update service status on host "user" since it is not registered.: nova.exception_Remote.ComputeHostNotFound_Remote: Compute host user could not be found.
Aug 09 02:12:47 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.mount [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Initialising _HostMountState generation 0 {{(pid=72351) host_up /opt/stack/nova/nova/virt/libvirt/volume/mount.py:130}}
Aug 09 02:12:54 user nova-compute[72351]: INFO nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Libvirt host capabilities <capabilities>
Aug 09 02:12:54 user nova-compute[72351]:   <host>
Aug 09 02:12:54 user nova-compute[72351]:     <uuid>e20c3142-5af9-7467-ecd8-70b2e4a210d6</uuid>
Aug 09 02:12:54 user nova-compute[72351]:     <cpu>
Aug 09 02:12:54 user nova-compute[72351]:       <arch>x86_64</arch>
Aug 09 02:12:54 user nova-compute[72351]:       <model>IvyBridge-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <vendor>Intel</vendor>
Aug 09 02:12:54 user nova-compute[72351]:       <microcode version='1070'/>
Aug 09 02:12:54 user nova-compute[72351]:       <counter name='tsc' frequency='2094952000' scaling='no'/>
Aug 09 02:12:54 user nova-compute[72351]:       <topology sockets='6' dies='1' cores='1' threads='1'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature name='ss'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature name='vmx'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature name='pcid'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature name='osxsave'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature name='hypervisor'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature name='arat'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature name='tsc_adjust'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature name='md-clear'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature name='stibp'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature name='arch-capabilities'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature name='ssbd'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature name='invtsc'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature name='rsba'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature name='skip-l1dfl-vmentry'/>
Aug 09 02:12:54 user nova-compute[72351]:       <pages unit='KiB' size='4'/>
Aug 09 02:12:54 user nova-compute[72351]:       <pages unit='KiB' size='2048'/>
Aug 09 02:12:54 user nova-compute[72351]:     </cpu>
Aug 09 02:12:54 user nova-compute[72351]:     <power_management>
Aug 09 02:12:54 user nova-compute[72351]:       <suspend_mem/>
Aug 09 02:12:54 user nova-compute[72351]:       <suspend_disk/>
Aug 09 02:12:54 user nova-compute[72351]:       <suspend_hybrid/>
Aug 09 02:12:54 user nova-compute[72351]:     </power_management>
Aug 09 02:12:54 user nova-compute[72351]:     <iommu support='no'/>
Aug 09 02:12:54 user nova-compute[72351]:     <migration_features>
Aug 09 02:12:54 user nova-compute[72351]:       <live/>
Aug 09 02:12:54 user nova-compute[72351]:       <uri_transports>
Aug 09 02:12:54 user nova-compute[72351]:         <uri_transport>tcp</uri_transport>
Aug 09 02:12:54 user nova-compute[72351]:         <uri_transport>rdma</uri_transport>
Aug 09 02:12:54 user nova-compute[72351]:       </uri_transports>
Aug 09 02:12:54 user nova-compute[72351]:     </migration_features>
Aug 09 02:12:54 user nova-compute[72351]:     <topology>
Aug 09 02:12:54 user nova-compute[72351]:       <cells num='2'>
Aug 09 02:12:54 user nova-compute[72351]:         <cell id='0'>
Aug 09 02:12:54 user nova-compute[72351]:           <memory unit='KiB'>8140616</memory>
Aug 09 02:12:54 user nova-compute[72351]:           <pages unit='KiB' size='4'>2035154</pages>
Aug 09 02:12:54 user nova-compute[72351]:           <pages unit='KiB' size='2048'>0</pages>
Aug 09 02:12:54 user nova-compute[72351]:           <distances>
Aug 09 02:12:54 user nova-compute[72351]:             <sibling id='0' value='10'/>
Aug 09 02:12:54 user nova-compute[72351]:             <sibling id='1' value='20'/>
Aug 09 02:12:54 user nova-compute[72351]:           </distances>
Aug 09 02:12:54 user nova-compute[72351]:           <cpus num='6'>
Aug 09 02:12:54 user nova-compute[72351]:             <cpu id='0' socket_id='0' die_id='0' core_id='0' siblings='0'/>
Aug 09 02:12:54 user nova-compute[72351]:             <cpu id='1' socket_id='2' die_id='0' core_id='0' siblings='1'/>
Aug 09 02:12:54 user nova-compute[72351]:             <cpu id='2' socket_id='4' die_id='0' core_id='0' siblings='2'/>
Aug 09 02:12:54 user nova-compute[72351]:             <cpu id='3' socket_id='6' die_id='0' core_id='0' siblings='3'/>
Aug 09 02:12:54 user nova-compute[72351]:             <cpu id='4' socket_id='8' die_id='0' core_id='0' siblings='4'/>
Aug 09 02:12:54 user nova-compute[72351]:             <cpu id='5' socket_id='10' die_id='0' core_id='0' siblings='5'/>
Aug 09 02:12:54 user nova-compute[72351]:           </cpus>
Aug 09 02:12:54 user nova-compute[72351]:         </cell>
Aug 09 02:12:54 user nova-compute[72351]:         <cell id='1'>
Aug 09 02:12:54 user nova-compute[72351]:           <memory unit='KiB'>8255068</memory>
Aug 09 02:12:54 user nova-compute[72351]:           <pages unit='KiB' size='4'>2063767</pages>
Aug 09 02:12:54 user nova-compute[72351]:           <pages unit='KiB' size='2048'>0</pages>
Aug 09 02:12:54 user nova-compute[72351]:           <distances>
Aug 09 02:12:54 user nova-compute[72351]:             <sibling id='0' value='20'/>
Aug 09 02:12:54 user nova-compute[72351]:             <sibling id='1' value='10'/>
Aug 09 02:12:54 user nova-compute[72351]:           </distances>
Aug 09 02:12:54 user nova-compute[72351]:           <cpus num='6'>
Aug 09 02:12:54 user nova-compute[72351]:             <cpu id='6' socket_id='12' die_id='0' core_id='0' siblings='6'/>
Aug 09 02:12:54 user nova-compute[72351]:             <cpu id='7' socket_id='14' die_id='0' core_id='0' siblings='7'/>
Aug 09 02:12:54 user nova-compute[72351]:             <cpu id='8' socket_id='16' die_id='0' core_id='0' siblings='8'/>
Aug 09 02:12:54 user nova-compute[72351]:             <cpu id='9' socket_id='18' die_id='0' core_id='0' siblings='9'/>
Aug 09 02:12:54 user nova-compute[72351]:             <cpu id='10' socket_id='20' die_id='0' core_id='0' siblings='10'/>
Aug 09 02:12:54 user nova-compute[72351]:             <cpu id='11' socket_id='22' die_id='0' core_id='0' siblings='11'/>
Aug 09 02:12:54 user nova-compute[72351]:           </cpus>
Aug 09 02:12:54 user nova-compute[72351]:         </cell>
Aug 09 02:12:54 user nova-compute[72351]:       </cells>
Aug 09 02:12:54 user nova-compute[72351]:     </topology>
Aug 09 02:12:54 user nova-compute[72351]:     <cache>
Aug 09 02:12:54 user nova-compute[72351]:       <bank id='0' level='3' type='both' size='15' unit='MiB' cpus='0'/>
Aug 09 02:12:54 user nova-compute[72351]:       <bank id='2' level='3' type='both' size='15' unit='MiB' cpus='1'/>
Aug 09 02:12:54 user nova-compute[72351]:       <bank id='4' level='3' type='both' size='15' unit='MiB' cpus='2'/>
Aug 09 02:12:54 user nova-compute[72351]:       <bank id='6' level='3' type='both' size='15' unit='MiB' cpus='3'/>
Aug 09 02:12:54 user nova-compute[72351]:       <bank id='8' level='3' type='both' size='15' unit='MiB' cpus='4'/>
Aug 09 02:12:54 user nova-compute[72351]:       <bank id='10' level='3' type='both' size='15' unit='MiB' cpus='5'/>
Aug 09 02:12:54 user nova-compute[72351]:       <bank id='12' level='3' type='both' size='15' unit='MiB' cpus='6'/>
Aug 09 02:12:54 user nova-compute[72351]:       <bank id='14' level='3' type='both' size='15' unit='MiB' cpus='7'/>
Aug 09 02:12:54 user nova-compute[72351]:       <bank id='16' level='3' type='both' size='15' unit='MiB' cpus='8'/>
Aug 09 02:12:54 user nova-compute[72351]:       <bank id='18' level='3' type='both' size='15' unit='MiB' cpus='9'/>
Aug 09 02:12:54 user nova-compute[72351]:       <bank id='20' level='3' type='both' size='15' unit='MiB' cpus='10'/>
Aug 09 02:12:54 user nova-compute[72351]:       <bank id='22' level='3' type='both' size='15' unit='MiB' cpus='11'/>
Aug 09 02:12:54 user nova-compute[72351]:     </cache>
Aug 09 02:12:54 user nova-compute[72351]:     <secmodel>
Aug 09 02:12:54 user nova-compute[72351]:       <model>apparmor</model>
Aug 09 02:12:54 user nova-compute[72351]:       <doi>0</doi>
Aug 09 02:12:54 user nova-compute[72351]:     </secmodel>
Aug 09 02:12:54 user nova-compute[72351]:     <secmodel>
Aug 09 02:12:54 user nova-compute[72351]:       <model>dac</model>
Aug 09 02:12:54 user nova-compute[72351]:       <doi>0</doi>
Aug 09 02:12:54 user nova-compute[72351]:       <baselabel type='kvm'>+64055:+108</baselabel>
Aug 09 02:12:54 user nova-compute[72351]:       <baselabel type='qemu'>+64055:+108</baselabel>
Aug 09 02:12:54 user nova-compute[72351]:     </secmodel>
Aug 09 02:12:54 user nova-compute[72351]:   </host>
Aug 09 02:12:54 user nova-compute[72351]:   <guest>
Aug 09 02:12:54 user nova-compute[72351]:     <os_type>hvm</os_type>
Aug 09 02:12:54 user nova-compute[72351]:     <arch name='alpha'>
Aug 09 02:12:54 user nova-compute[72351]:       <wordsize>64</wordsize>
Aug 09 02:12:54 user nova-compute[72351]:       <emulator>/usr/bin/qemu-system-alpha</emulator>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='4'>clipper</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <domain type='qemu'/>
Aug 09 02:12:54 user nova-compute[72351]:     </arch>
Aug 09 02:12:54 user nova-compute[72351]:     <features>
Aug 09 02:12:54 user nova-compute[72351]:       <cpuselection/>
Aug 09 02:12:54 user nova-compute[72351]:       <deviceboot/>
Aug 09 02:12:54 user nova-compute[72351]:       <disksnapshot default='on' toggle='no'/>
Aug 09 02:12:54 user nova-compute[72351]:     </features>
Aug 09 02:12:54 user nova-compute[72351]:   </guest>
Aug 09 02:12:54 user nova-compute[72351]:   <guest>
Aug 09 02:12:54 user nova-compute[72351]:     <os_type>hvm</os_type>
Aug 09 02:12:54 user nova-compute[72351]:     <arch name='armv6l'>
Aug 09 02:12:54 user nova-compute[72351]:       <wordsize>32</wordsize>
Aug 09 02:12:54 user nova-compute[72351]:       <emulator>/usr/bin/qemu-system-arm</emulator>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>integratorcp</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2'>ast2600-evb</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>borzoi</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>spitz</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>virt-2.7</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2'>nuri</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2'>mcimx7d-sabre</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>romulus-bmc</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='512'>virt-3.0</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='512'>virt-5.0</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2'>npcm750-evb</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>virt-2.10</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2'>rainier-bmc</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>mps3-an547</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2'>musca-b1</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='4'>realview-pbx-a9</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>versatileab</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>kzm</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>virt-2.8</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2'>musca-a</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='512'>virt-3.1</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>mcimx6ul-evk</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='512'>virt-5.1</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2'>smdkc210</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>sx1</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>virt-2.11</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>imx25-pdk</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>stm32vldiscovery</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>virt-2.9</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='4'>orangepi-pc</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>quanta-q71l-bmc</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>z2</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='512'>virt-5.2</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>xilinx-zynq-a9</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>tosa</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>mps2-an500</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>virt-2.12</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2'>mps2-an521</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='4'>sabrelite</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>mps2-an511</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>canon-a1100</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>realview-eb</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2'>quanta-gbs-bmc</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>emcraft-sf2</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>realview-pb-a8</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='512'>virt-4.0</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>raspi1ap</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>palmetto-bmc</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>sx1-v1</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>n810</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>g220a-bmc</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>n800</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2'>tacoma-bmc</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='512'>virt-4.1</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2'>quanta-gsj</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>versatilepb</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>terrier</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>mainstone</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='4'>realview-eb-mpcore</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>supermicrox11-bmc</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='512'>virt-4.2</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>witherspoon-bmc</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2'>mps3-an524</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1' deprecated='yes'>swift-bmc</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2'>kudo-bmc</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='4'>vexpress-a9</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='4'>midway</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>musicpal</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>lm3s811evb</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>lm3s6965evb</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>microbit</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>mps2-an505</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>mps2-an385</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='512'>virt-6.0</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>cubieboard</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>verdex</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>netduino2</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>mps2-an386</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='512'>virt-6.1</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='4'>raspi2b</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='4'>vexpress-a15</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2'>fuji-bmc</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='512'>virt-6.2</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine canonical='virt-6.2' maxCpus='512'>virt</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>sonorapass-bmc</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>cheetah</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>virt-2.6</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>ast2500-evb</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='4'>highbank</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>akita</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>connex</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>netduinoplus2</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>collie</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>raspi0</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>fp5280g2-bmc</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <domain type='qemu'/>
Aug 09 02:12:54 user nova-compute[72351]:     </arch>
Aug 09 02:12:54 user nova-compute[72351]:     <features>
Aug 09 02:12:54 user nova-compute[72351]:       <cpuselection/>
Aug 09 02:12:54 user nova-compute[72351]:       <deviceboot/>
Aug 09 02:12:54 user nova-compute[72351]:       <disksnapshot default='on' toggle='no'/>
Aug 09 02:12:54 user nova-compute[72351]:     </features>
Aug 09 02:12:54 user nova-compute[72351]:   </guest>
Aug 09 02:12:54 user nova-compute[72351]:   <guest>
Aug 09 02:12:54 user nova-compute[72351]:     <os_type>hvm</os_type>
Aug 09 02:12:54 user nova-compute[72351]:     <arch name='armv7l'>
Aug 09 02:12:54 user nova-compute[72351]:       <wordsize>32</wordsize>
Aug 09 02:12:54 user nova-compute[72351]:       <emulator>/usr/bin/qemu-system-arm</emulator>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>integratorcp</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2'>ast2600-evb</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>borzoi</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>spitz</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>virt-2.7</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2'>nuri</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2'>mcimx7d-sabre</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>romulus-bmc</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='512'>virt-3.0</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='512'>virt-5.0</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2'>npcm750-evb</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>virt-2.10</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2'>rainier-bmc</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>mps3-an547</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2'>musca-b1</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='4'>realview-pbx-a9</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>versatileab</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>kzm</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>virt-2.8</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2'>musca-a</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='512'>virt-3.1</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>mcimx6ul-evk</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='512'>virt-5.1</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2'>smdkc210</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>sx1</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>virt-2.11</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>imx25-pdk</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>stm32vldiscovery</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>virt-2.9</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='4'>orangepi-pc</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>quanta-q71l-bmc</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>z2</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='512'>virt-5.2</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>xilinx-zynq-a9</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>tosa</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>mps2-an500</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>virt-2.12</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2'>mps2-an521</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='4'>sabrelite</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>mps2-an511</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>canon-a1100</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>realview-eb</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2'>quanta-gbs-bmc</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>emcraft-sf2</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>realview-pb-a8</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='512'>virt-4.0</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>raspi1ap</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>palmetto-bmc</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>sx1-v1</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>n810</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>g220a-bmc</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>n800</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2'>tacoma-bmc</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='512'>virt-4.1</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2'>quanta-gsj</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>versatilepb</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>terrier</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>mainstone</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='4'>realview-eb-mpcore</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>supermicrox11-bmc</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='512'>virt-4.2</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>witherspoon-bmc</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2'>mps3-an524</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1' deprecated='yes'>swift-bmc</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2'>kudo-bmc</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='4'>vexpress-a9</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='4'>midway</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>musicpal</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>lm3s811evb</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>lm3s6965evb</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>microbit</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>mps2-an505</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>mps2-an385</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='512'>virt-6.0</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>cubieboard</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>verdex</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>netduino2</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>mps2-an386</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='512'>virt-6.1</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='4'>raspi2b</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='4'>vexpress-a15</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2'>fuji-bmc</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='512'>virt-6.2</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine canonical='virt-6.2' maxCpus='512'>virt</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>sonorapass-bmc</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>cheetah</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>virt-2.6</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>ast2500-evb</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='4'>highbank</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>akita</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>connex</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>netduinoplus2</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>collie</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>raspi0</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>fp5280g2-bmc</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <domain type='qemu'/>
Aug 09 02:12:54 user nova-compute[72351]:     </arch>
Aug 09 02:12:54 user nova-compute[72351]:     <features>
Aug 09 02:12:54 user nova-compute[72351]:       <cpuselection/>
Aug 09 02:12:54 user nova-compute[72351]:       <deviceboot/>
Aug 09 02:12:54 user nova-compute[72351]:       <disksnapshot default='on' toggle='no'/>
Aug 09 02:12:54 user nova-compute[72351]:     </features>
Aug 09 02:12:54 user nova-compute[72351]:   </guest>
Aug 09 02:12:54 user nova-compute[72351]:   <guest>
Aug 09 02:12:54 user nova-compute[72351]:     <os_type>hvm</os_type>
Aug 09 02:12:54 user nova-compute[72351]:     <arch name='aarch64'>
Aug 09 02:12:54 user nova-compute[72351]:       <wordsize>64</wordsize>
Aug 09 02:12:54 user nova-compute[72351]:       <emulator>/usr/bin/qemu-system-aarch64</emulator>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>integratorcp</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2'>ast2600-evb</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>borzoi</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>spitz</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>virt-2.7</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2'>nuri</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2'>mcimx7d-sabre</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>romulus-bmc</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='512'>virt-3.0</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='512'>virt-5.0</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2'>npcm750-evb</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>virt-2.10</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2'>rainier-bmc</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>mps3-an547</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>virt-2.8</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2'>musca-b1</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='4'>realview-pbx-a9</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>versatileab</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>kzm</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2'>musca-a</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='512'>virt-3.1</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>mcimx6ul-evk</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='512'>virt-5.1</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2'>smdkc210</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>sx1</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>virt-2.11</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>imx25-pdk</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>stm32vldiscovery</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>virt-2.9</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='4'>orangepi-pc</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>quanta-q71l-bmc</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>z2</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='512'>virt-5.2</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>xilinx-zynq-a9</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='6'>xlnx-zcu102</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>tosa</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>mps2-an500</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>virt-2.12</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2'>mps2-an521</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='4'>sabrelite</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>mps2-an511</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>canon-a1100</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>realview-eb</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2'>quanta-gbs-bmc</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>emcraft-sf2</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>realview-pb-a8</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='512'>sbsa-ref</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='512'>virt-4.0</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>raspi1ap</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>palmetto-bmc</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>sx1-v1</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>n810</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>g220a-bmc</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>n800</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2'>tacoma-bmc</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='512'>virt-4.1</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2'>quanta-gsj</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>versatilepb</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>terrier</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>mainstone</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='4'>realview-eb-mpcore</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>supermicrox11-bmc</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='512'>virt-4.2</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>witherspoon-bmc</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2'>mps3-an524</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1' deprecated='yes'>swift-bmc</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2'>kudo-bmc</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='4'>vexpress-a9</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='4'>midway</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>musicpal</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>lm3s811evb</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>lm3s6965evb</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>microbit</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>mps2-an505</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>mps2-an385</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='512'>virt-6.0</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='4'>raspi3ap</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>cubieboard</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>verdex</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>netduino2</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2'>xlnx-versal-virt</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>mps2-an386</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='512'>virt-6.1</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='4'>raspi3b</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='4'>raspi2b</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='4'>vexpress-a15</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2'>fuji-bmc</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='512'>virt-6.2</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine canonical='virt-6.2' maxCpus='512'>virt</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>sonorapass-bmc</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>cheetah</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>virt-2.6</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>ast2500-evb</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='4'>highbank</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>akita</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>connex</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>netduinoplus2</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>collie</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>raspi0</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>fp5280g2-bmc</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <domain type='qemu'/>
Aug 09 02:12:54 user nova-compute[72351]:     </arch>
Aug 09 02:12:54 user nova-compute[72351]:     <features>
Aug 09 02:12:54 user nova-compute[72351]:       <acpi default='on' toggle='yes'/>
Aug 09 02:12:54 user nova-compute[72351]:       <cpuselection/>
Aug 09 02:12:54 user nova-compute[72351]:       <deviceboot/>
Aug 09 02:12:54 user nova-compute[72351]:       <disksnapshot default='on' toggle='no'/>
Aug 09 02:12:54 user nova-compute[72351]:     </features>
Aug 09 02:12:54 user nova-compute[72351]:   </guest>
Aug 09 02:12:54 user nova-compute[72351]:   <guest>
Aug 09 02:12:54 user nova-compute[72351]:     <os_type>hvm</os_type>
Aug 09 02:12:54 user nova-compute[72351]:     <arch name='cris'>
Aug 09 02:12:54 user nova-compute[72351]:       <wordsize>32</wordsize>
Aug 09 02:12:54 user nova-compute[72351]:       <emulator>/usr/bin/qemu-system-cris</emulator>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>axis-dev88</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <domain type='qemu'/>
Aug 09 02:12:54 user nova-compute[72351]:     </arch>
Aug 09 02:12:54 user nova-compute[72351]:     <features>
Aug 09 02:12:54 user nova-compute[72351]:       <cpuselection/>
Aug 09 02:12:54 user nova-compute[72351]:       <deviceboot/>
Aug 09 02:12:54 user nova-compute[72351]:       <disksnapshot default='on' toggle='no'/>
Aug 09 02:12:54 user nova-compute[72351]:     </features>
Aug 09 02:12:54 user nova-compute[72351]:   </guest>
Aug 09 02:12:54 user nova-compute[72351]:   <guest>
Aug 09 02:12:54 user nova-compute[72351]:     <os_type>hvm</os_type>
Aug 09 02:12:54 user nova-compute[72351]:     <arch name='i686'>
Aug 09 02:12:54 user nova-compute[72351]:       <wordsize>32</wordsize>
Aug 09 02:12:54 user nova-compute[72351]:       <emulator>/usr/bin/qemu-system-i386</emulator>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-jammy</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine canonical='pc-i440fx-jammy' maxCpus='255'>ubuntu</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-impish-hpb</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-5.2</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-2.12</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-2.0</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-xenial</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-6.2</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine canonical='pc-i440fx-6.2' maxCpus='255'>pc</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-4.2</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-2.5</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-4.2</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-focal</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-hirsute</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-q35-xenial</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-jammy-hpb</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-5.2</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-1.5</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-q35-2.7</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-eoan-hpb</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-zesty</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-disco-hpb</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-groovy</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-groovy</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-artful</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-2.2</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-trusty</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-eoan-hpb</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-focal-hpb</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-bionic-hpb</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-artful</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-2.7</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-6.1</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-yakkety</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-q35-2.4</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-cosmic-hpb</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-2.10</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>x-remote</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-5.1</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-1.7</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-2.9</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-2.11</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-3.1</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-6.1</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-4.1</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-jammy</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine canonical='pc-q35-jammy' maxCpus='288'>ubuntu-q35</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-2.4</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-4.1</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-eoan</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-jammy-hpb</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-5.1</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-2.9</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-bionic-hpb</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>isapc</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-1.4</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-cosmic</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-q35-2.6</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-3.1</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-bionic</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-disco-hpb</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-cosmic</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-2.12</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-bionic</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-groovy-hpb</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-disco</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-cosmic-hpb</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-2.1</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-wily</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-impish</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-6.0</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-impish</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-2.6</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-impish-hpb</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-hirsute</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-4.0.1</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-hirsute-hpb</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-1.6</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-5.0</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-2.8</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-2.10</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-3.0</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-6.0</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-zesty</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-4.0</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-focal</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>microvm</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-2.3</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-focal-hpb</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-disco</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-4.0</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-groovy-hpb</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-hirsute-hpb</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-5.0</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-6.2</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine canonical='pc-q35-6.2' maxCpus='288'>q35</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-2.8</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-eoan</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-q35-2.5</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-3.0</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-q35-yakkety</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-2.11</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <domain type='qemu'/>
Aug 09 02:12:54 user nova-compute[72351]:       <domain type='kvm'/>
Aug 09 02:12:54 user nova-compute[72351]:     </arch>
Aug 09 02:12:54 user nova-compute[72351]:     <features>
Aug 09 02:12:54 user nova-compute[72351]:       <pae/>
Aug 09 02:12:54 user nova-compute[72351]:       <nonpae/>
Aug 09 02:12:54 user nova-compute[72351]:       <acpi default='on' toggle='yes'/>
Aug 09 02:12:54 user nova-compute[72351]:       <apic default='on' toggle='no'/>
Aug 09 02:12:54 user nova-compute[72351]:       <cpuselection/>
Aug 09 02:12:54 user nova-compute[72351]:       <deviceboot/>
Aug 09 02:12:54 user nova-compute[72351]:       <disksnapshot default='on' toggle='no'/>
Aug 09 02:12:54 user nova-compute[72351]:     </features>
Aug 09 02:12:54 user nova-compute[72351]:   </guest>
Aug 09 02:12:54 user nova-compute[72351]:   <guest>
Aug 09 02:12:54 user nova-compute[72351]:     <os_type>hvm</os_type>
Aug 09 02:12:54 user nova-compute[72351]:     <arch name='m68k'>
Aug 09 02:12:54 user nova-compute[72351]:       <wordsize>32</wordsize>
Aug 09 02:12:54 user nova-compute[72351]:       <emulator>/usr/bin/qemu-system-m68k</emulator>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>mcf5208evb</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>an5206</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>virt-6.0</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>q800</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>virt-6.2</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine canonical='virt-6.2' maxCpus='1'>virt</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>next-cube</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>virt-6.1</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <domain type='qemu'/>
Aug 09 02:12:54 user nova-compute[72351]:     </arch>
Aug 09 02:12:54 user nova-compute[72351]:     <features>
Aug 09 02:12:54 user nova-compute[72351]:       <cpuselection/>
Aug 09 02:12:54 user nova-compute[72351]:       <deviceboot/>
Aug 09 02:12:54 user nova-compute[72351]:       <disksnapshot default='on' toggle='no'/>
Aug 09 02:12:54 user nova-compute[72351]:     </features>
Aug 09 02:12:54 user nova-compute[72351]:   </guest>
Aug 09 02:12:54 user nova-compute[72351]:   <guest>
Aug 09 02:12:54 user nova-compute[72351]:     <os_type>hvm</os_type>
Aug 09 02:12:54 user nova-compute[72351]:     <arch name='microblaze'>
Aug 09 02:12:54 user nova-compute[72351]:       <wordsize>32</wordsize>
Aug 09 02:12:54 user nova-compute[72351]:       <emulator>/usr/bin/qemu-system-microblaze</emulator>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>petalogix-s3adsp1800</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>petalogix-ml605</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>xlnx-zynqmp-pmu</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <domain type='qemu'/>
Aug 09 02:12:54 user nova-compute[72351]:     </arch>
Aug 09 02:12:54 user nova-compute[72351]:     <features>
Aug 09 02:12:54 user nova-compute[72351]:       <cpuselection/>
Aug 09 02:12:54 user nova-compute[72351]:       <deviceboot/>
Aug 09 02:12:54 user nova-compute[72351]:       <disksnapshot default='on' toggle='no'/>
Aug 09 02:12:54 user nova-compute[72351]:     </features>
Aug 09 02:12:54 user nova-compute[72351]:   </guest>
Aug 09 02:12:54 user nova-compute[72351]:   <guest>
Aug 09 02:12:54 user nova-compute[72351]:     <os_type>hvm</os_type>
Aug 09 02:12:54 user nova-compute[72351]:     <arch name='microblazeel'>
Aug 09 02:12:54 user nova-compute[72351]:       <wordsize>32</wordsize>
Aug 09 02:12:54 user nova-compute[72351]:       <emulator>/usr/bin/qemu-system-microblazeel</emulator>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>petalogix-s3adsp1800</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>petalogix-ml605</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>xlnx-zynqmp-pmu</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <domain type='qemu'/>
Aug 09 02:12:54 user nova-compute[72351]:     </arch>
Aug 09 02:12:54 user nova-compute[72351]:     <features>
Aug 09 02:12:54 user nova-compute[72351]:       <cpuselection/>
Aug 09 02:12:54 user nova-compute[72351]:       <deviceboot/>
Aug 09 02:12:54 user nova-compute[72351]:       <disksnapshot default='on' toggle='no'/>
Aug 09 02:12:54 user nova-compute[72351]:     </features>
Aug 09 02:12:54 user nova-compute[72351]:   </guest>
Aug 09 02:12:54 user nova-compute[72351]:   <guest>
Aug 09 02:12:54 user nova-compute[72351]:     <os_type>hvm</os_type>
Aug 09 02:12:54 user nova-compute[72351]:     <arch name='mips'>
Aug 09 02:12:54 user nova-compute[72351]:       <wordsize>32</wordsize>
Aug 09 02:12:54 user nova-compute[72351]:       <emulator>/usr/bin/qemu-system-mips</emulator>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='16'>malta</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>mipssim</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <domain type='qemu'/>
Aug 09 02:12:54 user nova-compute[72351]:     </arch>
Aug 09 02:12:54 user nova-compute[72351]:     <features>
Aug 09 02:12:54 user nova-compute[72351]:       <cpuselection/>
Aug 09 02:12:54 user nova-compute[72351]:       <deviceboot/>
Aug 09 02:12:54 user nova-compute[72351]:       <disksnapshot default='on' toggle='no'/>
Aug 09 02:12:54 user nova-compute[72351]:     </features>
Aug 09 02:12:54 user nova-compute[72351]:   </guest>
Aug 09 02:12:54 user nova-compute[72351]:   <guest>
Aug 09 02:12:54 user nova-compute[72351]:     <os_type>hvm</os_type>
Aug 09 02:12:54 user nova-compute[72351]:     <arch name='mipsel'>
Aug 09 02:12:54 user nova-compute[72351]:       <wordsize>32</wordsize>
Aug 09 02:12:54 user nova-compute[72351]:       <emulator>/usr/bin/qemu-system-mipsel</emulator>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='16'>malta</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>mipssim</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <domain type='qemu'/>
Aug 09 02:12:54 user nova-compute[72351]:     </arch>
Aug 09 02:12:54 user nova-compute[72351]:     <features>
Aug 09 02:12:54 user nova-compute[72351]:       <cpuselection/>
Aug 09 02:12:54 user nova-compute[72351]:       <deviceboot/>
Aug 09 02:12:54 user nova-compute[72351]:       <disksnapshot default='on' toggle='no'/>
Aug 09 02:12:54 user nova-compute[72351]:     </features>
Aug 09 02:12:54 user nova-compute[72351]:   </guest>
Aug 09 02:12:54 user nova-compute[72351]:   <guest>
Aug 09 02:12:54 user nova-compute[72351]:     <os_type>hvm</os_type>
Aug 09 02:12:54 user nova-compute[72351]:     <arch name='mips64'>
Aug 09 02:12:54 user nova-compute[72351]:       <wordsize>64</wordsize>
Aug 09 02:12:54 user nova-compute[72351]:       <emulator>/usr/bin/qemu-system-mips64</emulator>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='16'>malta</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>mipssim</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>pica61</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>magnum</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <domain type='qemu'/>
Aug 09 02:12:54 user nova-compute[72351]:     </arch>
Aug 09 02:12:54 user nova-compute[72351]:     <features>
Aug 09 02:12:54 user nova-compute[72351]:       <cpuselection/>
Aug 09 02:12:54 user nova-compute[72351]:       <deviceboot/>
Aug 09 02:12:54 user nova-compute[72351]:       <disksnapshot default='on' toggle='no'/>
Aug 09 02:12:54 user nova-compute[72351]:     </features>
Aug 09 02:12:54 user nova-compute[72351]:   </guest>
Aug 09 02:12:54 user nova-compute[72351]:   <guest>
Aug 09 02:12:54 user nova-compute[72351]:     <os_type>hvm</os_type>
Aug 09 02:12:54 user nova-compute[72351]:     <arch name='mips64el'>
Aug 09 02:12:54 user nova-compute[72351]:       <wordsize>64</wordsize>
Aug 09 02:12:54 user nova-compute[72351]:       <emulator>/usr/bin/qemu-system-mips64el</emulator>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='16'>malta</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='16'>loongson3-virt</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>mipssim</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>pica61</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>magnum</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='16'>boston</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>fuloong2e</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <domain type='qemu'/>
Aug 09 02:12:54 user nova-compute[72351]:     </arch>
Aug 09 02:12:54 user nova-compute[72351]:     <features>
Aug 09 02:12:54 user nova-compute[72351]:       <cpuselection/>
Aug 09 02:12:54 user nova-compute[72351]:       <deviceboot/>
Aug 09 02:12:54 user nova-compute[72351]:       <disksnapshot default='on' toggle='no'/>
Aug 09 02:12:54 user nova-compute[72351]:     </features>
Aug 09 02:12:54 user nova-compute[72351]:   </guest>
Aug 09 02:12:54 user nova-compute[72351]:   <guest>
Aug 09 02:12:54 user nova-compute[72351]:     <os_type>hvm</os_type>
Aug 09 02:12:54 user nova-compute[72351]:     <arch name='ppc'>
Aug 09 02:12:54 user nova-compute[72351]:       <wordsize>32</wordsize>
Aug 09 02:12:54 user nova-compute[72351]:       <emulator>/usr/bin/qemu-system-ppc</emulator>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>g3beige</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>virtex-ml507</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>mac99</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='32'>ppce500</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>pegasos2</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>sam460ex</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>bamboo</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>40p</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>ref405ep</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='15'>mpc8544ds</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>taihu</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <domain type='qemu'/>
Aug 09 02:12:54 user nova-compute[72351]:     </arch>
Aug 09 02:12:54 user nova-compute[72351]:     <features>
Aug 09 02:12:54 user nova-compute[72351]:       <cpuselection/>
Aug 09 02:12:54 user nova-compute[72351]:       <deviceboot/>
Aug 09 02:12:54 user nova-compute[72351]:       <disksnapshot default='on' toggle='no'/>
Aug 09 02:12:54 user nova-compute[72351]:     </features>
Aug 09 02:12:54 user nova-compute[72351]:   </guest>
Aug 09 02:12:54 user nova-compute[72351]:   <guest>
Aug 09 02:12:54 user nova-compute[72351]:     <os_type>hvm</os_type>
Aug 09 02:12:54 user nova-compute[72351]:     <arch name='ppc64'>
Aug 09 02:12:54 user nova-compute[72351]:       <wordsize>64</wordsize>
Aug 09 02:12:54 user nova-compute[72351]:       <emulator>/usr/bin/qemu-system-ppc64</emulator>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-jammy</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine canonical='pseries-jammy' maxCpus='2147483647'>pseries</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2048'>powernv9</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine canonical='powernv9' maxCpus='2048'>powernv</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>taihu</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-4.1</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='15'>mpc8544ds</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-6.1</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-2.5</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2048'>powernv10</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-xenial</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-4.2</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-6.2</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-yakkety</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-2.6</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='32'>ppce500</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-bionic-sxxm</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-2.7</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-3.0</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-5.0</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>40p</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-2.8</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>pegasos2</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-hirsute</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-3.1</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-5.1</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-eoan</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-2.9</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-zesty</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>bamboo</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-groovy</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-focal</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>g3beige</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-5.2</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-disco</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-2.12-sxxm</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-2.10</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>virtex-ml507</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-2.11</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-2.1</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-cosmic</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-bionic</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-2.12</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-2.2</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>mac99</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-impish</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-artful</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>sam460ex</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>ref405ep</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-2.3</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2048'>powernv8</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-4.0</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-6.0</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-2.4</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <domain type='qemu'/>
Aug 09 02:12:54 user nova-compute[72351]:     </arch>
Aug 09 02:12:54 user nova-compute[72351]:     <features>
Aug 09 02:12:54 user nova-compute[72351]:       <cpuselection/>
Aug 09 02:12:54 user nova-compute[72351]:       <deviceboot/>
Aug 09 02:12:54 user nova-compute[72351]:       <disksnapshot default='on' toggle='no'/>
Aug 09 02:12:54 user nova-compute[72351]:     </features>
Aug 09 02:12:54 user nova-compute[72351]:   </guest>
Aug 09 02:12:54 user nova-compute[72351]:   <guest>
Aug 09 02:12:54 user nova-compute[72351]:     <os_type>hvm</os_type>
Aug 09 02:12:54 user nova-compute[72351]:     <arch name='ppc64le'>
Aug 09 02:12:54 user nova-compute[72351]:       <wordsize>64</wordsize>
Aug 09 02:12:54 user nova-compute[72351]:       <emulator>/usr/bin/qemu-system-ppc64le</emulator>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-jammy</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine canonical='pseries-jammy' maxCpus='2147483647'>pseries</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2048'>powernv9</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine canonical='powernv9' maxCpus='2048'>powernv</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>taihu</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-4.1</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='15'>mpc8544ds</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-6.1</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-2.5</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2048'>powernv10</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-xenial</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-4.2</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-6.2</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-yakkety</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-2.6</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='32'>ppce500</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-bionic-sxxm</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-2.7</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-3.0</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-5.0</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>40p</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-2.8</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>pegasos2</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-hirsute</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-3.1</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-5.1</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-eoan</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-2.9</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-zesty</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>bamboo</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-groovy</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-focal</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>g3beige</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-5.2</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-disco</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-2.12-sxxm</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-2.10</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>virtex-ml507</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-2.11</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-2.1</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-cosmic</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-bionic</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-2.12</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-2.2</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>mac99</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-impish</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-artful</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>sam460ex</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>ref405ep</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-2.3</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2048'>powernv8</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-4.0</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-6.0</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='2147483647'>pseries-2.4</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <domain type='qemu'/>
Aug 09 02:12:54 user nova-compute[72351]:     </arch>
Aug 09 02:12:54 user nova-compute[72351]:     <features>
Aug 09 02:12:54 user nova-compute[72351]:       <cpuselection/>
Aug 09 02:12:54 user nova-compute[72351]:       <deviceboot/>
Aug 09 02:12:54 user nova-compute[72351]:       <disksnapshot default='on' toggle='no'/>
Aug 09 02:12:54 user nova-compute[72351]:     </features>
Aug 09 02:12:54 user nova-compute[72351]:   </guest>
Aug 09 02:12:54 user nova-compute[72351]:   <guest>
Aug 09 02:12:54 user nova-compute[72351]:     <os_type>hvm</os_type>
Aug 09 02:12:54 user nova-compute[72351]:     <arch name='riscv32'>
Aug 09 02:12:54 user nova-compute[72351]:       <wordsize>32</wordsize>
Aug 09 02:12:54 user nova-compute[72351]:       <emulator>/usr/bin/qemu-system-riscv32</emulator>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='8'>spike</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>opentitan</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='5'>sifive_u</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>sifive_e</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='8'>virt</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <domain type='qemu'/>
Aug 09 02:12:54 user nova-compute[72351]:     </arch>
Aug 09 02:12:54 user nova-compute[72351]:     <features>
Aug 09 02:12:54 user nova-compute[72351]:       <cpuselection/>
Aug 09 02:12:54 user nova-compute[72351]:       <deviceboot/>
Aug 09 02:12:54 user nova-compute[72351]:       <disksnapshot default='on' toggle='no'/>
Aug 09 02:12:54 user nova-compute[72351]:     </features>
Aug 09 02:12:54 user nova-compute[72351]:   </guest>
Aug 09 02:12:54 user nova-compute[72351]:   <guest>
Aug 09 02:12:54 user nova-compute[72351]:     <os_type>hvm</os_type>
Aug 09 02:12:54 user nova-compute[72351]:     <arch name='riscv64'>
Aug 09 02:12:54 user nova-compute[72351]:       <wordsize>64</wordsize>
Aug 09 02:12:54 user nova-compute[72351]:       <emulator>/usr/bin/qemu-system-riscv64</emulator>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='8'>spike</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='5'>microchip-icicle-kit</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='5'>sifive_u</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>shakti_c</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>sifive_e</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='8'>virt</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <domain type='qemu'/>
Aug 09 02:12:54 user nova-compute[72351]:     </arch>
Aug 09 02:12:54 user nova-compute[72351]:     <features>
Aug 09 02:12:54 user nova-compute[72351]:       <cpuselection/>
Aug 09 02:12:54 user nova-compute[72351]:       <deviceboot/>
Aug 09 02:12:54 user nova-compute[72351]:       <disksnapshot default='on' toggle='no'/>
Aug 09 02:12:54 user nova-compute[72351]:     </features>
Aug 09 02:12:54 user nova-compute[72351]:   </guest>
Aug 09 02:12:54 user nova-compute[72351]:   <guest>
Aug 09 02:12:54 user nova-compute[72351]:     <os_type>hvm</os_type>
Aug 09 02:12:54 user nova-compute[72351]:     <arch name='s390x'>
Aug 09 02:12:54 user nova-compute[72351]:       <wordsize>64</wordsize>
Aug 09 02:12:54 user nova-compute[72351]:       <emulator>/usr/bin/qemu-system-s390x</emulator>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='248'>s390-ccw-virtio-jammy</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine canonical='s390-ccw-virtio-jammy' maxCpus='248'>s390-ccw-virtio</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='248'>s390-ccw-virtio-4.0</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='248'>s390-ccw-virtio-5.2</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='248'>s390-ccw-virtio-artful</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='248'>s390-ccw-virtio-3.1</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='248'>s390-ccw-virtio-groovy</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='248'>s390-ccw-virtio-hirsute</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='248'>s390-ccw-virtio-disco</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='248'>s390-ccw-virtio-2.12</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='248'>s390-ccw-virtio-2.6</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='248'>s390-ccw-virtio-yakkety</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='248'>s390-ccw-virtio-eoan</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='248'>s390-ccw-virtio-2.9</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='248'>s390-ccw-virtio-6.0</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='248'>s390-ccw-virtio-5.1</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='248'>s390-ccw-virtio-3.0</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='248'>s390-ccw-virtio-4.2</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='248'>s390-ccw-virtio-2.5</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='248'>s390-ccw-virtio-2.11</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='248'>s390-ccw-virtio-xenial</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='248'>s390-ccw-virtio-focal</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='248'>s390-ccw-virtio-2.8</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='248'>s390-ccw-virtio-impish</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='248'>s390-ccw-virtio-bionic</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='248'>s390-ccw-virtio-5.0</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='248'>s390-ccw-virtio-6.2</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='248'>s390-ccw-virtio-zesty</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='248'>s390-ccw-virtio-4.1</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='248'>s390-ccw-virtio-cosmic</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='248'>s390-ccw-virtio-2.4</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='248'>s390-ccw-virtio-2.10</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='248'>s390-ccw-virtio-2.7</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='248'>s390-ccw-virtio-6.1</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <domain type='qemu'/>
Aug 09 02:12:54 user nova-compute[72351]:     </arch>
Aug 09 02:12:54 user nova-compute[72351]:     <features>
Aug 09 02:12:54 user nova-compute[72351]:       <cpuselection/>
Aug 09 02:12:54 user nova-compute[72351]:       <deviceboot/>
Aug 09 02:12:54 user nova-compute[72351]:       <disksnapshot default='on' toggle='no'/>
Aug 09 02:12:54 user nova-compute[72351]:     </features>
Aug 09 02:12:54 user nova-compute[72351]:   </guest>
Aug 09 02:12:54 user nova-compute[72351]:   <guest>
Aug 09 02:12:54 user nova-compute[72351]:     <os_type>hvm</os_type>
Aug 09 02:12:54 user nova-compute[72351]:     <arch name='sh4'>
Aug 09 02:12:54 user nova-compute[72351]:       <wordsize>32</wordsize>
Aug 09 02:12:54 user nova-compute[72351]:       <emulator>/usr/bin/qemu-system-sh4</emulator>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>shix</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>r2d</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <domain type='qemu'/>
Aug 09 02:12:54 user nova-compute[72351]:     </arch>
Aug 09 02:12:54 user nova-compute[72351]:     <features>
Aug 09 02:12:54 user nova-compute[72351]:       <cpuselection/>
Aug 09 02:12:54 user nova-compute[72351]:       <deviceboot/>
Aug 09 02:12:54 user nova-compute[72351]:       <disksnapshot default='on' toggle='no'/>
Aug 09 02:12:54 user nova-compute[72351]:     </features>
Aug 09 02:12:54 user nova-compute[72351]:   </guest>
Aug 09 02:12:54 user nova-compute[72351]:   <guest>
Aug 09 02:12:54 user nova-compute[72351]:     <os_type>hvm</os_type>
Aug 09 02:12:54 user nova-compute[72351]:     <arch name='sh4eb'>
Aug 09 02:12:54 user nova-compute[72351]:       <wordsize>64</wordsize>
Aug 09 02:12:54 user nova-compute[72351]:       <emulator>/usr/bin/qemu-system-sh4eb</emulator>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>shix</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>r2d</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <domain type='qemu'/>
Aug 09 02:12:54 user nova-compute[72351]:     </arch>
Aug 09 02:12:54 user nova-compute[72351]:     <features>
Aug 09 02:12:54 user nova-compute[72351]:       <cpuselection/>
Aug 09 02:12:54 user nova-compute[72351]:       <deviceboot/>
Aug 09 02:12:54 user nova-compute[72351]:       <disksnapshot default='on' toggle='no'/>
Aug 09 02:12:54 user nova-compute[72351]:     </features>
Aug 09 02:12:54 user nova-compute[72351]:   </guest>
Aug 09 02:12:54 user nova-compute[72351]:   <guest>
Aug 09 02:12:54 user nova-compute[72351]:     <os_type>hvm</os_type>
Aug 09 02:12:54 user nova-compute[72351]:     <arch name='sparc'>
Aug 09 02:12:54 user nova-compute[72351]:       <wordsize>32</wordsize>
Aug 09 02:12:54 user nova-compute[72351]:       <emulator>/usr/bin/qemu-system-sparc</emulator>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>SS-5</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='4'>SS-20</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>LX</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>SPARCClassic</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>leon3_generic</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>SPARCbook</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>SS-4</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='4'>SS-600MP</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='4'>SS-10</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>Voyager</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <domain type='qemu'/>
Aug 09 02:12:54 user nova-compute[72351]:     </arch>
Aug 09 02:12:54 user nova-compute[72351]:     <features>
Aug 09 02:12:54 user nova-compute[72351]:       <cpuselection/>
Aug 09 02:12:54 user nova-compute[72351]:       <deviceboot/>
Aug 09 02:12:54 user nova-compute[72351]:       <disksnapshot default='on' toggle='no'/>
Aug 09 02:12:54 user nova-compute[72351]:     </features>
Aug 09 02:12:54 user nova-compute[72351]:   </guest>
Aug 09 02:12:54 user nova-compute[72351]:   <guest>
Aug 09 02:12:54 user nova-compute[72351]:     <os_type>hvm</os_type>
Aug 09 02:12:54 user nova-compute[72351]:     <arch name='sparc64'>
Aug 09 02:12:54 user nova-compute[72351]:       <wordsize>64</wordsize>
Aug 09 02:12:54 user nova-compute[72351]:       <emulator>/usr/bin/qemu-system-sparc64</emulator>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>sun4u</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>niagara</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>sun4v</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <domain type='qemu'/>
Aug 09 02:12:54 user nova-compute[72351]:     </arch>
Aug 09 02:12:54 user nova-compute[72351]:     <features>
Aug 09 02:12:54 user nova-compute[72351]:       <cpuselection/>
Aug 09 02:12:54 user nova-compute[72351]:       <deviceboot/>
Aug 09 02:12:54 user nova-compute[72351]:       <disksnapshot default='on' toggle='no'/>
Aug 09 02:12:54 user nova-compute[72351]:     </features>
Aug 09 02:12:54 user nova-compute[72351]:   </guest>
Aug 09 02:12:54 user nova-compute[72351]:   <guest>
Aug 09 02:12:54 user nova-compute[72351]:     <os_type>hvm</os_type>
Aug 09 02:12:54 user nova-compute[72351]:     <arch name='x86_64'>
Aug 09 02:12:54 user nova-compute[72351]:       <wordsize>64</wordsize>
Aug 09 02:12:54 user nova-compute[72351]:       <emulator>/usr/bin/qemu-system-x86_64</emulator>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-jammy</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine canonical='pc-i440fx-jammy' maxCpus='255'>ubuntu</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-impish-hpb</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-5.2</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-2.12</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-2.0</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-xenial</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-6.2</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine canonical='pc-i440fx-6.2' maxCpus='255'>pc</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-4.2</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-2.5</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-4.2</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-hirsute</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-focal</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-q35-xenial</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-jammy-hpb</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-5.2</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-1.5</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-q35-2.7</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-eoan-hpb</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-zesty</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-disco-hpb</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-groovy</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-groovy</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-artful</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-trusty</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-2.2</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-focal-hpb</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-eoan-hpb</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-bionic-hpb</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-artful</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-2.7</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-6.1</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-yakkety</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-q35-2.4</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-cosmic-hpb</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-2.10</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>x-remote</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-5.1</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-1.7</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-2.9</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-2.11</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-3.1</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-6.1</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-4.1</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-jammy</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine canonical='pc-q35-jammy' maxCpus='288'>ubuntu-q35</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-2.4</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-4.1</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-eoan</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-jammy-hpb</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-5.1</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-2.9</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-bionic-hpb</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='1'>isapc</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-1.4</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-cosmic</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-q35-2.6</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-3.1</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-bionic</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-disco-hpb</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-cosmic</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-2.12</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-bionic</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-groovy-hpb</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-disco</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-cosmic-hpb</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-2.1</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-wily</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-impish</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-2.6</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-6.0</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-impish</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-impish-hpb</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-hirsute</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-4.0.1</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-hirsute-hpb</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-1.6</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-5.0</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-2.8</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-2.10</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-3.0</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-zesty</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-4.0</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-focal</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>microvm</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-6.0</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-2.3</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-disco</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-focal-hpb</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-4.0</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-groovy-hpb</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-hirsute-hpb</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-5.0</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-2.8</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-6.2</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine canonical='pc-q35-6.2' maxCpus='288'>q35</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-eoan</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-q35-2.5</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-i440fx-3.0</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='255'>pc-q35-yakkety</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='288'>pc-q35-2.11</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <domain type='qemu'/>
Aug 09 02:12:54 user nova-compute[72351]:       <domain type='kvm'/>
Aug 09 02:12:54 user nova-compute[72351]:     </arch>
Aug 09 02:12:54 user nova-compute[72351]:     <features>
Aug 09 02:12:54 user nova-compute[72351]:       <acpi default='on' toggle='yes'/>
Aug 09 02:12:54 user nova-compute[72351]:       <apic default='on' toggle='no'/>
Aug 09 02:12:54 user nova-compute[72351]:       <cpuselection/>
Aug 09 02:12:54 user nova-compute[72351]:       <deviceboot/>
Aug 09 02:12:54 user nova-compute[72351]:       <disksnapshot default='on' toggle='no'/>
Aug 09 02:12:54 user nova-compute[72351]:     </features>
Aug 09 02:12:54 user nova-compute[72351]:   </guest>
Aug 09 02:12:54 user nova-compute[72351]:   <guest>
Aug 09 02:12:54 user nova-compute[72351]:     <os_type>hvm</os_type>
Aug 09 02:12:54 user nova-compute[72351]:     <arch name='xtensa'>
Aug 09 02:12:54 user nova-compute[72351]:       <wordsize>32</wordsize>
Aug 09 02:12:54 user nova-compute[72351]:       <emulator>/usr/bin/qemu-system-xtensa</emulator>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='4'>sim</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='32'>kc705</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='32'>ml605</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='32'>ml605-nommu</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='32'>virt</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='32'>lx60-nommu</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='32'>lx200</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='32'>lx200-nommu</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='32'>lx60</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='32'>kc705-nommu</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <domain type='qemu'/>
Aug 09 02:12:54 user nova-compute[72351]:     </arch>
Aug 09 02:12:54 user nova-compute[72351]:     <features>
Aug 09 02:12:54 user nova-compute[72351]:       <cpuselection/>
Aug 09 02:12:54 user nova-compute[72351]:       <deviceboot/>
Aug 09 02:12:54 user nova-compute[72351]:       <disksnapshot default='on' toggle='no'/>
Aug 09 02:12:54 user nova-compute[72351]:     </features>
Aug 09 02:12:54 user nova-compute[72351]:   </guest>
Aug 09 02:12:54 user nova-compute[72351]:   <guest>
Aug 09 02:12:54 user nova-compute[72351]:     <os_type>hvm</os_type>
Aug 09 02:12:54 user nova-compute[72351]:     <arch name='xtensaeb'>
Aug 09 02:12:54 user nova-compute[72351]:       <wordsize>32</wordsize>
Aug 09 02:12:54 user nova-compute[72351]:       <emulator>/usr/bin/qemu-system-xtensaeb</emulator>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='4'>sim</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='32'>kc705</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='32'>ml605</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='32'>ml605-nommu</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='32'>virt</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='32'>lx60-nommu</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='32'>lx200</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='32'>lx200-nommu</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='32'>lx60</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <machine maxCpus='32'>kc705-nommu</machine>
Aug 09 02:12:54 user nova-compute[72351]:       <domain type='qemu'/>
Aug 09 02:12:54 user nova-compute[72351]:     </arch>
Aug 09 02:12:54 user nova-compute[72351]:     <features>
Aug 09 02:12:54 user nova-compute[72351]:       <cpuselection/>
Aug 09 02:12:54 user nova-compute[72351]:       <deviceboot/>
Aug 09 02:12:54 user nova-compute[72351]:       <disksnapshot default='on' toggle='no'/>
Aug 09 02:12:54 user nova-compute[72351]:     </features>
Aug 09 02:12:54 user nova-compute[72351]:   </guest>
Aug 09 02:12:54 user nova-compute[72351]: </capabilities>
Aug 09 02:12:54 user nova-compute[72351]: 
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Getting domain capabilities for alpha via machine types: {None} {{(pid=72351) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Error from libvirt when retrieving domain capabilities for arch alpha / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-alpha' on this host {{(pid=72351) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Getting domain capabilities for armv6l via machine types: {'virt', None} {{(pid=72351) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Error from libvirt when retrieving domain capabilities for arch armv6l / virt_type kvm / machine_type virt: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-arm' on this host {{(pid=72351) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Error from libvirt when retrieving domain capabilities for arch armv6l / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-arm' on this host {{(pid=72351) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Getting domain capabilities for armv7l via machine types: {'virt'} {{(pid=72351) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Error from libvirt when retrieving domain capabilities for arch armv7l / virt_type kvm / machine_type virt: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-arm' on this host {{(pid=72351) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Getting domain capabilities for aarch64 via machine types: {'virt'} {{(pid=72351) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Error from libvirt when retrieving domain capabilities for arch aarch64 / virt_type kvm / machine_type virt: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-aarch64' on this host {{(pid=72351) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Getting domain capabilities for cris via machine types: {None} {{(pid=72351) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Error from libvirt when retrieving domain capabilities for arch cris / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-cris' on this host {{(pid=72351) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Getting domain capabilities for i686 via machine types: {'ubuntu-q35', 'ubuntu', 'pc', 'q35'} {{(pid=72351) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Libvirt host hypervisor capabilities for arch=i686 and machine_type=ubuntu-q35:
Aug 09 02:12:54 user nova-compute[72351]: <domainCapabilities>
Aug 09 02:12:54 user nova-compute[72351]:   <path>/usr/bin/qemu-system-i386</path>
Aug 09 02:12:54 user nova-compute[72351]:   <domain>kvm</domain>
Aug 09 02:12:54 user nova-compute[72351]:   <machine>pc-q35-jammy</machine>
Aug 09 02:12:54 user nova-compute[72351]:   <arch>i686</arch>
Aug 09 02:12:54 user nova-compute[72351]:   <vcpu max='288'/>
Aug 09 02:12:54 user nova-compute[72351]:   <iothreads supported='yes'/>
Aug 09 02:12:54 user nova-compute[72351]:   <os supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:     <enum name='firmware'/>
Aug 09 02:12:54 user nova-compute[72351]:     <loader supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <value>/usr/share/OVMF/OVMF_CODE.fd</value>
Aug 09 02:12:54 user nova-compute[72351]:       <value>/usr/share/OVMF/OVMF_CODE.secboot.fd</value>
Aug 09 02:12:54 user nova-compute[72351]:       <value>/usr/share/AAVMF/AAVMF_CODE.fd</value>
Aug 09 02:12:54 user nova-compute[72351]:       <value>/usr/share/AAVMF/AAVMF32_CODE.fd</value>
Aug 09 02:12:54 user nova-compute[72351]:       <value>/usr/share/OVMF/OVMF_CODE.ms.fd</value>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='type'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>rom</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>pflash</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='readonly'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>yes</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>no</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='secure'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>no</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </loader>
Aug 09 02:12:54 user nova-compute[72351]:   </os>
Aug 09 02:12:54 user nova-compute[72351]:   <cpu>
Aug 09 02:12:54 user nova-compute[72351]:     <mode name='host-passthrough' supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='hostPassthroughMigratable'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>off</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </mode>
Aug 09 02:12:54 user nova-compute[72351]:     <mode name='maximum' supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='maximumMigratable'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>on</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>off</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </mode>
Aug 09 02:12:54 user nova-compute[72351]:     <mode name='host-model' supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <model fallback='forbid'>IvyBridge-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <vendor>Intel</vendor>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='ss'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='vmx'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='pcid'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='hypervisor'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='arat'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='tsc_adjust'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='umip'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='md-clear'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='stibp'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='arch-capabilities'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='ssbd'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='ibpb'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='ibrs'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='amd-stibp'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='amd-ssbd'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='disable' name='erms'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='disable' name='xsaveopt'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='disable' name='lm'/>
Aug 09 02:12:54 user nova-compute[72351]:     </mode>
Aug 09 02:12:54 user nova-compute[72351]:     <mode name='custom' supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>qemu64</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>qemu32</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>phenom</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>pentium3</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>pentium2</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>pentium</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>n270</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>kvm64</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>kvm32</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>coreduo</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>core2duo</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>athlon</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Westmere-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Westmere</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Snowridge</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Skylake-Server-noTSX-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Skylake-Server-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Skylake-Server</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Skylake-Client-noTSX-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Skylake-Client-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Skylake-Client</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>SandyBridge-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>SandyBridge</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Penryn</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Opteron_G5</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Opteron_G4</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Opteron_G3</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Opteron_G2</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Opteron_G1</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Nehalem-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Nehalem</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>IvyBridge-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>IvyBridge</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Icelake-Server-noTSX</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Icelake-Server</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no' deprecated='yes'>Icelake-Client-noTSX</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no' deprecated='yes'>Icelake-Client</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Haswell-noTSX-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Haswell-noTSX</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Haswell-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Haswell</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>EPYC-Rome</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>EPYC-Milan</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>EPYC-IBPB</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>EPYC</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Dhyana</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Cooperlake</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Conroe</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Cascadelake-Server-noTSX</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Cascadelake-Server</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Broadwell-noTSX-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Broadwell-noTSX</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Broadwell-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Broadwell</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>486</model>
Aug 09 02:12:54 user nova-compute[72351]:     </mode>
Aug 09 02:12:54 user nova-compute[72351]:   </cpu>
Aug 09 02:12:54 user nova-compute[72351]:   <memoryBacking supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:     <enum name='sourceType'>
Aug 09 02:12:54 user nova-compute[72351]:       <value>file</value>
Aug 09 02:12:54 user nova-compute[72351]:       <value>anonymous</value>
Aug 09 02:12:54 user nova-compute[72351]:       <value>memfd</value>
Aug 09 02:12:54 user nova-compute[72351]:     </enum>
Aug 09 02:12:54 user nova-compute[72351]:   </memoryBacking>
Aug 09 02:12:54 user nova-compute[72351]:   <devices>
Aug 09 02:12:54 user nova-compute[72351]:     <disk supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='diskDevice'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>disk</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>cdrom</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>floppy</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>lun</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='bus'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>fdc</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>scsi</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>usb</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>sata</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='model'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio-transitional</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio-non-transitional</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </disk>
Aug 09 02:12:54 user nova-compute[72351]:     <graphics supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='type'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>sdl</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>vnc</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>spice</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>egl-headless</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </graphics>
Aug 09 02:12:54 user nova-compute[72351]:     <video supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='modelType'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>vga</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>cirrus</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>vmvga</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>qxl</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>none</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>bochs</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>ramfb</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </video>
Aug 09 02:12:54 user nova-compute[72351]:     <hostdev supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='mode'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>subsystem</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='startupPolicy'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>default</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>mandatory</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>requisite</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>optional</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='subsysType'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>usb</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>pci</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>scsi</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='capsType'/>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='pciBackend'/>
Aug 09 02:12:54 user nova-compute[72351]:     </hostdev>
Aug 09 02:12:54 user nova-compute[72351]:     <rng supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='model'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio-transitional</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio-non-transitional</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='backendModel'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>random</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>egd</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>builtin</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </rng>
Aug 09 02:12:54 user nova-compute[72351]:     <filesystem supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='driverType'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>path</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>handle</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtiofs</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </filesystem>
Aug 09 02:12:54 user nova-compute[72351]:     <tpm supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='model'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>tpm-tis</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>tpm-crb</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='backendModel'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>passthrough</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>emulator</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </tpm>
Aug 09 02:12:54 user nova-compute[72351]:   </devices>
Aug 09 02:12:54 user nova-compute[72351]:   <features>
Aug 09 02:12:54 user nova-compute[72351]:     <gic supported='no'/>
Aug 09 02:12:54 user nova-compute[72351]:     <vmcoreinfo supported='yes'/>
Aug 09 02:12:54 user nova-compute[72351]:     <genid supported='yes'/>
Aug 09 02:12:54 user nova-compute[72351]:     <backingStoreInput supported='yes'/>
Aug 09 02:12:54 user nova-compute[72351]:     <backup supported='yes'/>
Aug 09 02:12:54 user nova-compute[72351]:     <sev supported='no'/>
Aug 09 02:12:54 user nova-compute[72351]:   </features>
Aug 09 02:12:54 user nova-compute[72351]: </domainCapabilities>
Aug 09 02:12:54 user nova-compute[72351]:  {{(pid=72351) _get_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:1037}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Libvirt host hypervisor capabilities for arch=i686 and machine_type=ubuntu:
Aug 09 02:12:54 user nova-compute[72351]: <domainCapabilities>
Aug 09 02:12:54 user nova-compute[72351]:   <path>/usr/bin/qemu-system-i386</path>
Aug 09 02:12:54 user nova-compute[72351]:   <domain>kvm</domain>
Aug 09 02:12:54 user nova-compute[72351]:   <machine>pc-i440fx-jammy</machine>
Aug 09 02:12:54 user nova-compute[72351]:   <arch>i686</arch>
Aug 09 02:12:54 user nova-compute[72351]:   <vcpu max='255'/>
Aug 09 02:12:54 user nova-compute[72351]:   <iothreads supported='yes'/>
Aug 09 02:12:54 user nova-compute[72351]:   <os supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:     <enum name='firmware'/>
Aug 09 02:12:54 user nova-compute[72351]:     <loader supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <value>/usr/share/OVMF/OVMF_CODE.fd</value>
Aug 09 02:12:54 user nova-compute[72351]:       <value>/usr/share/OVMF/OVMF_CODE.secboot.fd</value>
Aug 09 02:12:54 user nova-compute[72351]:       <value>/usr/share/AAVMF/AAVMF_CODE.fd</value>
Aug 09 02:12:54 user nova-compute[72351]:       <value>/usr/share/AAVMF/AAVMF32_CODE.fd</value>
Aug 09 02:12:54 user nova-compute[72351]:       <value>/usr/share/OVMF/OVMF_CODE.ms.fd</value>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='type'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>rom</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>pflash</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='readonly'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>yes</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>no</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='secure'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>no</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </loader>
Aug 09 02:12:54 user nova-compute[72351]:   </os>
Aug 09 02:12:54 user nova-compute[72351]:   <cpu>
Aug 09 02:12:54 user nova-compute[72351]:     <mode name='host-passthrough' supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='hostPassthroughMigratable'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>off</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </mode>
Aug 09 02:12:54 user nova-compute[72351]:     <mode name='maximum' supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='maximumMigratable'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>on</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>off</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </mode>
Aug 09 02:12:54 user nova-compute[72351]:     <mode name='host-model' supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <model fallback='forbid'>IvyBridge-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <vendor>Intel</vendor>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='ss'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='vmx'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='pcid'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='hypervisor'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='arat'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='tsc_adjust'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='umip'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='md-clear'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='stibp'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='arch-capabilities'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='ssbd'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='ibpb'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='ibrs'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='amd-stibp'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='amd-ssbd'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='disable' name='erms'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='disable' name='xsaveopt'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='disable' name='lm'/>
Aug 09 02:12:54 user nova-compute[72351]:     </mode>
Aug 09 02:12:54 user nova-compute[72351]:     <mode name='custom' supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>qemu64</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>qemu32</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>phenom</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>pentium3</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>pentium2</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>pentium</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>n270</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>kvm64</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>kvm32</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>coreduo</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>core2duo</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>athlon</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Westmere-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Westmere</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Snowridge</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Skylake-Server-noTSX-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Skylake-Server-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Skylake-Server</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Skylake-Client-noTSX-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Skylake-Client-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Skylake-Client</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>SandyBridge-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>SandyBridge</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Penryn</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Opteron_G5</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Opteron_G4</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Opteron_G3</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Opteron_G2</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Opteron_G1</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Nehalem-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Nehalem</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>IvyBridge-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>IvyBridge</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Icelake-Server-noTSX</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Icelake-Server</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no' deprecated='yes'>Icelake-Client-noTSX</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no' deprecated='yes'>Icelake-Client</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Haswell-noTSX-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Haswell-noTSX</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Haswell-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Haswell</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>EPYC-Rome</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>EPYC-Milan</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>EPYC-IBPB</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>EPYC</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Dhyana</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Cooperlake</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Conroe</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Cascadelake-Server-noTSX</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Cascadelake-Server</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Broadwell-noTSX-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Broadwell-noTSX</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Broadwell-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Broadwell</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>486</model>
Aug 09 02:12:54 user nova-compute[72351]:     </mode>
Aug 09 02:12:54 user nova-compute[72351]:   </cpu>
Aug 09 02:12:54 user nova-compute[72351]:   <memoryBacking supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:     <enum name='sourceType'>
Aug 09 02:12:54 user nova-compute[72351]:       <value>file</value>
Aug 09 02:12:54 user nova-compute[72351]:       <value>anonymous</value>
Aug 09 02:12:54 user nova-compute[72351]:       <value>memfd</value>
Aug 09 02:12:54 user nova-compute[72351]:     </enum>
Aug 09 02:12:54 user nova-compute[72351]:   </memoryBacking>
Aug 09 02:12:54 user nova-compute[72351]:   <devices>
Aug 09 02:12:54 user nova-compute[72351]:     <disk supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='diskDevice'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>disk</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>cdrom</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>floppy</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>lun</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='bus'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>ide</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>fdc</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>scsi</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>usb</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>sata</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='model'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio-transitional</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio-non-transitional</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </disk>
Aug 09 02:12:54 user nova-compute[72351]:     <graphics supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='type'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>sdl</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>vnc</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>spice</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>egl-headless</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </graphics>
Aug 09 02:12:54 user nova-compute[72351]:     <video supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='modelType'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>vga</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>cirrus</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>vmvga</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>qxl</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>none</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>bochs</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>ramfb</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </video>
Aug 09 02:12:54 user nova-compute[72351]:     <hostdev supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='mode'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>subsystem</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='startupPolicy'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>default</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>mandatory</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>requisite</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>optional</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='subsysType'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>usb</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>pci</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>scsi</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='capsType'/>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='pciBackend'/>
Aug 09 02:12:54 user nova-compute[72351]:     </hostdev>
Aug 09 02:12:54 user nova-compute[72351]:     <rng supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='model'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio-transitional</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio-non-transitional</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='backendModel'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>random</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>egd</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>builtin</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </rng>
Aug 09 02:12:54 user nova-compute[72351]:     <filesystem supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='driverType'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>path</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>handle</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtiofs</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </filesystem>
Aug 09 02:12:54 user nova-compute[72351]:     <tpm supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='model'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>tpm-tis</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>tpm-crb</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='backendModel'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>passthrough</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>emulator</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </tpm>
Aug 09 02:12:54 user nova-compute[72351]:   </devices>
Aug 09 02:12:54 user nova-compute[72351]:   <features>
Aug 09 02:12:54 user nova-compute[72351]:     <gic supported='no'/>
Aug 09 02:12:54 user nova-compute[72351]:     <vmcoreinfo supported='yes'/>
Aug 09 02:12:54 user nova-compute[72351]:     <genid supported='yes'/>
Aug 09 02:12:54 user nova-compute[72351]:     <backingStoreInput supported='yes'/>
Aug 09 02:12:54 user nova-compute[72351]:     <backup supported='yes'/>
Aug 09 02:12:54 user nova-compute[72351]:     <sev supported='no'/>
Aug 09 02:12:54 user nova-compute[72351]:   </features>
Aug 09 02:12:54 user nova-compute[72351]: </domainCapabilities>
Aug 09 02:12:54 user nova-compute[72351]:  {{(pid=72351) _get_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:1037}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Libvirt host hypervisor capabilities for arch=i686 and machine_type=pc:
Aug 09 02:12:54 user nova-compute[72351]: <domainCapabilities>
Aug 09 02:12:54 user nova-compute[72351]:   <path>/usr/bin/qemu-system-i386</path>
Aug 09 02:12:54 user nova-compute[72351]:   <domain>kvm</domain>
Aug 09 02:12:54 user nova-compute[72351]:   <machine>pc-i440fx-6.2</machine>
Aug 09 02:12:54 user nova-compute[72351]:   <arch>i686</arch>
Aug 09 02:12:54 user nova-compute[72351]:   <vcpu max='255'/>
Aug 09 02:12:54 user nova-compute[72351]:   <iothreads supported='yes'/>
Aug 09 02:12:54 user nova-compute[72351]:   <os supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:     <enum name='firmware'/>
Aug 09 02:12:54 user nova-compute[72351]:     <loader supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <value>/usr/share/OVMF/OVMF_CODE.fd</value>
Aug 09 02:12:54 user nova-compute[72351]:       <value>/usr/share/OVMF/OVMF_CODE.secboot.fd</value>
Aug 09 02:12:54 user nova-compute[72351]:       <value>/usr/share/AAVMF/AAVMF_CODE.fd</value>
Aug 09 02:12:54 user nova-compute[72351]:       <value>/usr/share/AAVMF/AAVMF32_CODE.fd</value>
Aug 09 02:12:54 user nova-compute[72351]:       <value>/usr/share/OVMF/OVMF_CODE.ms.fd</value>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='type'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>rom</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>pflash</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='readonly'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>yes</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>no</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='secure'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>no</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </loader>
Aug 09 02:12:54 user nova-compute[72351]:   </os>
Aug 09 02:12:54 user nova-compute[72351]:   <cpu>
Aug 09 02:12:54 user nova-compute[72351]:     <mode name='host-passthrough' supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='hostPassthroughMigratable'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>off</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </mode>
Aug 09 02:12:54 user nova-compute[72351]:     <mode name='maximum' supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='maximumMigratable'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>on</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>off</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </mode>
Aug 09 02:12:54 user nova-compute[72351]:     <mode name='host-model' supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <model fallback='forbid'>IvyBridge-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <vendor>Intel</vendor>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='ss'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='vmx'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='pcid'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='hypervisor'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='arat'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='tsc_adjust'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='umip'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='md-clear'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='stibp'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='arch-capabilities'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='ssbd'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='ibpb'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='ibrs'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='amd-stibp'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='amd-ssbd'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='disable' name='erms'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='disable' name='xsaveopt'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='disable' name='lm'/>
Aug 09 02:12:54 user nova-compute[72351]:     </mode>
Aug 09 02:12:54 user nova-compute[72351]:     <mode name='custom' supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>qemu64</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>qemu32</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>phenom</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>pentium3</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>pentium2</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>pentium</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>n270</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>kvm64</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>kvm32</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>coreduo</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>core2duo</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>athlon</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Westmere-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Westmere</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Snowridge</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Skylake-Server-noTSX-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Skylake-Server-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Skylake-Server</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Skylake-Client-noTSX-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Skylake-Client-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Skylake-Client</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>SandyBridge-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>SandyBridge</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Penryn</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Opteron_G5</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Opteron_G4</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Opteron_G3</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Opteron_G2</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Opteron_G1</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Nehalem-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Nehalem</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>IvyBridge-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>IvyBridge</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Icelake-Server-noTSX</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Icelake-Server</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no' deprecated='yes'>Icelake-Client-noTSX</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no' deprecated='yes'>Icelake-Client</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Haswell-noTSX-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Haswell-noTSX</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Haswell-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Haswell</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>EPYC-Rome</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>EPYC-Milan</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>EPYC-IBPB</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>EPYC</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Dhyana</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Cooperlake</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Conroe</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Cascadelake-Server-noTSX</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Cascadelake-Server</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Broadwell-noTSX-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Broadwell-noTSX</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Broadwell-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Broadwell</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>486</model>
Aug 09 02:12:54 user nova-compute[72351]:     </mode>
Aug 09 02:12:54 user nova-compute[72351]:   </cpu>
Aug 09 02:12:54 user nova-compute[72351]:   <memoryBacking supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:     <enum name='sourceType'>
Aug 09 02:12:54 user nova-compute[72351]:       <value>file</value>
Aug 09 02:12:54 user nova-compute[72351]:       <value>anonymous</value>
Aug 09 02:12:54 user nova-compute[72351]:       <value>memfd</value>
Aug 09 02:12:54 user nova-compute[72351]:     </enum>
Aug 09 02:12:54 user nova-compute[72351]:   </memoryBacking>
Aug 09 02:12:54 user nova-compute[72351]:   <devices>
Aug 09 02:12:54 user nova-compute[72351]:     <disk supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='diskDevice'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>disk</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>cdrom</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>floppy</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>lun</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='bus'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>ide</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>fdc</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>scsi</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>usb</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>sata</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='model'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio-transitional</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio-non-transitional</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </disk>
Aug 09 02:12:54 user nova-compute[72351]:     <graphics supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='type'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>sdl</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>vnc</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>spice</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>egl-headless</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </graphics>
Aug 09 02:12:54 user nova-compute[72351]:     <video supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='modelType'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>vga</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>cirrus</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>vmvga</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>qxl</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>none</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>bochs</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>ramfb</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </video>
Aug 09 02:12:54 user nova-compute[72351]:     <hostdev supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='mode'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>subsystem</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='startupPolicy'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>default</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>mandatory</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>requisite</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>optional</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='subsysType'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>usb</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>pci</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>scsi</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='capsType'/>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='pciBackend'/>
Aug 09 02:12:54 user nova-compute[72351]:     </hostdev>
Aug 09 02:12:54 user nova-compute[72351]:     <rng supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='model'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio-transitional</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio-non-transitional</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='backendModel'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>random</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>egd</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>builtin</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </rng>
Aug 09 02:12:54 user nova-compute[72351]:     <filesystem supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='driverType'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>path</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>handle</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtiofs</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </filesystem>
Aug 09 02:12:54 user nova-compute[72351]:     <tpm supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='model'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>tpm-tis</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>tpm-crb</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='backendModel'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>passthrough</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>emulator</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </tpm>
Aug 09 02:12:54 user nova-compute[72351]:   </devices>
Aug 09 02:12:54 user nova-compute[72351]:   <features>
Aug 09 02:12:54 user nova-compute[72351]:     <gic supported='no'/>
Aug 09 02:12:54 user nova-compute[72351]:     <vmcoreinfo supported='yes'/>
Aug 09 02:12:54 user nova-compute[72351]:     <genid supported='yes'/>
Aug 09 02:12:54 user nova-compute[72351]:     <backingStoreInput supported='yes'/>
Aug 09 02:12:54 user nova-compute[72351]:     <backup supported='yes'/>
Aug 09 02:12:54 user nova-compute[72351]:     <sev supported='no'/>
Aug 09 02:12:54 user nova-compute[72351]:   </features>
Aug 09 02:12:54 user nova-compute[72351]: </domainCapabilities>
Aug 09 02:12:54 user nova-compute[72351]:  {{(pid=72351) _get_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:1037}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Libvirt host hypervisor capabilities for arch=i686 and machine_type=q35:
Aug 09 02:12:54 user nova-compute[72351]: <domainCapabilities>
Aug 09 02:12:54 user nova-compute[72351]:   <path>/usr/bin/qemu-system-i386</path>
Aug 09 02:12:54 user nova-compute[72351]:   <domain>kvm</domain>
Aug 09 02:12:54 user nova-compute[72351]:   <machine>pc-q35-6.2</machine>
Aug 09 02:12:54 user nova-compute[72351]:   <arch>i686</arch>
Aug 09 02:12:54 user nova-compute[72351]:   <vcpu max='288'/>
Aug 09 02:12:54 user nova-compute[72351]:   <iothreads supported='yes'/>
Aug 09 02:12:54 user nova-compute[72351]:   <os supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:     <enum name='firmware'/>
Aug 09 02:12:54 user nova-compute[72351]:     <loader supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <value>/usr/share/OVMF/OVMF_CODE.fd</value>
Aug 09 02:12:54 user nova-compute[72351]:       <value>/usr/share/OVMF/OVMF_CODE.secboot.fd</value>
Aug 09 02:12:54 user nova-compute[72351]:       <value>/usr/share/AAVMF/AAVMF_CODE.fd</value>
Aug 09 02:12:54 user nova-compute[72351]:       <value>/usr/share/AAVMF/AAVMF32_CODE.fd</value>
Aug 09 02:12:54 user nova-compute[72351]:       <value>/usr/share/OVMF/OVMF_CODE.ms.fd</value>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='type'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>rom</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>pflash</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='readonly'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>yes</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>no</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='secure'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>no</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </loader>
Aug 09 02:12:54 user nova-compute[72351]:   </os>
Aug 09 02:12:54 user nova-compute[72351]:   <cpu>
Aug 09 02:12:54 user nova-compute[72351]:     <mode name='host-passthrough' supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='hostPassthroughMigratable'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>off</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </mode>
Aug 09 02:12:54 user nova-compute[72351]:     <mode name='maximum' supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='maximumMigratable'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>on</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>off</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </mode>
Aug 09 02:12:54 user nova-compute[72351]:     <mode name='host-model' supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <model fallback='forbid'>IvyBridge-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <vendor>Intel</vendor>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='ss'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='vmx'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='pcid'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='hypervisor'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='arat'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='tsc_adjust'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='umip'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='md-clear'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='stibp'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='arch-capabilities'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='ssbd'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='ibpb'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='ibrs'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='amd-stibp'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='amd-ssbd'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='disable' name='erms'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='disable' name='xsaveopt'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='disable' name='lm'/>
Aug 09 02:12:54 user nova-compute[72351]:     </mode>
Aug 09 02:12:54 user nova-compute[72351]:     <mode name='custom' supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>qemu64</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>qemu32</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>phenom</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>pentium3</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>pentium2</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>pentium</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>n270</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>kvm64</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>kvm32</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>coreduo</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>core2duo</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>athlon</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Westmere-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Westmere</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Snowridge</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Skylake-Server-noTSX-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Skylake-Server-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Skylake-Server</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Skylake-Client-noTSX-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Skylake-Client-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Skylake-Client</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>SandyBridge-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>SandyBridge</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Penryn</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Opteron_G5</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Opteron_G4</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Opteron_G3</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Opteron_G2</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Opteron_G1</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Nehalem-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Nehalem</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>IvyBridge-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>IvyBridge</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Icelake-Server-noTSX</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Icelake-Server</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no' deprecated='yes'>Icelake-Client-noTSX</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no' deprecated='yes'>Icelake-Client</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Haswell-noTSX-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Haswell-noTSX</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Haswell-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Haswell</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>EPYC-Rome</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>EPYC-Milan</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>EPYC-IBPB</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>EPYC</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Dhyana</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Cooperlake</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Conroe</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Cascadelake-Server-noTSX</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Cascadelake-Server</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Broadwell-noTSX-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Broadwell-noTSX</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Broadwell-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Broadwell</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>486</model>
Aug 09 02:12:54 user nova-compute[72351]:     </mode>
Aug 09 02:12:54 user nova-compute[72351]:   </cpu>
Aug 09 02:12:54 user nova-compute[72351]:   <memoryBacking supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:     <enum name='sourceType'>
Aug 09 02:12:54 user nova-compute[72351]:       <value>file</value>
Aug 09 02:12:54 user nova-compute[72351]:       <value>anonymous</value>
Aug 09 02:12:54 user nova-compute[72351]:       <value>memfd</value>
Aug 09 02:12:54 user nova-compute[72351]:     </enum>
Aug 09 02:12:54 user nova-compute[72351]:   </memoryBacking>
Aug 09 02:12:54 user nova-compute[72351]:   <devices>
Aug 09 02:12:54 user nova-compute[72351]:     <disk supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='diskDevice'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>disk</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>cdrom</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>floppy</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>lun</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='bus'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>fdc</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>scsi</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>usb</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>sata</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='model'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio-transitional</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio-non-transitional</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </disk>
Aug 09 02:12:54 user nova-compute[72351]:     <graphics supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='type'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>sdl</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>vnc</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>spice</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>egl-headless</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </graphics>
Aug 09 02:12:54 user nova-compute[72351]:     <video supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='modelType'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>vga</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>cirrus</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>vmvga</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>qxl</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>none</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>bochs</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>ramfb</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </video>
Aug 09 02:12:54 user nova-compute[72351]:     <hostdev supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='mode'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>subsystem</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='startupPolicy'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>default</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>mandatory</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>requisite</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>optional</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='subsysType'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>usb</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>pci</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>scsi</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='capsType'/>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='pciBackend'/>
Aug 09 02:12:54 user nova-compute[72351]:     </hostdev>
Aug 09 02:12:54 user nova-compute[72351]:     <rng supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='model'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio-transitional</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio-non-transitional</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='backendModel'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>random</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>egd</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>builtin</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </rng>
Aug 09 02:12:54 user nova-compute[72351]:     <filesystem supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='driverType'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>path</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>handle</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtiofs</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </filesystem>
Aug 09 02:12:54 user nova-compute[72351]:     <tpm supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='model'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>tpm-tis</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>tpm-crb</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='backendModel'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>passthrough</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>emulator</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </tpm>
Aug 09 02:12:54 user nova-compute[72351]:   </devices>
Aug 09 02:12:54 user nova-compute[72351]:   <features>
Aug 09 02:12:54 user nova-compute[72351]:     <gic supported='no'/>
Aug 09 02:12:54 user nova-compute[72351]:     <vmcoreinfo supported='yes'/>
Aug 09 02:12:54 user nova-compute[72351]:     <genid supported='yes'/>
Aug 09 02:12:54 user nova-compute[72351]:     <backingStoreInput supported='yes'/>
Aug 09 02:12:54 user nova-compute[72351]:     <backup supported='yes'/>
Aug 09 02:12:54 user nova-compute[72351]:     <sev supported='no'/>
Aug 09 02:12:54 user nova-compute[72351]:   </features>
Aug 09 02:12:54 user nova-compute[72351]: </domainCapabilities>
Aug 09 02:12:54 user nova-compute[72351]:  {{(pid=72351) _get_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:1037}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Getting domain capabilities for m68k via machine types: {'virt', None} {{(pid=72351) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Error from libvirt when retrieving domain capabilities for arch m68k / virt_type kvm / machine_type virt: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-m68k' on this host {{(pid=72351) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Error from libvirt when retrieving domain capabilities for arch m68k / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-m68k' on this host {{(pid=72351) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Getting domain capabilities for microblaze via machine types: {None} {{(pid=72351) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Error from libvirt when retrieving domain capabilities for arch microblaze / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-microblaze' on this host {{(pid=72351) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Getting domain capabilities for microblazeel via machine types: {None} {{(pid=72351) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Error from libvirt when retrieving domain capabilities for arch microblazeel / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-microblazeel' on this host {{(pid=72351) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Getting domain capabilities for mips via machine types: {None} {{(pid=72351) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Error from libvirt when retrieving domain capabilities for arch mips / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-mips' on this host {{(pid=72351) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Getting domain capabilities for mipsel via machine types: {None} {{(pid=72351) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Error from libvirt when retrieving domain capabilities for arch mipsel / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-mipsel' on this host {{(pid=72351) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Getting domain capabilities for mips64 via machine types: {None} {{(pid=72351) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Error from libvirt when retrieving domain capabilities for arch mips64 / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-mips64' on this host {{(pid=72351) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Getting domain capabilities for mips64el via machine types: {None} {{(pid=72351) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Error from libvirt when retrieving domain capabilities for arch mips64el / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-mips64el' on this host {{(pid=72351) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Getting domain capabilities for ppc via machine types: {None} {{(pid=72351) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Error from libvirt when retrieving domain capabilities for arch ppc / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-ppc' on this host {{(pid=72351) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Getting domain capabilities for ppc64 via machine types: {'pseries', None, 'powernv'} {{(pid=72351) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Error from libvirt when retrieving domain capabilities for arch ppc64 / virt_type kvm / machine_type pseries: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-ppc64' on this host {{(pid=72351) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Error from libvirt when retrieving domain capabilities for arch ppc64 / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-ppc64' on this host {{(pid=72351) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Error from libvirt when retrieving domain capabilities for arch ppc64 / virt_type kvm / machine_type powernv: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-ppc64' on this host {{(pid=72351) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Getting domain capabilities for ppc64le via machine types: {'pseries', 'powernv'} {{(pid=72351) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Error from libvirt when retrieving domain capabilities for arch ppc64le / virt_type kvm / machine_type pseries: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-ppc64le' on this host {{(pid=72351) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Error from libvirt when retrieving domain capabilities for arch ppc64le / virt_type kvm / machine_type powernv: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-ppc64le' on this host {{(pid=72351) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Getting domain capabilities for riscv32 via machine types: {None} {{(pid=72351) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Error from libvirt when retrieving domain capabilities for arch riscv32 / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-riscv32' on this host {{(pid=72351) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Getting domain capabilities for riscv64 via machine types: {None} {{(pid=72351) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Error from libvirt when retrieving domain capabilities for arch riscv64 / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-riscv64' on this host {{(pid=72351) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Getting domain capabilities for s390x via machine types: {'s390-ccw-virtio'} {{(pid=72351) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Error from libvirt when retrieving domain capabilities for arch s390x / virt_type kvm / machine_type s390-ccw-virtio: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-s390x' on this host {{(pid=72351) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Getting domain capabilities for sh4 via machine types: {None} {{(pid=72351) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Error from libvirt when retrieving domain capabilities for arch sh4 / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-sh4' on this host {{(pid=72351) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Getting domain capabilities for sh4eb via machine types: {None} {{(pid=72351) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Error from libvirt when retrieving domain capabilities for arch sh4eb / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-sh4eb' on this host {{(pid=72351) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Getting domain capabilities for sparc via machine types: {None} {{(pid=72351) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Error from libvirt when retrieving domain capabilities for arch sparc / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-sparc' on this host {{(pid=72351) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Getting domain capabilities for sparc64 via machine types: {None} {{(pid=72351) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Error from libvirt when retrieving domain capabilities for arch sparc64 / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-sparc64' on this host {{(pid=72351) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Getting domain capabilities for x86_64 via machine types: {'ubuntu-q35', 'ubuntu', 'pc', 'q35'} {{(pid=72351) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Libvirt host hypervisor capabilities for arch=x86_64 and machine_type=ubuntu-q35:
Aug 09 02:12:54 user nova-compute[72351]: <domainCapabilities>
Aug 09 02:12:54 user nova-compute[72351]:   <path>/usr/bin/qemu-system-x86_64</path>
Aug 09 02:12:54 user nova-compute[72351]:   <domain>kvm</domain>
Aug 09 02:12:54 user nova-compute[72351]:   <machine>pc-q35-jammy</machine>
Aug 09 02:12:54 user nova-compute[72351]:   <arch>x86_64</arch>
Aug 09 02:12:54 user nova-compute[72351]:   <vcpu max='288'/>
Aug 09 02:12:54 user nova-compute[72351]:   <iothreads supported='yes'/>
Aug 09 02:12:54 user nova-compute[72351]:   <os supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:     <enum name='firmware'>
Aug 09 02:12:54 user nova-compute[72351]:       <value>efi</value>
Aug 09 02:12:54 user nova-compute[72351]:     </enum>
Aug 09 02:12:54 user nova-compute[72351]:     <loader supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <value>/usr/share/OVMF/OVMF_CODE_4M.ms.fd</value>
Aug 09 02:12:54 user nova-compute[72351]:       <value>/usr/share/OVMF/OVMF_CODE_4M.secboot.fd</value>
Aug 09 02:12:54 user nova-compute[72351]:       <value>/usr/share/OVMF/OVMF_CODE_4M.fd</value>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='type'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>rom</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>pflash</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='readonly'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>yes</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>no</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='secure'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>yes</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>no</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </loader>
Aug 09 02:12:54 user nova-compute[72351]:   </os>
Aug 09 02:12:54 user nova-compute[72351]:   <cpu>
Aug 09 02:12:54 user nova-compute[72351]:     <mode name='host-passthrough' supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='hostPassthroughMigratable'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>on</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>off</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </mode>
Aug 09 02:12:54 user nova-compute[72351]:     <mode name='maximum' supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='maximumMigratable'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>on</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>off</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </mode>
Aug 09 02:12:54 user nova-compute[72351]:     <mode name='host-model' supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <model fallback='forbid'>IvyBridge-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <vendor>Intel</vendor>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='ss'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='vmx'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='pcid'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='hypervisor'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='arat'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='tsc_adjust'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='umip'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='md-clear'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='stibp'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='arch-capabilities'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='ssbd'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='invtsc'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='ibpb'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='ibrs'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='amd-stibp'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='amd-ssbd'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='rsba'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='skip-l1dfl-vmentry'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='pschange-mc-no'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='disable' name='erms'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='disable' name='xsaveopt'/>
Aug 09 02:12:54 user nova-compute[72351]:     </mode>
Aug 09 02:12:54 user nova-compute[72351]:     <mode name='custom' supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>qemu64</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>qemu32</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>phenom</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>pentium3</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>pentium2</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>pentium</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>n270</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>kvm64</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>kvm32</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>coreduo</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>core2duo</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>athlon</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>Westmere-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>Westmere</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Snowridge</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Skylake-Server-noTSX-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Skylake-Server-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Skylake-Server</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Skylake-Client-noTSX-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Skylake-Client-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Skylake-Client</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>SandyBridge-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>SandyBridge</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>Penryn</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Opteron_G5</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Opteron_G4</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Opteron_G3</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Opteron_G2</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>Opteron_G1</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>Nehalem-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>Nehalem</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>IvyBridge-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>IvyBridge</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Icelake-Server-noTSX</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Icelake-Server</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no' deprecated='yes'>Icelake-Client-noTSX</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no' deprecated='yes'>Icelake-Client</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Haswell-noTSX-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Haswell-noTSX</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Haswell-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Haswell</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>EPYC-Rome</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>EPYC-Milan</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>EPYC-IBPB</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>EPYC</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Dhyana</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Cooperlake</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>Conroe</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Cascadelake-Server-noTSX</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Cascadelake-Server</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Broadwell-noTSX-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Broadwell-noTSX</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Broadwell-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Broadwell</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>486</model>
Aug 09 02:12:54 user nova-compute[72351]:     </mode>
Aug 09 02:12:54 user nova-compute[72351]:   </cpu>
Aug 09 02:12:54 user nova-compute[72351]:   <memoryBacking supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:     <enum name='sourceType'>
Aug 09 02:12:54 user nova-compute[72351]:       <value>file</value>
Aug 09 02:12:54 user nova-compute[72351]:       <value>anonymous</value>
Aug 09 02:12:54 user nova-compute[72351]:       <value>memfd</value>
Aug 09 02:12:54 user nova-compute[72351]:     </enum>
Aug 09 02:12:54 user nova-compute[72351]:   </memoryBacking>
Aug 09 02:12:54 user nova-compute[72351]:   <devices>
Aug 09 02:12:54 user nova-compute[72351]:     <disk supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='diskDevice'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>disk</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>cdrom</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>floppy</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>lun</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='bus'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>fdc</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>scsi</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>usb</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>sata</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='model'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio-transitional</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio-non-transitional</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </disk>
Aug 09 02:12:54 user nova-compute[72351]:     <graphics supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='type'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>sdl</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>vnc</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>spice</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>egl-headless</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </graphics>
Aug 09 02:12:54 user nova-compute[72351]:     <video supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='modelType'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>vga</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>cirrus</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>vmvga</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>qxl</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>none</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>bochs</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>ramfb</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </video>
Aug 09 02:12:54 user nova-compute[72351]:     <hostdev supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='mode'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>subsystem</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='startupPolicy'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>default</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>mandatory</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>requisite</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>optional</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='subsysType'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>usb</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>pci</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>scsi</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='capsType'/>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='pciBackend'/>
Aug 09 02:12:54 user nova-compute[72351]:     </hostdev>
Aug 09 02:12:54 user nova-compute[72351]:     <rng supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='model'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio-transitional</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio-non-transitional</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='backendModel'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>random</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>egd</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>builtin</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </rng>
Aug 09 02:12:54 user nova-compute[72351]:     <filesystem supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='driverType'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>path</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>handle</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtiofs</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </filesystem>
Aug 09 02:12:54 user nova-compute[72351]:     <tpm supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='model'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>tpm-tis</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>tpm-crb</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='backendModel'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>passthrough</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>emulator</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </tpm>
Aug 09 02:12:54 user nova-compute[72351]:   </devices>
Aug 09 02:12:54 user nova-compute[72351]:   <features>
Aug 09 02:12:54 user nova-compute[72351]:     <gic supported='no'/>
Aug 09 02:12:54 user nova-compute[72351]:     <vmcoreinfo supported='yes'/>
Aug 09 02:12:54 user nova-compute[72351]:     <genid supported='yes'/>
Aug 09 02:12:54 user nova-compute[72351]:     <backingStoreInput supported='yes'/>
Aug 09 02:12:54 user nova-compute[72351]:     <backup supported='yes'/>
Aug 09 02:12:54 user nova-compute[72351]:     <sev supported='no'/>
Aug 09 02:12:54 user nova-compute[72351]:   </features>
Aug 09 02:12:54 user nova-compute[72351]: </domainCapabilities>
Aug 09 02:12:54 user nova-compute[72351]:  {{(pid=72351) _get_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:1037}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Libvirt host hypervisor capabilities for arch=x86_64 and machine_type=ubuntu:
Aug 09 02:12:54 user nova-compute[72351]: <domainCapabilities>
Aug 09 02:12:54 user nova-compute[72351]:   <path>/usr/bin/qemu-system-x86_64</path>
Aug 09 02:12:54 user nova-compute[72351]:   <domain>kvm</domain>
Aug 09 02:12:54 user nova-compute[72351]:   <machine>pc-i440fx-jammy</machine>
Aug 09 02:12:54 user nova-compute[72351]:   <arch>x86_64</arch>
Aug 09 02:12:54 user nova-compute[72351]:   <vcpu max='255'/>
Aug 09 02:12:54 user nova-compute[72351]:   <iothreads supported='yes'/>
Aug 09 02:12:54 user nova-compute[72351]:   <os supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:     <enum name='firmware'>
Aug 09 02:12:54 user nova-compute[72351]:       <value>efi</value>
Aug 09 02:12:54 user nova-compute[72351]:     </enum>
Aug 09 02:12:54 user nova-compute[72351]:     <loader supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <value>/usr/share/OVMF/OVMF_CODE_4M.fd</value>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='type'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>rom</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>pflash</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='readonly'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>yes</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>no</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='secure'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>no</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </loader>
Aug 09 02:12:54 user nova-compute[72351]:   </os>
Aug 09 02:12:54 user nova-compute[72351]:   <cpu>
Aug 09 02:12:54 user nova-compute[72351]:     <mode name='host-passthrough' supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='hostPassthroughMigratable'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>on</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>off</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </mode>
Aug 09 02:12:54 user nova-compute[72351]:     <mode name='maximum' supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='maximumMigratable'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>on</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>off</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </mode>
Aug 09 02:12:54 user nova-compute[72351]:     <mode name='host-model' supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <model fallback='forbid'>IvyBridge-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <vendor>Intel</vendor>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='ss'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='vmx'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='pcid'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='hypervisor'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='arat'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='tsc_adjust'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='umip'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='md-clear'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='stibp'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='arch-capabilities'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='ssbd'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='invtsc'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='ibpb'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='ibrs'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='amd-stibp'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='amd-ssbd'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='rsba'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='skip-l1dfl-vmentry'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='pschange-mc-no'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='disable' name='erms'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='disable' name='xsaveopt'/>
Aug 09 02:12:54 user nova-compute[72351]:     </mode>
Aug 09 02:12:54 user nova-compute[72351]:     <mode name='custom' supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>qemu64</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>qemu32</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>phenom</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>pentium3</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>pentium2</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>pentium</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>n270</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>kvm64</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>kvm32</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>coreduo</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>core2duo</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>athlon</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>Westmere-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>Westmere</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Snowridge</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Skylake-Server-noTSX-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Skylake-Server-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Skylake-Server</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Skylake-Client-noTSX-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Skylake-Client-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Skylake-Client</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>SandyBridge-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>SandyBridge</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>Penryn</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Opteron_G5</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Opteron_G4</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Opteron_G3</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Opteron_G2</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>Opteron_G1</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>Nehalem-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>Nehalem</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>IvyBridge-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>IvyBridge</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Icelake-Server-noTSX</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Icelake-Server</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no' deprecated='yes'>Icelake-Client-noTSX</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no' deprecated='yes'>Icelake-Client</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Haswell-noTSX-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Haswell-noTSX</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Haswell-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Haswell</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>EPYC-Rome</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>EPYC-Milan</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>EPYC-IBPB</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>EPYC</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Dhyana</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Cooperlake</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>Conroe</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Cascadelake-Server-noTSX</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Cascadelake-Server</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Broadwell-noTSX-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Broadwell-noTSX</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Broadwell-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Broadwell</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>486</model>
Aug 09 02:12:54 user nova-compute[72351]:     </mode>
Aug 09 02:12:54 user nova-compute[72351]:   </cpu>
Aug 09 02:12:54 user nova-compute[72351]:   <memoryBacking supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:     <enum name='sourceType'>
Aug 09 02:12:54 user nova-compute[72351]:       <value>file</value>
Aug 09 02:12:54 user nova-compute[72351]:       <value>anonymous</value>
Aug 09 02:12:54 user nova-compute[72351]:       <value>memfd</value>
Aug 09 02:12:54 user nova-compute[72351]:     </enum>
Aug 09 02:12:54 user nova-compute[72351]:   </memoryBacking>
Aug 09 02:12:54 user nova-compute[72351]:   <devices>
Aug 09 02:12:54 user nova-compute[72351]:     <disk supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='diskDevice'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>disk</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>cdrom</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>floppy</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>lun</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='bus'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>ide</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>fdc</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>scsi</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>usb</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>sata</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='model'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio-transitional</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio-non-transitional</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </disk>
Aug 09 02:12:54 user nova-compute[72351]:     <graphics supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='type'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>sdl</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>vnc</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>spice</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>egl-headless</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </graphics>
Aug 09 02:12:54 user nova-compute[72351]:     <video supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='modelType'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>vga</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>cirrus</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>vmvga</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>qxl</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>none</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>bochs</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>ramfb</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </video>
Aug 09 02:12:54 user nova-compute[72351]:     <hostdev supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='mode'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>subsystem</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='startupPolicy'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>default</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>mandatory</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>requisite</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>optional</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='subsysType'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>usb</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>pci</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>scsi</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='capsType'/>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='pciBackend'/>
Aug 09 02:12:54 user nova-compute[72351]:     </hostdev>
Aug 09 02:12:54 user nova-compute[72351]:     <rng supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='model'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio-transitional</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio-non-transitional</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='backendModel'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>random</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>egd</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>builtin</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </rng>
Aug 09 02:12:54 user nova-compute[72351]:     <filesystem supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='driverType'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>path</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>handle</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtiofs</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </filesystem>
Aug 09 02:12:54 user nova-compute[72351]:     <tpm supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='model'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>tpm-tis</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>tpm-crb</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='backendModel'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>passthrough</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>emulator</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </tpm>
Aug 09 02:12:54 user nova-compute[72351]:   </devices>
Aug 09 02:12:54 user nova-compute[72351]:   <features>
Aug 09 02:12:54 user nova-compute[72351]:     <gic supported='no'/>
Aug 09 02:12:54 user nova-compute[72351]:     <vmcoreinfo supported='yes'/>
Aug 09 02:12:54 user nova-compute[72351]:     <genid supported='yes'/>
Aug 09 02:12:54 user nova-compute[72351]:     <backingStoreInput supported='yes'/>
Aug 09 02:12:54 user nova-compute[72351]:     <backup supported='yes'/>
Aug 09 02:12:54 user nova-compute[72351]:     <sev supported='no'/>
Aug 09 02:12:54 user nova-compute[72351]:   </features>
Aug 09 02:12:54 user nova-compute[72351]: </domainCapabilities>
Aug 09 02:12:54 user nova-compute[72351]:  {{(pid=72351) _get_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:1037}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Libvirt host hypervisor capabilities for arch=x86_64 and machine_type=pc:
Aug 09 02:12:54 user nova-compute[72351]: <domainCapabilities>
Aug 09 02:12:54 user nova-compute[72351]:   <path>/usr/bin/qemu-system-x86_64</path>
Aug 09 02:12:54 user nova-compute[72351]:   <domain>kvm</domain>
Aug 09 02:12:54 user nova-compute[72351]:   <machine>pc-i440fx-6.2</machine>
Aug 09 02:12:54 user nova-compute[72351]:   <arch>x86_64</arch>
Aug 09 02:12:54 user nova-compute[72351]:   <vcpu max='255'/>
Aug 09 02:12:54 user nova-compute[72351]:   <iothreads supported='yes'/>
Aug 09 02:12:54 user nova-compute[72351]:   <os supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:     <enum name='firmware'>
Aug 09 02:12:54 user nova-compute[72351]:       <value>efi</value>
Aug 09 02:12:54 user nova-compute[72351]:     </enum>
Aug 09 02:12:54 user nova-compute[72351]:     <loader supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <value>/usr/share/OVMF/OVMF_CODE_4M.fd</value>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='type'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>rom</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>pflash</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='readonly'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>yes</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>no</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='secure'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>no</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </loader>
Aug 09 02:12:54 user nova-compute[72351]:   </os>
Aug 09 02:12:54 user nova-compute[72351]:   <cpu>
Aug 09 02:12:54 user nova-compute[72351]:     <mode name='host-passthrough' supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='hostPassthroughMigratable'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>on</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>off</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </mode>
Aug 09 02:12:54 user nova-compute[72351]:     <mode name='maximum' supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='maximumMigratable'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>on</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>off</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </mode>
Aug 09 02:12:54 user nova-compute[72351]:     <mode name='host-model' supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <model fallback='forbid'>IvyBridge-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <vendor>Intel</vendor>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='ss'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='vmx'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='pcid'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='hypervisor'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='arat'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='tsc_adjust'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='umip'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='md-clear'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='stibp'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='arch-capabilities'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='ssbd'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='invtsc'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='ibpb'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='ibrs'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='amd-stibp'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='amd-ssbd'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='rsba'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='skip-l1dfl-vmentry'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='pschange-mc-no'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='disable' name='erms'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='disable' name='xsaveopt'/>
Aug 09 02:12:54 user nova-compute[72351]:     </mode>
Aug 09 02:12:54 user nova-compute[72351]:     <mode name='custom' supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>qemu64</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>qemu32</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>phenom</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>pentium3</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>pentium2</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>pentium</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>n270</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>kvm64</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>kvm32</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>coreduo</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>core2duo</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>athlon</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>Westmere-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>Westmere</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Snowridge</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Skylake-Server-noTSX-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Skylake-Server-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Skylake-Server</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Skylake-Client-noTSX-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Skylake-Client-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Skylake-Client</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>SandyBridge-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>SandyBridge</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>Penryn</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Opteron_G5</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Opteron_G4</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Opteron_G3</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Opteron_G2</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>Opteron_G1</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>Nehalem-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>Nehalem</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>IvyBridge-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>IvyBridge</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Icelake-Server-noTSX</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Icelake-Server</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no' deprecated='yes'>Icelake-Client-noTSX</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no' deprecated='yes'>Icelake-Client</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Haswell-noTSX-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Haswell-noTSX</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Haswell-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Haswell</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>EPYC-Rome</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>EPYC-Milan</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>EPYC-IBPB</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>EPYC</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Dhyana</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Cooperlake</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>Conroe</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Cascadelake-Server-noTSX</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Cascadelake-Server</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Broadwell-noTSX-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Broadwell-noTSX</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Broadwell-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Broadwell</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>486</model>
Aug 09 02:12:54 user nova-compute[72351]:     </mode>
Aug 09 02:12:54 user nova-compute[72351]:   </cpu>
Aug 09 02:12:54 user nova-compute[72351]:   <memoryBacking supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:     <enum name='sourceType'>
Aug 09 02:12:54 user nova-compute[72351]:       <value>file</value>
Aug 09 02:12:54 user nova-compute[72351]:       <value>anonymous</value>
Aug 09 02:12:54 user nova-compute[72351]:       <value>memfd</value>
Aug 09 02:12:54 user nova-compute[72351]:     </enum>
Aug 09 02:12:54 user nova-compute[72351]:   </memoryBacking>
Aug 09 02:12:54 user nova-compute[72351]:   <devices>
Aug 09 02:12:54 user nova-compute[72351]:     <disk supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='diskDevice'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>disk</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>cdrom</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>floppy</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>lun</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='bus'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>ide</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>fdc</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>scsi</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>usb</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>sata</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='model'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio-transitional</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio-non-transitional</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </disk>
Aug 09 02:12:54 user nova-compute[72351]:     <graphics supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='type'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>sdl</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>vnc</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>spice</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>egl-headless</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </graphics>
Aug 09 02:12:54 user nova-compute[72351]:     <video supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='modelType'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>vga</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>cirrus</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>vmvga</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>qxl</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>none</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>bochs</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>ramfb</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </video>
Aug 09 02:12:54 user nova-compute[72351]:     <hostdev supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='mode'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>subsystem</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='startupPolicy'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>default</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>mandatory</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>requisite</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>optional</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='subsysType'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>usb</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>pci</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>scsi</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='capsType'/>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='pciBackend'/>
Aug 09 02:12:54 user nova-compute[72351]:     </hostdev>
Aug 09 02:12:54 user nova-compute[72351]:     <rng supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='model'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio-transitional</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio-non-transitional</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='backendModel'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>random</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>egd</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>builtin</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </rng>
Aug 09 02:12:54 user nova-compute[72351]:     <filesystem supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='driverType'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>path</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>handle</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtiofs</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </filesystem>
Aug 09 02:12:54 user nova-compute[72351]:     <tpm supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='model'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>tpm-tis</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>tpm-crb</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='backendModel'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>passthrough</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>emulator</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </tpm>
Aug 09 02:12:54 user nova-compute[72351]:   </devices>
Aug 09 02:12:54 user nova-compute[72351]:   <features>
Aug 09 02:12:54 user nova-compute[72351]:     <gic supported='no'/>
Aug 09 02:12:54 user nova-compute[72351]:     <vmcoreinfo supported='yes'/>
Aug 09 02:12:54 user nova-compute[72351]:     <genid supported='yes'/>
Aug 09 02:12:54 user nova-compute[72351]:     <backingStoreInput supported='yes'/>
Aug 09 02:12:54 user nova-compute[72351]:     <backup supported='yes'/>
Aug 09 02:12:54 user nova-compute[72351]:     <sev supported='no'/>
Aug 09 02:12:54 user nova-compute[72351]:   </features>
Aug 09 02:12:54 user nova-compute[72351]: </domainCapabilities>
Aug 09 02:12:54 user nova-compute[72351]:  {{(pid=72351) _get_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:1037}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Libvirt host hypervisor capabilities for arch=x86_64 and machine_type=q35:
Aug 09 02:12:54 user nova-compute[72351]: <domainCapabilities>
Aug 09 02:12:54 user nova-compute[72351]:   <path>/usr/bin/qemu-system-x86_64</path>
Aug 09 02:12:54 user nova-compute[72351]:   <domain>kvm</domain>
Aug 09 02:12:54 user nova-compute[72351]:   <machine>pc-q35-6.2</machine>
Aug 09 02:12:54 user nova-compute[72351]:   <arch>x86_64</arch>
Aug 09 02:12:54 user nova-compute[72351]:   <vcpu max='288'/>
Aug 09 02:12:54 user nova-compute[72351]:   <iothreads supported='yes'/>
Aug 09 02:12:54 user nova-compute[72351]:   <os supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:     <enum name='firmware'>
Aug 09 02:12:54 user nova-compute[72351]:       <value>efi</value>
Aug 09 02:12:54 user nova-compute[72351]:     </enum>
Aug 09 02:12:54 user nova-compute[72351]:     <loader supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <value>/usr/share/OVMF/OVMF_CODE_4M.ms.fd</value>
Aug 09 02:12:54 user nova-compute[72351]:       <value>/usr/share/OVMF/OVMF_CODE_4M.secboot.fd</value>
Aug 09 02:12:54 user nova-compute[72351]:       <value>/usr/share/OVMF/OVMF_CODE_4M.fd</value>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='type'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>rom</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>pflash</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='readonly'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>yes</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>no</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='secure'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>yes</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>no</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </loader>
Aug 09 02:12:54 user nova-compute[72351]:   </os>
Aug 09 02:12:54 user nova-compute[72351]:   <cpu>
Aug 09 02:12:54 user nova-compute[72351]:     <mode name='host-passthrough' supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='hostPassthroughMigratable'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>on</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>off</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </mode>
Aug 09 02:12:54 user nova-compute[72351]:     <mode name='maximum' supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='maximumMigratable'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>on</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>off</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </mode>
Aug 09 02:12:54 user nova-compute[72351]:     <mode name='host-model' supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <model fallback='forbid'>IvyBridge-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <vendor>Intel</vendor>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='ss'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='vmx'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='pcid'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='hypervisor'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='arat'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='tsc_adjust'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='umip'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='md-clear'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='stibp'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='arch-capabilities'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='ssbd'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='invtsc'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='ibpb'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='ibrs'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='amd-stibp'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='amd-ssbd'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='rsba'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='skip-l1dfl-vmentry'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='require' name='pschange-mc-no'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='disable' name='erms'/>
Aug 09 02:12:54 user nova-compute[72351]:       <feature policy='disable' name='xsaveopt'/>
Aug 09 02:12:54 user nova-compute[72351]:     </mode>
Aug 09 02:12:54 user nova-compute[72351]:     <mode name='custom' supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>qemu64</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>qemu32</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>phenom</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>pentium3</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>pentium2</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>pentium</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>n270</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>kvm64</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>kvm32</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>coreduo</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>core2duo</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>athlon</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>Westmere-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>Westmere</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Snowridge</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Skylake-Server-noTSX-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Skylake-Server-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Skylake-Server</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Skylake-Client-noTSX-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Skylake-Client-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Skylake-Client</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>SandyBridge-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>SandyBridge</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>Penryn</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Opteron_G5</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Opteron_G4</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Opteron_G3</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Opteron_G2</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>Opteron_G1</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>Nehalem-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>Nehalem</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>IvyBridge-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>IvyBridge</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Icelake-Server-noTSX</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Icelake-Server</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no' deprecated='yes'>Icelake-Client-noTSX</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no' deprecated='yes'>Icelake-Client</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Haswell-noTSX-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Haswell-noTSX</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Haswell-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Haswell</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>EPYC-Rome</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>EPYC-Milan</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>EPYC-IBPB</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>EPYC</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Dhyana</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Cooperlake</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>Conroe</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Cascadelake-Server-noTSX</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Cascadelake-Server</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Broadwell-noTSX-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Broadwell-noTSX</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Broadwell-IBRS</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='no'>Broadwell</model>
Aug 09 02:12:54 user nova-compute[72351]:       <model usable='yes'>486</model>
Aug 09 02:12:54 user nova-compute[72351]:     </mode>
Aug 09 02:12:54 user nova-compute[72351]:   </cpu>
Aug 09 02:12:54 user nova-compute[72351]:   <memoryBacking supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:     <enum name='sourceType'>
Aug 09 02:12:54 user nova-compute[72351]:       <value>file</value>
Aug 09 02:12:54 user nova-compute[72351]:       <value>anonymous</value>
Aug 09 02:12:54 user nova-compute[72351]:       <value>memfd</value>
Aug 09 02:12:54 user nova-compute[72351]:     </enum>
Aug 09 02:12:54 user nova-compute[72351]:   </memoryBacking>
Aug 09 02:12:54 user nova-compute[72351]:   <devices>
Aug 09 02:12:54 user nova-compute[72351]:     <disk supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='diskDevice'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>disk</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>cdrom</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>floppy</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>lun</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='bus'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>fdc</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>scsi</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>usb</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>sata</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='model'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio-transitional</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio-non-transitional</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </disk>
Aug 09 02:12:54 user nova-compute[72351]:     <graphics supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='type'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>sdl</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>vnc</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>spice</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>egl-headless</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </graphics>
Aug 09 02:12:54 user nova-compute[72351]:     <video supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='modelType'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>vga</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>cirrus</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>vmvga</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>qxl</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>none</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>bochs</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>ramfb</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </video>
Aug 09 02:12:54 user nova-compute[72351]:     <hostdev supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='mode'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>subsystem</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='startupPolicy'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>default</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>mandatory</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>requisite</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>optional</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='subsysType'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>usb</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>pci</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>scsi</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='capsType'/>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='pciBackend'/>
Aug 09 02:12:54 user nova-compute[72351]:     </hostdev>
Aug 09 02:12:54 user nova-compute[72351]:     <rng supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='model'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio-transitional</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtio-non-transitional</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='backendModel'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>random</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>egd</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>builtin</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </rng>
Aug 09 02:12:54 user nova-compute[72351]:     <filesystem supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='driverType'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>path</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>handle</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>virtiofs</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </filesystem>
Aug 09 02:12:54 user nova-compute[72351]:     <tpm supported='yes'>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='model'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>tpm-tis</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>tpm-crb</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:       <enum name='backendModel'>
Aug 09 02:12:54 user nova-compute[72351]:         <value>passthrough</value>
Aug 09 02:12:54 user nova-compute[72351]:         <value>emulator</value>
Aug 09 02:12:54 user nova-compute[72351]:       </enum>
Aug 09 02:12:54 user nova-compute[72351]:     </tpm>
Aug 09 02:12:54 user nova-compute[72351]:   </devices>
Aug 09 02:12:54 user nova-compute[72351]:   <features>
Aug 09 02:12:54 user nova-compute[72351]:     <gic supported='no'/>
Aug 09 02:12:54 user nova-compute[72351]:     <vmcoreinfo supported='yes'/>
Aug 09 02:12:54 user nova-compute[72351]:     <genid supported='yes'/>
Aug 09 02:12:54 user nova-compute[72351]:     <backingStoreInput supported='yes'/>
Aug 09 02:12:54 user nova-compute[72351]:     <backup supported='yes'/>
Aug 09 02:12:54 user nova-compute[72351]:     <sev supported='no'/>
Aug 09 02:12:54 user nova-compute[72351]:   </features>
Aug 09 02:12:54 user nova-compute[72351]: </domainCapabilities>
Aug 09 02:12:54 user nova-compute[72351]:  {{(pid=72351) _get_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:1037}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Getting domain capabilities for xtensa via machine types: {None} {{(pid=72351) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Error from libvirt when retrieving domain capabilities for arch xtensa / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-xtensa' on this host {{(pid=72351) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Getting domain capabilities for xtensaeb via machine types: {None} {{(pid=72351) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Error from libvirt when retrieving domain capabilities for arch xtensaeb / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-xtensaeb' on this host {{(pid=72351) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Checking secure boot support for host arch (x86_64) {{(pid=72351) supports_secure_boot /opt/stack/nova/nova/virt/libvirt/host.py:1779}}
Aug 09 02:12:54 user nova-compute[72351]: INFO nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Secure Boot support detected
Aug 09 02:12:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] cpu compare xml: <cpu match="exact">
Aug 09 02:12:54 user nova-compute[72351]:   <model>Nehalem</model>
Aug 09 02:12:54 user nova-compute[72351]: </cpu>
Aug 09 02:12:54 user nova-compute[72351]:  {{(pid=72351) _compare_cpu /opt/stack/nova/nova/virt/libvirt/driver.py:10023}}
Aug 09 02:12:54 user nova-compute[72351]: INFO nova.virt.node [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Generated node identity 8eae9b37-1236-4e6d-91a5-b650de27b009
Aug 09 02:12:54 user nova-compute[72351]: INFO nova.virt.node [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Wrote node identity 8eae9b37-1236-4e6d-91a5-b650de27b009 to /opt/stack/data/nova/compute_id
Aug 09 02:12:54 user nova-compute[72351]: WARNING nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Compute nodes ['8eae9b37-1236-4e6d-91a5-b650de27b009'] for host user were not found in the database. If this is the first time this service is starting on this host, then you can ignore this warning.
Aug 09 02:12:55 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Looking for unclaimed instances stuck in BUILDING status for nodes managed by this host
Aug 09 02:12:55 user nova-compute[72351]: WARNING nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] No compute node record found for host user. If this is the first time this service is starting on this host, then you can ignore this warning.: nova.exception_Remote.ComputeHostNotFound_Remote: Compute host user could not be found.
Aug 09 02:12:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:12:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:12:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:12:55 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Auditing locally available compute resources for user (node: user) {{(pid=72351) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Aug 09 02:12:55 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:12:55 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:12:55 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Hypervisor/Node resource view: name=user free_ram=10847MB free_disk=26.32941436767578GB free_vcpus=12 pci_devices=[{"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72351) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Aug 09 02:12:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:12:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:12:55 user nova-compute[72351]: WARNING nova.compute.resource_tracker [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] No compute node record for user:8eae9b37-1236-4e6d-91a5-b650de27b009: nova.exception_Remote.ComputeHostNotFound_Remote: Compute host 8eae9b37-1236-4e6d-91a5-b650de27b009 could not be found.
Aug 09 02:12:55 user nova-compute[72351]: INFO nova.compute.resource_tracker [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Compute node record created for user:user with uuid: 8eae9b37-1236-4e6d-91a5-b650de27b009
Aug 09 02:12:55 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Total usable vcpus: 12, total allocated vcpus: 0 {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Aug 09 02:12:55 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Final resource view: name=user phys_ram=16011MB used_ram=512MB phys_disk=40GB used_disk=0GB total_vcpus=12 used_vcpus=0 pci_stats=[] {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Aug 09 02:12:55 user nova-compute[72351]: INFO nova.scheduler.client.report [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [req-336f2e90-a43e-4486-8a8e-c5ff2e82591b] Created resource provider record via placement API for resource provider with UUID 8eae9b37-1236-4e6d-91a5-b650de27b009 and name user.
Aug 09 02:12:55 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] /sys/module/kvm_amd/parameters/sev does not exist {{(pid=72351) _kernel_supports_amd_sev /opt/stack/nova/nova/virt/libvirt/host.py:1795}}
Aug 09 02:12:55 user nova-compute[72351]: INFO nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] kernel doesn't support AMD SEV
Aug 09 02:12:55 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Updating inventory in ProviderTree for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 with inventory: {'MEMORY_MB': {'total': 16011, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0, 'reserved': 512}, 'VCPU': {'total': 12, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0, 'reserved': 0}, 'DISK_GB': {'total': 40, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0, 'reserved': 0}} {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:176}}
Aug 09 02:12:55 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:12:55 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Libvirt baseline CPU <cpu>
Aug 09 02:12:55 user nova-compute[72351]:   <arch>x86_64</arch>
Aug 09 02:12:55 user nova-compute[72351]:   <model>Nehalem</model>
Aug 09 02:12:55 user nova-compute[72351]:   <vendor>Intel</vendor>
Aug 09 02:12:55 user nova-compute[72351]:   <topology sockets="6" cores="1" threads="1"/>
Aug 09 02:12:55 user nova-compute[72351]: </cpu>
Aug 09 02:12:55 user nova-compute[72351]:  {{(pid=72351) _get_guest_baseline_cpu_features /opt/stack/nova/nova/virt/libvirt/driver.py:12513}}
Aug 09 02:12:55 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Updated inventory for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 with generation 0 in Placement from set_inventory_for_provider using data: {'MEMORY_MB': {'total': 16011, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0, 'reserved': 512}, 'VCPU': {'total': 12, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0, 'reserved': 0}, 'DISK_GB': {'total': 40, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0, 'reserved': 0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:957}}
Aug 09 02:12:55 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Updating resource provider 8eae9b37-1236-4e6d-91a5-b650de27b009 generation from 0 to 1 during operation: update_inventory {{(pid=72351) _update_generation /opt/stack/nova/nova/compute/provider_tree.py:164}}
Aug 09 02:12:55 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Updating inventory in ProviderTree for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 with inventory: {'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:176}}
Aug 09 02:12:55 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Updating resource provider 8eae9b37-1236-4e6d-91a5-b650de27b009 generation from 1 to 2 during operation: update_traits {{(pid=72351) _update_generation /opt/stack/nova/nova/compute/provider_tree.py:164}}
Aug 09 02:12:55 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Compute_service record updated for user:user {{(pid=72351) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Aug 09 02:12:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.598s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:12:55 user nova-compute[72351]: DEBUG nova.service [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Creating RPC server for service compute {{(pid=72351) start /opt/stack/nova/nova/service.py:182}}
Aug 09 02:12:55 user nova-compute[72351]: DEBUG nova.service [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Join ServiceGroup membership for this service compute {{(pid=72351) start /opt/stack/nova/nova/service.py:199}}
Aug 09 02:12:55 user nova-compute[72351]: DEBUG nova.servicegroup.drivers.db [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] DB_Driver: join new ServiceGroup member user to the compute group, service = <Service: host=user, binary=nova-compute, manager_class_name=nova.compute.manager.ComputeManager> {{(pid=72351) join /opt/stack/nova/nova/servicegroup/drivers/db.py:44}}
Aug 09 02:13:18 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._sync_power_states {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:13:18 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._cleanup_running_deleted_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:13:46 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:13:46 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:13:46 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Starting heal instance info cache {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9842}}
Aug 09 02:13:46 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Rebuilding the list of instances to heal {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9846}}
Aug 09 02:13:46 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Didn't find any instances for network info cache update. {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9928}}
Aug 09 02:13:46 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:13:46 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:13:46 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:13:46 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:13:46 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:13:46 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:13:46 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72351) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10461}}
Aug 09 02:13:46 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:13:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:13:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:13:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:13:46 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Auditing locally available compute resources for user (node: user) {{(pid=72351) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Aug 09 02:13:47 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:13:47 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:13:47 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Hypervisor/Node resource view: name=user free_ram=10222MB free_disk=26.23316192626953GB free_vcpus=12 pci_devices=[{"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72351) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Aug 09 02:13:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:13:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:13:47 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Total usable vcpus: 12, total allocated vcpus: 0 {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Aug 09 02:13:47 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Final resource view: name=user phys_ram=16011MB used_ram=512MB phys_disk=40GB used_disk=0GB total_vcpus=12 used_vcpus=0 pci_stats=[] {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Aug 09 02:13:47 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:13:47 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:13:47 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Compute_service record updated for user:user {{(pid=72351) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Aug 09 02:13:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.143s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:14:47 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:14:47 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:14:47 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:14:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:14:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:14:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:14:47 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Auditing locally available compute resources for user (node: user) {{(pid=72351) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Aug 09 02:14:47 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:14:47 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:14:47 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Hypervisor/Node resource view: name=user free_ram=10193MB free_disk=26.278217315673828GB free_vcpus=12 pci_devices=[{"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72351) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Aug 09 02:14:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:14:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:14:47 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Total usable vcpus: 12, total allocated vcpus: 0 {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Aug 09 02:14:47 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Final resource view: name=user phys_ram=16011MB used_ram=512MB phys_disk=40GB used_disk=0GB total_vcpus=12 used_vcpus=0 pci_stats=[] {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Aug 09 02:14:47 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:14:47 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:14:47 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Compute_service record updated for user:user {{(pid=72351) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Aug 09 02:14:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.096s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:14:48 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:14:48 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Starting heal instance info cache {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9842}}
Aug 09 02:14:48 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Rebuilding the list of instances to heal {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9846}}
Aug 09 02:14:48 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Didn't find any instances for network info cache update. {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9928}}
Aug 09 02:14:48 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:14:48 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:14:48 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:14:48 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72351) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10461}}
Aug 09 02:14:48 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:14:48 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:14:48 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:15:47 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:15:48 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:15:48 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:15:48 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Starting heal instance info cache {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9842}}
Aug 09 02:15:48 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Rebuilding the list of instances to heal {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9846}}
Aug 09 02:15:48 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Didn't find any instances for network info cache update. {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9928}}
Aug 09 02:15:48 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:15:48 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:15:48 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72351) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10461}}
Aug 09 02:15:48 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:15:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:15:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:15:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:15:48 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Auditing locally available compute resources for user (node: user) {{(pid=72351) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Aug 09 02:15:49 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:15:49 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:15:49 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Hypervisor/Node resource view: name=user free_ram=10169MB free_disk=26.05223846435547GB free_vcpus=12 pci_devices=[{"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72351) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Aug 09 02:15:49 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:15:49 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:15:49 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Total usable vcpus: 12, total allocated vcpus: 0 {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Aug 09 02:15:49 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Final resource view: name=user phys_ram=16011MB used_ram=512MB phys_disk=40GB used_disk=0GB total_vcpus=12 used_vcpus=0 pci_stats=[] {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Aug 09 02:15:49 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:15:49 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:15:49 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Compute_service record updated for user:user {{(pid=72351) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Aug 09 02:15:49 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.166s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:15:50 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:15:50 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:15:50 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:16:48 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:16:49 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:16:49 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:16:49 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:16:49 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72351) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10461}}
Aug 09 02:16:49 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:16:49 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:16:49 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:16:49 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:16:49 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Auditing locally available compute resources for user (node: user) {{(pid=72351) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Aug 09 02:16:50 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:16:50 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:16:50 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Hypervisor/Node resource view: name=user free_ram=9424MB free_disk=26.073814392089844GB free_vcpus=12 pci_devices=[{"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72351) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Aug 09 02:16:50 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:16:50 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:16:50 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Total usable vcpus: 12, total allocated vcpus: 0 {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Aug 09 02:16:50 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Final resource view: name=user phys_ram=16011MB used_ram=512MB phys_disk=40GB used_disk=0GB total_vcpus=12 used_vcpus=0 pci_stats=[] {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Aug 09 02:16:50 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:16:50 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:16:50 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Compute_service record updated for user:user {{(pid=72351) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Aug 09 02:16:50 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.201s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:16:51 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:16:51 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:16:51 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Starting heal instance info cache {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9842}}
Aug 09 02:16:51 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Rebuilding the list of instances to heal {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9846}}
Aug 09 02:16:51 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Didn't find any instances for network info cache update. {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9928}}
Aug 09 02:16:51 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:16:51 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:16:51 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:16:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Acquiring lock "f45b57fa-9264-45f0-9573-e5fb58848d1f" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:16:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Lock "f45b57fa-9264-45f0-9573-e5fb58848d1f" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:16:55 user nova-compute[72351]: DEBUG nova.compute.manager [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Starting instance... {{(pid=72351) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2406}}
Aug 09 02:16:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:16:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:16:55 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72351) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Aug 09 02:16:55 user nova-compute[72351]: INFO nova.compute.claims [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Claim successful on node user
Aug 09 02:16:55 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:16:55 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:16:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.331s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:16:55 user nova-compute[72351]: DEBUG nova.compute.manager [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Start building networks asynchronously for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2803}}
Aug 09 02:16:55 user nova-compute[72351]: DEBUG nova.compute.manager [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Allocating IP information in the background. {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1956}}
Aug 09 02:16:55 user nova-compute[72351]: DEBUG nova.network.neutron [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] allocate_for_instance() {{(pid=72351) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Aug 09 02:16:55 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Aug 09 02:16:55 user nova-compute[72351]: DEBUG nova.compute.manager [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Start building block device mappings for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2838}}
Aug 09 02:16:55 user nova-compute[72351]: DEBUG nova.compute.manager [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Start spawning the instance on the hypervisor. {{(pid=72351) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2612}}
Aug 09 02:16:55 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Creating instance directory {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4717}}
Aug 09 02:16:55 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Creating image(s)
Aug 09 02:16:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Acquiring lock "/opt/stack/data/nova/instances/f45b57fa-9264-45f0-9573-e5fb58848d1f/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:16:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Lock "/opt/stack/data/nova/instances/f45b57fa-9264-45f0-9573-e5fb58848d1f/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:16:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Lock "/opt/stack/data/nova/instances/f45b57fa-9264-45f0-9573-e5fb58848d1f/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:16:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Acquiring lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:16:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" acquired by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:16:56 user nova-compute[72351]: DEBUG nova.policy [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '77f49e6422b24e83887b084bc2c8e727', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e31482d532bf425386728e29c8c90f19', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72351) authorize /opt/stack/nova/nova/policy.py:203}}
Aug 09 02:16:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd.part --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:16:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd.part --force-share --output=json" returned: 0 in 0.195s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:16:57 user nova-compute[72351]: DEBUG nova.virt.images [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] 18ccda7f-31d3-4978-ab9d-4ed2f74eff94 was qcow2, converting to raw {{(pid=72351) fetch_to_raw /opt/stack/nova/nova/virt/images.py:165}}
Aug 09 02:16:57 user nova-compute[72351]: DEBUG nova.privsep.utils [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=72351) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Aug 09 02:16:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Running cmd (subprocess): qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd.part /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd.converted {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:16:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] CMD "qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd.part /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd.converted" returned: 0 in 0.181s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:16:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd.converted --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:16:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd.converted --force-share --output=json" returned: 0 in 0.139s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:16:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" "released" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: held 1.955s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:16:57 user nova-compute[72351]: INFO oslo.privsep.daemon [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Running privsep helper: ['sudo', 'nova-rootwrap', '/etc/nova/rootwrap.conf', 'privsep-helper', '--config-file', '/etc/nova/nova-cpu.conf', '--privsep_context', 'nova.privsep.sys_admin_pctxt', '--privsep_sock_path', '/tmp/tmp5t3e7scy/privsep.sock']
Aug 09 02:16:57 user sudo[81489]:    stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/nova-rootwrap /etc/nova/rootwrap.conf privsep-helper --config-file /etc/nova/nova-cpu.conf --privsep_context nova.privsep.sys_admin_pctxt --privsep_sock_path /tmp/tmp5t3e7scy/privsep.sock
Aug 09 02:16:57 user sudo[81489]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=1001)
Aug 09 02:16:58 user nova-compute[72351]: DEBUG nova.network.neutron [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Successfully created port: e9a7bda3-90fa-4274-acc9-3597835503e9 {{(pid=72351) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Aug 09 02:16:59 user sudo[81489]: pam_unix(sudo:session): session closed for user root
Aug 09 02:16:59 user nova-compute[72351]: INFO oslo.privsep.daemon [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Spawned new privsep daemon via rootwrap
Aug 09 02:16:59 user nova-compute[72351]: INFO oslo.privsep.daemon [-] privsep daemon starting
Aug 09 02:16:59 user nova-compute[72351]: INFO oslo.privsep.daemon [-] privsep process running with uid/gid: 0/0
Aug 09 02:16:59 user nova-compute[72351]: INFO oslo.privsep.daemon [-] privsep process running with capabilities (eff/prm/inh): CAP_CHOWN|CAP_DAC_OVERRIDE|CAP_DAC_READ_SEARCH|CAP_FOWNER|CAP_NET_ADMIN|CAP_SYS_ADMIN/CAP_CHOWN|CAP_DAC_OVERRIDE|CAP_DAC_READ_SEARCH|CAP_FOWNER|CAP_NET_ADMIN|CAP_SYS_ADMIN/none
Aug 09 02:16:59 user nova-compute[72351]: INFO oslo.privsep.daemon [-] privsep daemon running as pid 81494
Aug 09 02:17:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:17:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.150s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:17:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Acquiring lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:17:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:17:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:17:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.161s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:17:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/f45b57fa-9264-45f0-9573-e5fb58848d1f/disk 1073741824 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:17:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/f45b57fa-9264-45f0-9573-e5fb58848d1f/disk 1073741824" returned: 0 in 0.045s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:17:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.211s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:17:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:17:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.135s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:17:00 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Checking if we can resize image /opt/stack/data/nova/instances/f45b57fa-9264-45f0-9573-e5fb58848d1f/disk. size=1073741824 {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Aug 09 02:17:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f45b57fa-9264-45f0-9573-e5fb58848d1f/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:17:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f45b57fa-9264-45f0-9573-e5fb58848d1f/disk --force-share --output=json" returned: 0 in 0.124s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:17:00 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Cannot resize image /opt/stack/data/nova/instances/f45b57fa-9264-45f0-9573-e5fb58848d1f/disk to a smaller size. {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Aug 09 02:17:00 user nova-compute[72351]: DEBUG nova.objects.instance [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Lazy-loading 'migration_context' on Instance uuid f45b57fa-9264-45f0-9573-e5fb58848d1f {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:17:00 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Created local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4851}}
Aug 09 02:17:00 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Ensure instance console log exists: /opt/stack/data/nova/instances/f45b57fa-9264-45f0-9573-e5fb58848d1f/console.log {{(pid=72351) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4603}}
Aug 09 02:17:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:17:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:17:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:17:02 user nova-compute[72351]: DEBUG nova.network.neutron [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Successfully updated port: e9a7bda3-90fa-4274-acc9-3597835503e9 {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:17:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Acquiring lock "refresh_cache-f45b57fa-9264-45f0-9573-e5fb58848d1f" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:17:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Acquired lock "refresh_cache-f45b57fa-9264-45f0-9573-e5fb58848d1f" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:17:02 user nova-compute[72351]: DEBUG nova.network.neutron [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:17:02 user nova-compute[72351]: DEBUG nova.network.neutron [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:17:03 user nova-compute[72351]: DEBUG nova.compute.manager [req-a763fdd7-3d75-4de1-8451-bf06e9aee7b0 req-30eb1423-b768-4607-943f-01c13d1c30eb service nova] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Received event network-changed-e9a7bda3-90fa-4274-acc9-3597835503e9 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:17:03 user nova-compute[72351]: DEBUG nova.compute.manager [req-a763fdd7-3d75-4de1-8451-bf06e9aee7b0 req-30eb1423-b768-4607-943f-01c13d1c30eb service nova] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Refreshing instance network info cache due to event network-changed-e9a7bda3-90fa-4274-acc9-3597835503e9. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:17:03 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a763fdd7-3d75-4de1-8451-bf06e9aee7b0 req-30eb1423-b768-4607-943f-01c13d1c30eb service nova] Acquiring lock "refresh_cache-f45b57fa-9264-45f0-9573-e5fb58848d1f" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:17:04 user nova-compute[72351]: DEBUG nova.network.neutron [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Updating instance_info_cache with network_info: [{"id": "e9a7bda3-90fa-4274-acc9-3597835503e9", "address": "fa:16:3e:a0:ba:9c", "network": {"id": "518cac69-7290-48e2-8026-3e1a92d7b7fd", "bridge": "br-int", "label": "tempest-DeleteServersTestJSON-799055594-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e31482d532bf425386728e29c8c90f19", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape9a7bda3-90", "ovs_interfaceid": "e9a7bda3-90fa-4274-acc9-3597835503e9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:17:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Releasing lock "refresh_cache-f45b57fa-9264-45f0-9573-e5fb58848d1f" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:17:04 user nova-compute[72351]: DEBUG nova.compute.manager [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Instance network_info: |[{"id": "e9a7bda3-90fa-4274-acc9-3597835503e9", "address": "fa:16:3e:a0:ba:9c", "network": {"id": "518cac69-7290-48e2-8026-3e1a92d7b7fd", "bridge": "br-int", "label": "tempest-DeleteServersTestJSON-799055594-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e31482d532bf425386728e29c8c90f19", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape9a7bda3-90", "ovs_interfaceid": "e9a7bda3-90fa-4274-acc9-3597835503e9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1971}}
Aug 09 02:17:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a763fdd7-3d75-4de1-8451-bf06e9aee7b0 req-30eb1423-b768-4607-943f-01c13d1c30eb service nova] Acquired lock "refresh_cache-f45b57fa-9264-45f0-9573-e5fb58848d1f" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:17:04 user nova-compute[72351]: DEBUG nova.network.neutron [req-a763fdd7-3d75-4de1-8451-bf06e9aee7b0 req-30eb1423-b768-4607-943f-01c13d1c30eb service nova] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Refreshing network info cache for port e9a7bda3-90fa-4274-acc9-3597835503e9 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:17:04 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Start _get_guest_xml network_info=[{"id": "e9a7bda3-90fa-4274-acc9-3597835503e9", "address": "fa:16:3e:a0:ba:9c", "network": {"id": "518cac69-7290-48e2-8026-3e1a92d7b7fd", "bridge": "br-int", "label": "tempest-DeleteServersTestJSON-799055594-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e31482d532bf425386728e29c8c90f19", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape9a7bda3-90", "ovs_interfaceid": "e9a7bda3-90fa-4274-acc9-3597835503e9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'size': 0, 'device_type': 'disk', 'encryption_secret_uuid': None, 'guest_format': None, 'encryption_options': None, 'disk_bus': 'virtio', 'encrypted': False, 'encryption_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'image_id': '18ccda7f-31d3-4978-ab9d-4ed2f74eff94'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:17:04 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:17:04 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:17:04 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:17:04 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:17:04 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:17:04 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:17:04 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:17:04 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:17:04 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:17:04 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:17:04 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:17:04 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:17:04 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:17:04 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:17:04 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:17:04 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:17:04 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:17:04 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:17:04 user nova-compute[72351]: DEBUG nova.privsep.utils [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=72351) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Aug 09 02:17:04 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:16:53Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-DeleteServersTestJSON-server-1715605419',display_name='tempest-DeleteServersTestJSON-server-1715605419',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-deleteserverstestjson-server-1715605419',id=1,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='e31482d532bf425386728e29c8c90f19',ramdisk_id='',reservation_id='r-1qpnq7xe',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-DeleteServersTestJSON-845954460',owner_user_name='tempest-DeleteServersTestJSON-845954460-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:16:56Z,user_data=None,user_id='77f49e6422b24e83887b084bc2c8e727',uuid=f45b57fa-9264-45f0-9573-e5fb58848d1f,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "e9a7bda3-90fa-4274-acc9-3597835503e9", "address": "fa:16:3e:a0:ba:9c", "network": {"id": "518cac69-7290-48e2-8026-3e1a92d7b7fd", "bridge": "br-int", "label": "tempest-DeleteServersTestJSON-799055594-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e31482d532bf425386728e29c8c90f19", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape9a7bda3-90", "ovs_interfaceid": "e9a7bda3-90fa-4274-acc9-3597835503e9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:17:04 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Converting VIF {"id": "e9a7bda3-90fa-4274-acc9-3597835503e9", "address": "fa:16:3e:a0:ba:9c", "network": {"id": "518cac69-7290-48e2-8026-3e1a92d7b7fd", "bridge": "br-int", "label": "tempest-DeleteServersTestJSON-799055594-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e31482d532bf425386728e29c8c90f19", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape9a7bda3-90", "ovs_interfaceid": "e9a7bda3-90fa-4274-acc9-3597835503e9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:17:04 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:a0:ba:9c,bridge_name='br-int',has_traffic_filtering=True,id=e9a7bda3-90fa-4274-acc9-3597835503e9,network=Network(518cac69-7290-48e2-8026-3e1a92d7b7fd),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape9a7bda3-90') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:17:04 user nova-compute[72351]: DEBUG nova.objects.instance [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Lazy-loading 'pci_devices' on Instance uuid f45b57fa-9264-45f0-9573-e5fb58848d1f {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:17:04 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:17:04 user nova-compute[72351]:   <uuid>f45b57fa-9264-45f0-9573-e5fb58848d1f</uuid>
Aug 09 02:17:04 user nova-compute[72351]:   <name>instance-00000001</name>
Aug 09 02:17:04 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:17:04 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:17:04 user nova-compute[72351]:   <metadata>
Aug 09 02:17:04 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:17:04 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:17:04 user nova-compute[72351]:       <nova:name>tempest-DeleteServersTestJSON-server-1715605419</nova:name>
Aug 09 02:17:04 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:17:04</nova:creationTime>
Aug 09 02:17:04 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:17:04 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:17:04 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:17:04 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:17:04 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:17:04 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:17:04 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:17:04 user nova-compute[72351]:       <nova:owner>
Aug 09 02:17:04 user nova-compute[72351]:         <nova:user uuid="77f49e6422b24e83887b084bc2c8e727">tempest-DeleteServersTestJSON-845954460-project-member</nova:user>
Aug 09 02:17:04 user nova-compute[72351]:         <nova:project uuid="e31482d532bf425386728e29c8c90f19">tempest-DeleteServersTestJSON-845954460</nova:project>
Aug 09 02:17:04 user nova-compute[72351]:       </nova:owner>
Aug 09 02:17:04 user nova-compute[72351]:       <nova:root type="image" uuid="18ccda7f-31d3-4978-ab9d-4ed2f74eff94"/>
Aug 09 02:17:04 user nova-compute[72351]:       <nova:ports>
Aug 09 02:17:04 user nova-compute[72351]:         <nova:port uuid="e9a7bda3-90fa-4274-acc9-3597835503e9">
Aug 09 02:17:04 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.5" ipVersion="4"/>
Aug 09 02:17:04 user nova-compute[72351]:         </nova:port>
Aug 09 02:17:04 user nova-compute[72351]:       </nova:ports>
Aug 09 02:17:04 user nova-compute[72351]:     </nova:instance>
Aug 09 02:17:04 user nova-compute[72351]:   </metadata>
Aug 09 02:17:04 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:17:04 user nova-compute[72351]:     <system>
Aug 09 02:17:04 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:17:04 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:17:04 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:17:04 user nova-compute[72351]:       <entry name="serial">f45b57fa-9264-45f0-9573-e5fb58848d1f</entry>
Aug 09 02:17:04 user nova-compute[72351]:       <entry name="uuid">f45b57fa-9264-45f0-9573-e5fb58848d1f</entry>
Aug 09 02:17:04 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:17:04 user nova-compute[72351]:     </system>
Aug 09 02:17:04 user nova-compute[72351]:   </sysinfo>
Aug 09 02:17:04 user nova-compute[72351]:   <os>
Aug 09 02:17:04 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:17:04 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:17:04 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:17:04 user nova-compute[72351]:   </os>
Aug 09 02:17:04 user nova-compute[72351]:   <features>
Aug 09 02:17:04 user nova-compute[72351]:     <acpi/>
Aug 09 02:17:04 user nova-compute[72351]:     <apic/>
Aug 09 02:17:04 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:17:04 user nova-compute[72351]:   </features>
Aug 09 02:17:04 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:17:04 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:17:04 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:17:04 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:17:04 user nova-compute[72351]:   </clock>
Aug 09 02:17:04 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:17:04 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:17:04 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:17:04 user nova-compute[72351]:   </cpu>
Aug 09 02:17:04 user nova-compute[72351]:   <devices>
Aug 09 02:17:04 user nova-compute[72351]:     <disk type="file" device="disk">
Aug 09 02:17:04 user nova-compute[72351]:       <driver name="qemu" type="qcow2" cache="none"/>
Aug 09 02:17:04 user nova-compute[72351]:       <source file="/opt/stack/data/nova/instances/f45b57fa-9264-45f0-9573-e5fb58848d1f/disk"/>
Aug 09 02:17:04 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:17:04 user nova-compute[72351]:     </disk>
Aug 09 02:17:04 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:17:04 user nova-compute[72351]:       <mac address="fa:16:3e:a0:ba:9c"/>
Aug 09 02:17:04 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:17:04 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:17:04 user nova-compute[72351]:       <target dev="tape9a7bda3-90"/>
Aug 09 02:17:04 user nova-compute[72351]:     </interface>
Aug 09 02:17:04 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:17:04 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/f45b57fa-9264-45f0-9573-e5fb58848d1f/console.log" append="off"/>
Aug 09 02:17:04 user nova-compute[72351]:     </serial>
Aug 09 02:17:04 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:17:04 user nova-compute[72351]:     <video>
Aug 09 02:17:04 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:17:04 user nova-compute[72351]:     </video>
Aug 09 02:17:04 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:17:04 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:17:04 user nova-compute[72351]:     </rng>
Aug 09 02:17:04 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:17:04 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:17:04 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:17:04 user nova-compute[72351]:     </memballoon>
Aug 09 02:17:04 user nova-compute[72351]:   </devices>
Aug 09 02:17:04 user nova-compute[72351]: </domain>
Aug 09 02:17:04 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:17:04 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:16:53Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-DeleteServersTestJSON-server-1715605419',display_name='tempest-DeleteServersTestJSON-server-1715605419',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-deleteserverstestjson-server-1715605419',id=1,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='e31482d532bf425386728e29c8c90f19',ramdisk_id='',reservation_id='r-1qpnq7xe',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-DeleteServersTestJSON-845954460',owner_user_name='tempest-DeleteServersTestJSON-845954460-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:16:56Z,user_data=None,user_id='77f49e6422b24e83887b084bc2c8e727',uuid=f45b57fa-9264-45f0-9573-e5fb58848d1f,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "e9a7bda3-90fa-4274-acc9-3597835503e9", "address": "fa:16:3e:a0:ba:9c", "network": {"id": "518cac69-7290-48e2-8026-3e1a92d7b7fd", "bridge": "br-int", "label": "tempest-DeleteServersTestJSON-799055594-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e31482d532bf425386728e29c8c90f19", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape9a7bda3-90", "ovs_interfaceid": "e9a7bda3-90fa-4274-acc9-3597835503e9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:17:04 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Converting VIF {"id": "e9a7bda3-90fa-4274-acc9-3597835503e9", "address": "fa:16:3e:a0:ba:9c", "network": {"id": "518cac69-7290-48e2-8026-3e1a92d7b7fd", "bridge": "br-int", "label": "tempest-DeleteServersTestJSON-799055594-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e31482d532bf425386728e29c8c90f19", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape9a7bda3-90", "ovs_interfaceid": "e9a7bda3-90fa-4274-acc9-3597835503e9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:17:04 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:a0:ba:9c,bridge_name='br-int',has_traffic_filtering=True,id=e9a7bda3-90fa-4274-acc9-3597835503e9,network=Network(518cac69-7290-48e2-8026-3e1a92d7b7fd),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape9a7bda3-90') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:17:04 user nova-compute[72351]: DEBUG os_vif [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:a0:ba:9c,bridge_name='br-int',has_traffic_filtering=True,id=e9a7bda3-90fa-4274-acc9-3597835503e9,network=Network(518cac69-7290-48e2-8026-3e1a92d7b7fd),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape9a7bda3-90') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:17:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Created schema index Interface.name {{(pid=72351) autocreate_indices /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Aug 09 02:17:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Created schema index Port.name {{(pid=72351) autocreate_indices /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Aug 09 02:17:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Created schema index Bridge.name {{(pid=72351) autocreate_indices /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Aug 09 02:17:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] tcp:127.0.0.1:6640: entering CONNECTING {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:17:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] [POLLOUT] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:17:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:17:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:17:04 user nova-compute[72351]: INFO oslo.privsep.daemon [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Running privsep helper: ['sudo', 'nova-rootwrap', '/etc/nova/rootwrap.conf', 'privsep-helper', '--config-file', '/etc/nova/nova-cpu.conf', '--privsep_context', 'vif_plug_ovs.privsep.vif_plug', '--privsep_sock_path', '/tmp/tmps2stwkz5/privsep.sock']
Aug 09 02:17:04 user sudo[81518]:    stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/nova-rootwrap /etc/nova/rootwrap.conf privsep-helper --config-file /etc/nova/nova-cpu.conf --privsep_context vif_plug_ovs.privsep.vif_plug --privsep_sock_path /tmp/tmps2stwkz5/privsep.sock
Aug 09 02:17:04 user sudo[81518]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=1001)
Aug 09 02:17:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Acquiring lock "2d78475c-107d-4256-b12e-dbe9c8f3b9b2" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:17:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Lock "2d78475c-107d-4256-b12e-dbe9c8f3b9b2" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:17:05 user nova-compute[72351]: DEBUG nova.compute.manager [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Starting instance... {{(pid=72351) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2406}}
Aug 09 02:17:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:17:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:17:05 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72351) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Aug 09 02:17:05 user nova-compute[72351]: INFO nova.compute.claims [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Claim successful on node user
Aug 09 02:17:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:05 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:17:05 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:17:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.274s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:17:05 user nova-compute[72351]: DEBUG nova.compute.manager [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Start building networks asynchronously for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2803}}
Aug 09 02:17:05 user nova-compute[72351]: DEBUG nova.network.neutron [req-a763fdd7-3d75-4de1-8451-bf06e9aee7b0 req-30eb1423-b768-4607-943f-01c13d1c30eb service nova] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Updated VIF entry in instance network info cache for port e9a7bda3-90fa-4274-acc9-3597835503e9. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:17:05 user nova-compute[72351]: DEBUG nova.network.neutron [req-a763fdd7-3d75-4de1-8451-bf06e9aee7b0 req-30eb1423-b768-4607-943f-01c13d1c30eb service nova] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Updating instance_info_cache with network_info: [{"id": "e9a7bda3-90fa-4274-acc9-3597835503e9", "address": "fa:16:3e:a0:ba:9c", "network": {"id": "518cac69-7290-48e2-8026-3e1a92d7b7fd", "bridge": "br-int", "label": "tempest-DeleteServersTestJSON-799055594-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e31482d532bf425386728e29c8c90f19", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape9a7bda3-90", "ovs_interfaceid": "e9a7bda3-90fa-4274-acc9-3597835503e9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:17:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a763fdd7-3d75-4de1-8451-bf06e9aee7b0 req-30eb1423-b768-4607-943f-01c13d1c30eb service nova] Releasing lock "refresh_cache-f45b57fa-9264-45f0-9573-e5fb58848d1f" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:17:05 user nova-compute[72351]: DEBUG nova.compute.manager [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Allocating IP information in the background. {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1956}}
Aug 09 02:17:05 user nova-compute[72351]: DEBUG nova.network.neutron [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] allocate_for_instance() {{(pid=72351) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Aug 09 02:17:06 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:06 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Aug 09 02:17:06 user nova-compute[72351]: DEBUG nova.compute.manager [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Start building block device mappings for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2838}}
Aug 09 02:17:06 user nova-compute[72351]: DEBUG nova.compute.manager [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Start spawning the instance on the hypervisor. {{(pid=72351) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2612}}
Aug 09 02:17:06 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Creating instance directory {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4717}}
Aug 09 02:17:06 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Creating image(s)
Aug 09 02:17:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Acquiring lock "/opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:17:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Lock "/opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:17:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Lock "/opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:17:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:17:06 user nova-compute[72351]: DEBUG nova.policy [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '157ad7fa9970477a91de3d1347dee441', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '747d5750a2174aaea71849c70167a816', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72351) authorize /opt/stack/nova/nova/policy.py:203}}
Aug 09 02:17:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.184s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:17:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Acquiring lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:17:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:17:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:17:06 user sudo[81518]: pam_unix(sudo:session): session closed for user root
Aug 09 02:17:06 user nova-compute[72351]: INFO oslo.privsep.daemon [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Spawned new privsep daemon via rootwrap
Aug 09 02:17:06 user nova-compute[72351]: INFO oslo.privsep.daemon [-] privsep daemon starting
Aug 09 02:17:06 user nova-compute[72351]: INFO oslo.privsep.daemon [-] privsep process running with uid/gid: 0/0
Aug 09 02:17:06 user nova-compute[72351]: INFO oslo.privsep.daemon [-] privsep process running with capabilities (eff/prm/inh): CAP_DAC_OVERRIDE|CAP_NET_ADMIN/CAP_DAC_OVERRIDE|CAP_NET_ADMIN/none
Aug 09 02:17:06 user nova-compute[72351]: INFO oslo.privsep.daemon [-] privsep daemon running as pid 81527
Aug 09 02:17:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.136s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:17:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2/disk 1073741824 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:17:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2/disk 1073741824" returned: 0 in 0.053s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:17:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.194s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:17:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:17:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.155s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:17:06 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Checking if we can resize image /opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2/disk. size=1073741824 {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Aug 09 02:17:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:17:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2/disk --force-share --output=json" returned: 0 in 0.129s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:17:06 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Cannot resize image /opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2/disk to a smaller size. {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Aug 09 02:17:06 user nova-compute[72351]: DEBUG nova.objects.instance [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Lazy-loading 'migration_context' on Instance uuid 2d78475c-107d-4256-b12e-dbe9c8f3b9b2 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:17:07 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Created local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4851}}
Aug 09 02:17:07 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Ensure instance console log exists: /opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2/console.log {{(pid=72351) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4603}}
Aug 09 02:17:07 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:17:07 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:17:07 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:17:07 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:07 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tape9a7bda3-90, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:17:07 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tape9a7bda3-90, col_values=(('external_ids', {'iface-id': 'e9a7bda3-90fa-4274-acc9-3597835503e9', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:a0:ba:9c', 'vm-uuid': 'f45b57fa-9264-45f0-9573-e5fb58848d1f'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:17:07 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:07 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:17:07 user nova-compute[72351]: INFO os_vif [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:a0:ba:9c,bridge_name='br-int',has_traffic_filtering=True,id=e9a7bda3-90fa-4274-acc9-3597835503e9,network=Network(518cac69-7290-48e2-8026-3e1a92d7b7fd),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape9a7bda3-90')
Aug 09 02:17:07 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:17:07 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] No VIF found with MAC fa:16:3e:a0:ba:9c, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:17:07 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:08 user nova-compute[72351]: DEBUG nova.network.neutron [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Successfully created port: ff25eb15-1af0-4110-9d9a-ecd0a1d4627d {{(pid=72351) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Aug 09 02:17:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:09 user nova-compute[72351]: DEBUG nova.compute.manager [req-bb23c125-4906-499a-ab73-dca546bef499 req-93f1b56e-8b55-4424-99c8-8b74f5b9d6e6 service nova] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Received event network-vif-plugged-e9a7bda3-90fa-4274-acc9-3597835503e9 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:17:09 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-bb23c125-4906-499a-ab73-dca546bef499 req-93f1b56e-8b55-4424-99c8-8b74f5b9d6e6 service nova] Acquiring lock "f45b57fa-9264-45f0-9573-e5fb58848d1f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:17:09 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-bb23c125-4906-499a-ab73-dca546bef499 req-93f1b56e-8b55-4424-99c8-8b74f5b9d6e6 service nova] Lock "f45b57fa-9264-45f0-9573-e5fb58848d1f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:17:09 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-bb23c125-4906-499a-ab73-dca546bef499 req-93f1b56e-8b55-4424-99c8-8b74f5b9d6e6 service nova] Lock "f45b57fa-9264-45f0-9573-e5fb58848d1f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:17:09 user nova-compute[72351]: DEBUG nova.compute.manager [req-bb23c125-4906-499a-ab73-dca546bef499 req-93f1b56e-8b55-4424-99c8-8b74f5b9d6e6 service nova] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] No waiting events found dispatching network-vif-plugged-e9a7bda3-90fa-4274-acc9-3597835503e9 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:17:09 user nova-compute[72351]: WARNING nova.compute.manager [req-bb23c125-4906-499a-ab73-dca546bef499 req-93f1b56e-8b55-4424-99c8-8b74f5b9d6e6 service nova] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Received unexpected event network-vif-plugged-e9a7bda3-90fa-4274-acc9-3597835503e9 for instance with vm_state building and task_state spawning.
Aug 09 02:17:10 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547430.5456476, f45b57fa-9264-45f0-9573-e5fb58848d1f => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:17:10 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] VM Resumed (Lifecycle Event)
Aug 09 02:17:10 user nova-compute[72351]: DEBUG nova.compute.manager [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:17:10 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Guest created on hypervisor {{(pid=72351) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4411}}
Aug 09 02:17:10 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Instance spawned successfully.
Aug 09 02:17:10 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:894}}
Aug 09 02:17:10 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Found default for hw_cdrom_bus of ide {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:17:10 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Found default for hw_disk_bus of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:17:10 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Found default for hw_input_bus of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:17:10 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Found default for hw_pointer_model of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:17:10 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Found default for hw_video_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:17:10 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Found default for hw_vif_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:17:10 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:17:10 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:17:10 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:17:10 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547430.546071, f45b57fa-9264-45f0-9573-e5fb58848d1f => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:17:10 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] VM Started (Lifecycle Event)
Aug 09 02:17:10 user nova-compute[72351]: INFO nova.compute.manager [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Took 14.77 seconds to spawn the instance on the hypervisor.
Aug 09 02:17:10 user nova-compute[72351]: DEBUG nova.compute.manager [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:17:10 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:17:10 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:17:10 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:17:10 user nova-compute[72351]: INFO nova.compute.manager [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Took 15.65 seconds to build instance.
Aug 09 02:17:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-9f2b8d9a-0fc4-42a7-b243-0b9b48d699f3 tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Lock "f45b57fa-9264-45f0-9573-e5fb58848d1f" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 15.950s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:17:11 user nova-compute[72351]: DEBUG nova.network.neutron [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Successfully updated port: ff25eb15-1af0-4110-9d9a-ecd0a1d4627d {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:17:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Acquiring lock "refresh_cache-2d78475c-107d-4256-b12e-dbe9c8f3b9b2" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:17:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Acquired lock "refresh_cache-2d78475c-107d-4256-b12e-dbe9c8f3b9b2" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:17:11 user nova-compute[72351]: DEBUG nova.network.neutron [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:17:11 user nova-compute[72351]: DEBUG nova.compute.manager [req-989b17db-2e56-4978-90b4-b247573796d7 req-9ea47648-cd8a-4494-b8d8-1eb670f08188 service nova] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Received event network-vif-plugged-e9a7bda3-90fa-4274-acc9-3597835503e9 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:17:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-989b17db-2e56-4978-90b4-b247573796d7 req-9ea47648-cd8a-4494-b8d8-1eb670f08188 service nova] Acquiring lock "f45b57fa-9264-45f0-9573-e5fb58848d1f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:17:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-989b17db-2e56-4978-90b4-b247573796d7 req-9ea47648-cd8a-4494-b8d8-1eb670f08188 service nova] Lock "f45b57fa-9264-45f0-9573-e5fb58848d1f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:17:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-989b17db-2e56-4978-90b4-b247573796d7 req-9ea47648-cd8a-4494-b8d8-1eb670f08188 service nova] Lock "f45b57fa-9264-45f0-9573-e5fb58848d1f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:17:11 user nova-compute[72351]: DEBUG nova.compute.manager [req-989b17db-2e56-4978-90b4-b247573796d7 req-9ea47648-cd8a-4494-b8d8-1eb670f08188 service nova] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] No waiting events found dispatching network-vif-plugged-e9a7bda3-90fa-4274-acc9-3597835503e9 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:17:11 user nova-compute[72351]: WARNING nova.compute.manager [req-989b17db-2e56-4978-90b4-b247573796d7 req-9ea47648-cd8a-4494-b8d8-1eb670f08188 service nova] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Received unexpected event network-vif-plugged-e9a7bda3-90fa-4274-acc9-3597835503e9 for instance with vm_state active and task_state None.
Aug 09 02:17:12 user nova-compute[72351]: DEBUG nova.network.neutron [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:17:12 user nova-compute[72351]: DEBUG nova.compute.manager [req-4fe566ab-b3fd-4616-9d94-dc40697022c5 req-7c50c9b8-67ed-4d12-9664-462365fe221c service nova] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Received event network-changed-ff25eb15-1af0-4110-9d9a-ecd0a1d4627d {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:17:12 user nova-compute[72351]: DEBUG nova.compute.manager [req-4fe566ab-b3fd-4616-9d94-dc40697022c5 req-7c50c9b8-67ed-4d12-9664-462365fe221c service nova] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Refreshing instance network info cache due to event network-changed-ff25eb15-1af0-4110-9d9a-ecd0a1d4627d. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:17:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-4fe566ab-b3fd-4616-9d94-dc40697022c5 req-7c50c9b8-67ed-4d12-9664-462365fe221c service nova] Acquiring lock "refresh_cache-2d78475c-107d-4256-b12e-dbe9c8f3b9b2" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:17:12 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:12 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:12 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:12 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:13 user nova-compute[72351]: DEBUG nova.network.neutron [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Updating instance_info_cache with network_info: [{"id": "ff25eb15-1af0-4110-9d9a-ecd0a1d4627d", "address": "fa:16:3e:0a:89:6a", "network": {"id": "fd1fe4e0-5d09-43c1-8dbc-1f60cb7f38d7", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1014783449-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "747d5750a2174aaea71849c70167a816", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapff25eb15-1a", "ovs_interfaceid": "ff25eb15-1af0-4110-9d9a-ecd0a1d4627d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:17:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Releasing lock "refresh_cache-2d78475c-107d-4256-b12e-dbe9c8f3b9b2" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:17:13 user nova-compute[72351]: DEBUG nova.compute.manager [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Instance network_info: |[{"id": "ff25eb15-1af0-4110-9d9a-ecd0a1d4627d", "address": "fa:16:3e:0a:89:6a", "network": {"id": "fd1fe4e0-5d09-43c1-8dbc-1f60cb7f38d7", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1014783449-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "747d5750a2174aaea71849c70167a816", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapff25eb15-1a", "ovs_interfaceid": "ff25eb15-1af0-4110-9d9a-ecd0a1d4627d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1971}}
Aug 09 02:17:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-4fe566ab-b3fd-4616-9d94-dc40697022c5 req-7c50c9b8-67ed-4d12-9664-462365fe221c service nova] Acquired lock "refresh_cache-2d78475c-107d-4256-b12e-dbe9c8f3b9b2" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:17:13 user nova-compute[72351]: DEBUG nova.network.neutron [req-4fe566ab-b3fd-4616-9d94-dc40697022c5 req-7c50c9b8-67ed-4d12-9664-462365fe221c service nova] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Refreshing network info cache for port ff25eb15-1af0-4110-9d9a-ecd0a1d4627d {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:17:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Start _get_guest_xml network_info=[{"id": "ff25eb15-1af0-4110-9d9a-ecd0a1d4627d", "address": "fa:16:3e:0a:89:6a", "network": {"id": "fd1fe4e0-5d09-43c1-8dbc-1f60cb7f38d7", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1014783449-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "747d5750a2174aaea71849c70167a816", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapff25eb15-1a", "ovs_interfaceid": "ff25eb15-1af0-4110-9d9a-ecd0a1d4627d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'size': 0, 'device_type': 'disk', 'encryption_secret_uuid': None, 'guest_format': None, 'encryption_options': None, 'disk_bus': 'virtio', 'encrypted': False, 'encryption_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'image_id': '18ccda7f-31d3-4978-ab9d-4ed2f74eff94'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:17:13 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:17:13 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:17:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:17:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:17:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:17:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:17:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:17:13 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:17:13 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:17:13 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:17:13 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:17:13 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:17:13 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:17:13 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:17:13 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:17:13 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:17:13 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:17:13 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:17:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:17:04Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerStableDeviceRescueTest-server-1763341201',display_name='tempest-ServerStableDeviceRescueTest-server-1763341201',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serverstabledevicerescuetest-server-1763341201',id=2,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBPLsnKPb6VYEzHtPIe2+hjQmOnmHJvA1Ev7/Puz1ZEEURwvwHmMhKqMbBr/N8dvlnbS9aRgyU274IVV3B6lTROiOE1e6tZ3774qYOPMP3+6C+tI9LZ4+zAxuQqNbLNbP+Q==',key_name='tempest-keypair-1212195529',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='747d5750a2174aaea71849c70167a816',ramdisk_id='',reservation_id='r-d0n2amoe',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerStableDeviceRescueTest-790832465',owner_user_name='tempest-ServerStableDeviceRescueTest-790832465-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:17:06Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='157ad7fa9970477a91de3d1347dee441',uuid=2d78475c-107d-4256-b12e-dbe9c8f3b9b2,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "ff25eb15-1af0-4110-9d9a-ecd0a1d4627d", "address": "fa:16:3e:0a:89:6a", "network": {"id": "fd1fe4e0-5d09-43c1-8dbc-1f60cb7f38d7", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1014783449-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "747d5750a2174aaea71849c70167a816", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapff25eb15-1a", "ovs_interfaceid": "ff25eb15-1af0-4110-9d9a-ecd0a1d4627d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:17:13 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Converting VIF {"id": "ff25eb15-1af0-4110-9d9a-ecd0a1d4627d", "address": "fa:16:3e:0a:89:6a", "network": {"id": "fd1fe4e0-5d09-43c1-8dbc-1f60cb7f38d7", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1014783449-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "747d5750a2174aaea71849c70167a816", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapff25eb15-1a", "ovs_interfaceid": "ff25eb15-1af0-4110-9d9a-ecd0a1d4627d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:17:13 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:0a:89:6a,bridge_name='br-int',has_traffic_filtering=True,id=ff25eb15-1af0-4110-9d9a-ecd0a1d4627d,network=Network(fd1fe4e0-5d09-43c1-8dbc-1f60cb7f38d7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapff25eb15-1a') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:17:13 user nova-compute[72351]: DEBUG nova.objects.instance [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Lazy-loading 'pci_devices' on Instance uuid 2d78475c-107d-4256-b12e-dbe9c8f3b9b2 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:17:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:17:13 user nova-compute[72351]:   <uuid>2d78475c-107d-4256-b12e-dbe9c8f3b9b2</uuid>
Aug 09 02:17:13 user nova-compute[72351]:   <name>instance-00000002</name>
Aug 09 02:17:13 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:17:13 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:17:13 user nova-compute[72351]:   <metadata>
Aug 09 02:17:13 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:17:13 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:17:13 user nova-compute[72351]:       <nova:name>tempest-ServerStableDeviceRescueTest-server-1763341201</nova:name>
Aug 09 02:17:13 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:17:13</nova:creationTime>
Aug 09 02:17:13 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:17:13 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:17:13 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:17:13 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:17:13 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:17:13 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:17:13 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:17:13 user nova-compute[72351]:       <nova:owner>
Aug 09 02:17:13 user nova-compute[72351]:         <nova:user uuid="157ad7fa9970477a91de3d1347dee441">tempest-ServerStableDeviceRescueTest-790832465-project-member</nova:user>
Aug 09 02:17:13 user nova-compute[72351]:         <nova:project uuid="747d5750a2174aaea71849c70167a816">tempest-ServerStableDeviceRescueTest-790832465</nova:project>
Aug 09 02:17:13 user nova-compute[72351]:       </nova:owner>
Aug 09 02:17:13 user nova-compute[72351]:       <nova:root type="image" uuid="18ccda7f-31d3-4978-ab9d-4ed2f74eff94"/>
Aug 09 02:17:13 user nova-compute[72351]:       <nova:ports>
Aug 09 02:17:13 user nova-compute[72351]:         <nova:port uuid="ff25eb15-1af0-4110-9d9a-ecd0a1d4627d">
Aug 09 02:17:13 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.5" ipVersion="4"/>
Aug 09 02:17:13 user nova-compute[72351]:         </nova:port>
Aug 09 02:17:13 user nova-compute[72351]:       </nova:ports>
Aug 09 02:17:13 user nova-compute[72351]:     </nova:instance>
Aug 09 02:17:13 user nova-compute[72351]:   </metadata>
Aug 09 02:17:13 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:17:13 user nova-compute[72351]:     <system>
Aug 09 02:17:13 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:17:13 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:17:13 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:17:13 user nova-compute[72351]:       <entry name="serial">2d78475c-107d-4256-b12e-dbe9c8f3b9b2</entry>
Aug 09 02:17:13 user nova-compute[72351]:       <entry name="uuid">2d78475c-107d-4256-b12e-dbe9c8f3b9b2</entry>
Aug 09 02:17:13 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:17:13 user nova-compute[72351]:     </system>
Aug 09 02:17:13 user nova-compute[72351]:   </sysinfo>
Aug 09 02:17:13 user nova-compute[72351]:   <os>
Aug 09 02:17:13 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:17:13 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:17:13 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:17:13 user nova-compute[72351]:   </os>
Aug 09 02:17:13 user nova-compute[72351]:   <features>
Aug 09 02:17:13 user nova-compute[72351]:     <acpi/>
Aug 09 02:17:13 user nova-compute[72351]:     <apic/>
Aug 09 02:17:13 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:17:13 user nova-compute[72351]:   </features>
Aug 09 02:17:13 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:17:13 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:17:13 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:17:13 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:17:13 user nova-compute[72351]:   </clock>
Aug 09 02:17:13 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:17:13 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:17:13 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:17:13 user nova-compute[72351]:   </cpu>
Aug 09 02:17:13 user nova-compute[72351]:   <devices>
Aug 09 02:17:13 user nova-compute[72351]:     <disk type="file" device="disk">
Aug 09 02:17:13 user nova-compute[72351]:       <driver name="qemu" type="qcow2" cache="none"/>
Aug 09 02:17:13 user nova-compute[72351]:       <source file="/opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2/disk"/>
Aug 09 02:17:13 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:17:13 user nova-compute[72351]:     </disk>
Aug 09 02:17:13 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:17:13 user nova-compute[72351]:       <mac address="fa:16:3e:0a:89:6a"/>
Aug 09 02:17:13 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:17:13 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:17:13 user nova-compute[72351]:       <target dev="tapff25eb15-1a"/>
Aug 09 02:17:13 user nova-compute[72351]:     </interface>
Aug 09 02:17:13 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:17:13 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2/console.log" append="off"/>
Aug 09 02:17:13 user nova-compute[72351]:     </serial>
Aug 09 02:17:13 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:17:13 user nova-compute[72351]:     <video>
Aug 09 02:17:13 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:17:13 user nova-compute[72351]:     </video>
Aug 09 02:17:13 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:17:13 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:17:13 user nova-compute[72351]:     </rng>
Aug 09 02:17:13 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:17:13 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:17:13 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:17:13 user nova-compute[72351]:     </memballoon>
Aug 09 02:17:13 user nova-compute[72351]:   </devices>
Aug 09 02:17:13 user nova-compute[72351]: </domain>
Aug 09 02:17:13 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:17:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:17:04Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerStableDeviceRescueTest-server-1763341201',display_name='tempest-ServerStableDeviceRescueTest-server-1763341201',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serverstabledevicerescuetest-server-1763341201',id=2,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBPLsnKPb6VYEzHtPIe2+hjQmOnmHJvA1Ev7/Puz1ZEEURwvwHmMhKqMbBr/N8dvlnbS9aRgyU274IVV3B6lTROiOE1e6tZ3774qYOPMP3+6C+tI9LZ4+zAxuQqNbLNbP+Q==',key_name='tempest-keypair-1212195529',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='747d5750a2174aaea71849c70167a816',ramdisk_id='',reservation_id='r-d0n2amoe',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerStableDeviceRescueTest-790832465',owner_user_name='tempest-ServerStableDeviceRescueTest-790832465-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:17:06Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='157ad7fa9970477a91de3d1347dee441',uuid=2d78475c-107d-4256-b12e-dbe9c8f3b9b2,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "ff25eb15-1af0-4110-9d9a-ecd0a1d4627d", "address": "fa:16:3e:0a:89:6a", "network": {"id": "fd1fe4e0-5d09-43c1-8dbc-1f60cb7f38d7", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1014783449-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "747d5750a2174aaea71849c70167a816", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapff25eb15-1a", "ovs_interfaceid": "ff25eb15-1af0-4110-9d9a-ecd0a1d4627d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:17:13 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Converting VIF {"id": "ff25eb15-1af0-4110-9d9a-ecd0a1d4627d", "address": "fa:16:3e:0a:89:6a", "network": {"id": "fd1fe4e0-5d09-43c1-8dbc-1f60cb7f38d7", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1014783449-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "747d5750a2174aaea71849c70167a816", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapff25eb15-1a", "ovs_interfaceid": "ff25eb15-1af0-4110-9d9a-ecd0a1d4627d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:17:13 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:0a:89:6a,bridge_name='br-int',has_traffic_filtering=True,id=ff25eb15-1af0-4110-9d9a-ecd0a1d4627d,network=Network(fd1fe4e0-5d09-43c1-8dbc-1f60cb7f38d7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapff25eb15-1a') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:17:13 user nova-compute[72351]: DEBUG os_vif [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:0a:89:6a,bridge_name='br-int',has_traffic_filtering=True,id=ff25eb15-1af0-4110-9d9a-ecd0a1d4627d,network=Network(fd1fe4e0-5d09-43c1-8dbc-1f60cb7f38d7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapff25eb15-1a') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:17:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:17:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:17:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapff25eb15-1a, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:17:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapff25eb15-1a, col_values=(('external_ids', {'iface-id': 'ff25eb15-1af0-4110-9d9a-ecd0a1d4627d', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:0a:89:6a', 'vm-uuid': '2d78475c-107d-4256-b12e-dbe9c8f3b9b2'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:17:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:17:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:13 user nova-compute[72351]: INFO os_vif [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:0a:89:6a,bridge_name='br-int',has_traffic_filtering=True,id=ff25eb15-1af0-4110-9d9a-ecd0a1d4627d,network=Network(fd1fe4e0-5d09-43c1-8dbc-1f60cb7f38d7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapff25eb15-1a')
Aug 09 02:17:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:17:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] No VIF found with MAC fa:16:3e:0a:89:6a, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:17:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:15 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:15 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:15 user nova-compute[72351]: DEBUG nova.network.neutron [req-4fe566ab-b3fd-4616-9d94-dc40697022c5 req-7c50c9b8-67ed-4d12-9664-462365fe221c service nova] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Updated VIF entry in instance network info cache for port ff25eb15-1af0-4110-9d9a-ecd0a1d4627d. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:17:15 user nova-compute[72351]: DEBUG nova.network.neutron [req-4fe566ab-b3fd-4616-9d94-dc40697022c5 req-7c50c9b8-67ed-4d12-9664-462365fe221c service nova] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Updating instance_info_cache with network_info: [{"id": "ff25eb15-1af0-4110-9d9a-ecd0a1d4627d", "address": "fa:16:3e:0a:89:6a", "network": {"id": "fd1fe4e0-5d09-43c1-8dbc-1f60cb7f38d7", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1014783449-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "747d5750a2174aaea71849c70167a816", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapff25eb15-1a", "ovs_interfaceid": "ff25eb15-1af0-4110-9d9a-ecd0a1d4627d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:17:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-4fe566ab-b3fd-4616-9d94-dc40697022c5 req-7c50c9b8-67ed-4d12-9664-462365fe221c service nova] Releasing lock "refresh_cache-2d78475c-107d-4256-b12e-dbe9c8f3b9b2" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:17:15 user nova-compute[72351]: DEBUG nova.compute.manager [req-7380f5b6-23a4-45eb-9022-fb4779b76543 req-dd4b76d4-ca1b-417d-a5f1-7cd3d45dd945 service nova] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Received event network-vif-plugged-ff25eb15-1af0-4110-9d9a-ecd0a1d4627d {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:17:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-7380f5b6-23a4-45eb-9022-fb4779b76543 req-dd4b76d4-ca1b-417d-a5f1-7cd3d45dd945 service nova] Acquiring lock "2d78475c-107d-4256-b12e-dbe9c8f3b9b2-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:17:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-7380f5b6-23a4-45eb-9022-fb4779b76543 req-dd4b76d4-ca1b-417d-a5f1-7cd3d45dd945 service nova] Lock "2d78475c-107d-4256-b12e-dbe9c8f3b9b2-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:17:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-7380f5b6-23a4-45eb-9022-fb4779b76543 req-dd4b76d4-ca1b-417d-a5f1-7cd3d45dd945 service nova] Lock "2d78475c-107d-4256-b12e-dbe9c8f3b9b2-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:17:15 user nova-compute[72351]: DEBUG nova.compute.manager [req-7380f5b6-23a4-45eb-9022-fb4779b76543 req-dd4b76d4-ca1b-417d-a5f1-7cd3d45dd945 service nova] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] No waiting events found dispatching network-vif-plugged-ff25eb15-1af0-4110-9d9a-ecd0a1d4627d {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:17:15 user nova-compute[72351]: WARNING nova.compute.manager [req-7380f5b6-23a4-45eb-9022-fb4779b76543 req-dd4b76d4-ca1b-417d-a5f1-7cd3d45dd945 service nova] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Received unexpected event network-vif-plugged-ff25eb15-1af0-4110-9d9a-ecd0a1d4627d for instance with vm_state building and task_state spawning.
Aug 09 02:17:16 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:16 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:16 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:16 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:17 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:17 user nova-compute[72351]: DEBUG nova.compute.manager [req-2418a87f-3bfc-4ce6-bb43-7dff83a1905d req-26d28655-defb-4503-9b33-1614d720a30f service nova] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Received event network-vif-plugged-ff25eb15-1af0-4110-9d9a-ecd0a1d4627d {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:17:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-2418a87f-3bfc-4ce6-bb43-7dff83a1905d req-26d28655-defb-4503-9b33-1614d720a30f service nova] Acquiring lock "2d78475c-107d-4256-b12e-dbe9c8f3b9b2-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:17:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-2418a87f-3bfc-4ce6-bb43-7dff83a1905d req-26d28655-defb-4503-9b33-1614d720a30f service nova] Lock "2d78475c-107d-4256-b12e-dbe9c8f3b9b2-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:17:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-2418a87f-3bfc-4ce6-bb43-7dff83a1905d req-26d28655-defb-4503-9b33-1614d720a30f service nova] Lock "2d78475c-107d-4256-b12e-dbe9c8f3b9b2-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:17:17 user nova-compute[72351]: DEBUG nova.compute.manager [req-2418a87f-3bfc-4ce6-bb43-7dff83a1905d req-26d28655-defb-4503-9b33-1614d720a30f service nova] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] No waiting events found dispatching network-vif-plugged-ff25eb15-1af0-4110-9d9a-ecd0a1d4627d {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:17:17 user nova-compute[72351]: WARNING nova.compute.manager [req-2418a87f-3bfc-4ce6-bb43-7dff83a1905d req-26d28655-defb-4503-9b33-1614d720a30f service nova] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Received unexpected event network-vif-plugged-ff25eb15-1af0-4110-9d9a-ecd0a1d4627d for instance with vm_state building and task_state spawning.
Aug 09 02:17:18 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547438.2171352, 2d78475c-107d-4256-b12e-dbe9c8f3b9b2 => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:17:18 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] VM Resumed (Lifecycle Event)
Aug 09 02:17:18 user nova-compute[72351]: DEBUG nova.compute.manager [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:17:18 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Guest created on hypervisor {{(pid=72351) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4411}}
Aug 09 02:17:18 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Instance spawned successfully.
Aug 09 02:17:18 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:894}}
Aug 09 02:17:18 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:17:18 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:17:18 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Found default for hw_cdrom_bus of ide {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:17:18 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Found default for hw_disk_bus of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:17:18 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Found default for hw_input_bus of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:17:18 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Found default for hw_pointer_model of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:17:18 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Found default for hw_video_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:17:18 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Found default for hw_vif_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:17:18 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:17:18 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547438.2191966, 2d78475c-107d-4256-b12e-dbe9c8f3b9b2 => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:17:18 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] VM Started (Lifecycle Event)
Aug 09 02:17:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:18 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:17:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:18 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:17:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:18 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:17:18 user nova-compute[72351]: INFO nova.compute.manager [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Took 12.18 seconds to spawn the instance on the hypervisor.
Aug 09 02:17:18 user nova-compute[72351]: DEBUG nova.compute.manager [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:17:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:18 user nova-compute[72351]: INFO nova.compute.manager [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Took 13.14 seconds to build instance.
Aug 09 02:17:18 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a7db00a0-1580-49f9-bf2e-e93eda3b1332 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Lock "2d78475c-107d-4256-b12e-dbe9c8f3b9b2" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 13.315s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:17:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Acquiring lock "74728ca8-34d4-45e6-a1f2-21c0952ff835" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:17:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Lock "74728ca8-34d4-45e6-a1f2-21c0952ff835" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:17:20 user nova-compute[72351]: DEBUG nova.compute.manager [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Starting instance... {{(pid=72351) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2406}}
Aug 09 02:17:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Acquiring lock "50302d76-f6b9-4fe3-b55a-4a01e483e2e3" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:17:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Lock "50302d76-f6b9-4fe3-b55a-4a01e483e2e3" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.003s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:17:20 user nova-compute[72351]: DEBUG nova.compute.manager [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Starting instance... {{(pid=72351) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2406}}
Aug 09 02:17:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:17:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:17:20 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72351) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Aug 09 02:17:20 user nova-compute[72351]: INFO nova.compute.claims [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Claim successful on node user
Aug 09 02:17:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:17:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Acquiring lock "11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:17:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Lock "11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:17:20 user nova-compute[72351]: DEBUG nova.compute.manager [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Starting instance... {{(pid=72351) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2406}}
Aug 09 02:17:20 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:17:20 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:17:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:17:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.562s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:17:20 user nova-compute[72351]: DEBUG nova.compute.manager [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Start building networks asynchronously for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2803}}
Aug 09 02:17:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.511s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:17:20 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72351) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Aug 09 02:17:20 user nova-compute[72351]: INFO nova.compute.claims [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Claim successful on node user
Aug 09 02:17:20 user nova-compute[72351]: DEBUG nova.compute.manager [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Allocating IP information in the background. {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1956}}
Aug 09 02:17:20 user nova-compute[72351]: DEBUG nova.network.neutron [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] allocate_for_instance() {{(pid=72351) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Aug 09 02:17:20 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Aug 09 02:17:20 user nova-compute[72351]: DEBUG nova.compute.manager [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Start building block device mappings for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2838}}
Aug 09 02:17:21 user nova-compute[72351]: DEBUG nova.policy [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'fb9f73926e0d42f586e7c0055764d611', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bc5e05fbd7f44f85bc3b1cc169dd03a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72351) authorize /opt/stack/nova/nova/policy.py:203}}
Aug 09 02:17:21 user nova-compute[72351]: DEBUG nova.compute.manager [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Start spawning the instance on the hypervisor. {{(pid=72351) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2612}}
Aug 09 02:17:21 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Creating instance directory {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4717}}
Aug 09 02:17:21 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Creating image(s)
Aug 09 02:17:21 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Acquiring lock "/opt/stack/data/nova/instances/74728ca8-34d4-45e6-a1f2-21c0952ff835/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:17:21 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Lock "/opt/stack/data/nova/instances/74728ca8-34d4-45e6-a1f2-21c0952ff835/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:17:21 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Lock "/opt/stack/data/nova/instances/74728ca8-34d4-45e6-a1f2-21c0952ff835/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:17:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:17:21 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:17:21 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:17:21 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.567s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:17:21 user nova-compute[72351]: DEBUG nova.compute.manager [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Start building networks asynchronously for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2803}}
Aug 09 02:17:21 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.633s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:17:21 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72351) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Aug 09 02:17:21 user nova-compute[72351]: INFO nova.compute.claims [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Claim successful on node user
Aug 09 02:17:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.184s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:17:21 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Acquiring lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:17:21 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:17:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:17:21 user nova-compute[72351]: DEBUG nova.compute.manager [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Allocating IP information in the background. {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1956}}
Aug 09 02:17:21 user nova-compute[72351]: DEBUG nova.network.neutron [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] allocate_for_instance() {{(pid=72351) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Aug 09 02:17:21 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Ignoring supplied device name: /dev/sda. Libvirt can't honour user-supplied dev names
Aug 09 02:17:21 user nova-compute[72351]: DEBUG nova.compute.manager [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Start building block device mappings for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2838}}
Aug 09 02:17:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.202s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:17:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/74728ca8-34d4-45e6-a1f2-21c0952ff835/disk 1073741824 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:17:21 user nova-compute[72351]: DEBUG nova.policy [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '657c221a2b1840ad8e75165d89d779e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f04dea5c1d84aaf94feb47b8ea84f0e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72351) authorize /opt/stack/nova/nova/policy.py:203}}
Aug 09 02:17:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/74728ca8-34d4-45e6-a1f2-21c0952ff835/disk 1073741824" returned: 0 in 0.100s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:17:21 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.306s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:17:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:17:21 user nova-compute[72351]: DEBUG nova.compute.manager [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Start spawning the instance on the hypervisor. {{(pid=72351) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2612}}
Aug 09 02:17:21 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Creating instance directory {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4717}}
Aug 09 02:17:21 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Creating image(s)
Aug 09 02:17:21 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Acquiring lock "/opt/stack/data/nova/instances/50302d76-f6b9-4fe3-b55a-4a01e483e2e3/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:17:21 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Lock "/opt/stack/data/nova/instances/50302d76-f6b9-4fe3-b55a-4a01e483e2e3/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:17:21 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Lock "/opt/stack/data/nova/instances/50302d76-f6b9-4fe3-b55a-4a01e483e2e3/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:17:21 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Acquiring lock "85c4e52d495f393d9b5d4e4fdc36691ebb7ca1a2" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:17:21 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Lock "85c4e52d495f393d9b5d4e4fdc36691ebb7ca1a2" acquired by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:17:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.210s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:17:21 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Checking if we can resize image /opt/stack/data/nova/instances/74728ca8-34d4-45e6-a1f2-21c0952ff835/disk. size=1073741824 {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Aug 09 02:17:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/74728ca8-34d4-45e6-a1f2-21c0952ff835/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:17:22 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:17:22 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:17:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.729s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:17:22 user nova-compute[72351]: DEBUG nova.compute.manager [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Start building networks asynchronously for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2803}}
Aug 09 02:17:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/74728ca8-34d4-45e6-a1f2-21c0952ff835/disk --force-share --output=json" returned: 0 in 0.212s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:17:22 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Cannot resize image /opt/stack/data/nova/instances/74728ca8-34d4-45e6-a1f2-21c0952ff835/disk to a smaller size. {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Aug 09 02:17:22 user nova-compute[72351]: DEBUG nova.objects.instance [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Lazy-loading 'migration_context' on Instance uuid 74728ca8-34d4-45e6-a1f2-21c0952ff835 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:17:22 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Created local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4851}}
Aug 09 02:17:22 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Ensure instance console log exists: /opt/stack/data/nova/instances/74728ca8-34d4-45e6-a1f2-21c0952ff835/console.log {{(pid=72351) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4603}}
Aug 09 02:17:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:17:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:17:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:17:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/85c4e52d495f393d9b5d4e4fdc36691ebb7ca1a2.part --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:17:22 user nova-compute[72351]: DEBUG nova.network.neutron [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Successfully created port: 3718df01-0e23-424c-8ab1-bf0fd9c7e121 {{(pid=72351) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Aug 09 02:17:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:22 user nova-compute[72351]: DEBUG nova.compute.manager [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Allocating IP information in the background. {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1956}}
Aug 09 02:17:22 user nova-compute[72351]: DEBUG nova.network.neutron [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] allocate_for_instance() {{(pid=72351) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Aug 09 02:17:22 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Aug 09 02:17:22 user nova-compute[72351]: DEBUG nova.compute.manager [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Start building block device mappings for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2838}}
Aug 09 02:17:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Acquiring lock "7c5e633d-a32f-4c5e-bb9d-1e40da964bd0" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:17:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Lock "7c5e633d-a32f-4c5e-bb9d-1e40da964bd0" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:17:22 user nova-compute[72351]: DEBUG nova.compute.manager [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Starting instance... {{(pid=72351) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2406}}
Aug 09 02:17:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/85c4e52d495f393d9b5d4e4fdc36691ebb7ca1a2.part --force-share --output=json" returned: 0 in 0.233s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:17:22 user nova-compute[72351]: DEBUG nova.virt.images [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] 62f1da90-2927-452b-831f-d34d21ab862a was qcow2, converting to raw {{(pid=72351) fetch_to_raw /opt/stack/nova/nova/virt/images.py:165}}
Aug 09 02:17:22 user nova-compute[72351]: DEBUG nova.privsep.utils [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=72351) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Aug 09 02:17:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Running cmd (subprocess): qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/85c4e52d495f393d9b5d4e4fdc36691ebb7ca1a2.part /opt/stack/data/nova/instances/_base/85c4e52d495f393d9b5d4e4fdc36691ebb7ca1a2.converted {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:17:23 user nova-compute[72351]: DEBUG nova.compute.manager [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Start spawning the instance on the hypervisor. {{(pid=72351) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2612}}
Aug 09 02:17:23 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Creating instance directory {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4717}}
Aug 09 02:17:23 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Creating image(s)
Aug 09 02:17:23 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Acquiring lock "/opt/stack/data/nova/instances/11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:17:23 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Lock "/opt/stack/data/nova/instances/11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:17:23 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Lock "/opt/stack/data/nova/instances/11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.008s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:17:23 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:17:23 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:17:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:17:23 user nova-compute[72351]: DEBUG nova.policy [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '876134724b18462aba75896ddfad5705', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1277988bc35041249b8e315df5259f98', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72351) authorize /opt/stack/nova/nova/policy.py:203}}
Aug 09 02:17:23 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72351) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Aug 09 02:17:23 user nova-compute[72351]: INFO nova.compute.claims [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Claim successful on node user
Aug 09 02:17:23 user nova-compute[72351]: DEBUG nova.network.neutron [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Successfully created port: e5ba6e8d-49a5-4761-ab57-21cee6b0fbdd {{(pid=72351) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Aug 09 02:17:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.256s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:17:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] CMD "qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/85c4e52d495f393d9b5d4e4fdc36691ebb7ca1a2.part /opt/stack/data/nova/instances/_base/85c4e52d495f393d9b5d4e4fdc36691ebb7ca1a2.converted" returned: 0 in 0.482s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:17:23 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Acquiring lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:17:23 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:17:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:17:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/85c4e52d495f393d9b5d4e4fdc36691ebb7ca1a2.converted --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:17:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.169s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:17:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b/disk 1073741824 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:17:23 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/85c4e52d495f393d9b5d4e4fdc36691ebb7ca1a2.converted --force-share --output=json" returned: 0 in 0.169s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:17:23 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Lock "85c4e52d495f393d9b5d4e4fdc36691ebb7ca1a2" "released" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: held 1.743s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:17:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b/disk 1073741824" returned: 0 in 0.057s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:17:23 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.231s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:17:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:17:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/85c4e52d495f393d9b5d4e4fdc36691ebb7ca1a2 --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:17:23 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:17:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.261s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:17:23 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Checking if we can resize image /opt/stack/data/nova/instances/11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b/disk. size=1073741824 {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Aug 09 02:17:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:17:23 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:17:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/85c4e52d495f393d9b5d4e4fdc36691ebb7ca1a2 --force-share --output=json" returned: 0 in 0.287s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:17:23 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Acquiring lock "85c4e52d495f393d9b5d4e4fdc36691ebb7ca1a2" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:17:23 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Lock "85c4e52d495f393d9b5d4e4fdc36691ebb7ca1a2" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:17:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/85c4e52d495f393d9b5d4e4fdc36691ebb7ca1a2 --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:17:23 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.917s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:17:23 user nova-compute[72351]: DEBUG nova.compute.manager [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Start building networks asynchronously for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2803}}
Aug 09 02:17:24 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b/disk --force-share --output=json" returned: 0 in 0.206s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:17:24 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Cannot resize image /opt/stack/data/nova/instances/11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b/disk to a smaller size. {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Aug 09 02:17:24 user nova-compute[72351]: DEBUG nova.objects.instance [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Lazy-loading 'migration_context' on Instance uuid 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:17:24 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Created local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4851}}
Aug 09 02:17:24 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Ensure instance console log exists: /opt/stack/data/nova/instances/11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b/console.log {{(pid=72351) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4603}}
Aug 09 02:17:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:17:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:17:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.022s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:17:24 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/85c4e52d495f393d9b5d4e4fdc36691ebb7ca1a2 --force-share --output=json" returned: 0 in 0.186s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:17:24 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/85c4e52d495f393d9b5d4e4fdc36691ebb7ca1a2,backing_fmt=raw /opt/stack/data/nova/instances/50302d76-f6b9-4fe3-b55a-4a01e483e2e3/disk 1073741824 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:17:24 user nova-compute[72351]: DEBUG nova.compute.manager [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Allocating IP information in the background. {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1956}}
Aug 09 02:17:24 user nova-compute[72351]: DEBUG nova.network.neutron [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] allocate_for_instance() {{(pid=72351) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Aug 09 02:17:24 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/85c4e52d495f393d9b5d4e4fdc36691ebb7ca1a2,backing_fmt=raw /opt/stack/data/nova/instances/50302d76-f6b9-4fe3-b55a-4a01e483e2e3/disk 1073741824" returned: 0 in 0.085s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:17:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Lock "85c4e52d495f393d9b5d4e4fdc36691ebb7ca1a2" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.276s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:17:24 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/85c4e52d495f393d9b5d4e4fdc36691ebb7ca1a2 --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:17:24 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Aug 09 02:17:24 user nova-compute[72351]: DEBUG nova.compute.manager [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Start building block device mappings for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2838}}
Aug 09 02:17:24 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/85c4e52d495f393d9b5d4e4fdc36691ebb7ca1a2 --force-share --output=json" returned: 0 in 0.177s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:17:24 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Checking if we can resize image /opt/stack/data/nova/instances/50302d76-f6b9-4fe3-b55a-4a01e483e2e3/disk. size=1073741824 {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Aug 09 02:17:24 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/50302d76-f6b9-4fe3-b55a-4a01e483e2e3/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:17:24 user nova-compute[72351]: DEBUG nova.compute.manager [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Start spawning the instance on the hypervisor. {{(pid=72351) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2612}}
Aug 09 02:17:24 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Creating instance directory {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4717}}
Aug 09 02:17:24 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Creating image(s)
Aug 09 02:17:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Acquiring lock "/opt/stack/data/nova/instances/7c5e633d-a32f-4c5e-bb9d-1e40da964bd0/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:17:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Lock "/opt/stack/data/nova/instances/7c5e633d-a32f-4c5e-bb9d-1e40da964bd0/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:17:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Lock "/opt/stack/data/nova/instances/7c5e633d-a32f-4c5e-bb9d-1e40da964bd0/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:17:24 user nova-compute[72351]: DEBUG nova.policy [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '52bc0e2d81424ca19722e8af947d399b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26827058be4b48a3a7aed21aad7c5e77', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72351) authorize /opt/stack/nova/nova/policy.py:203}}
Aug 09 02:17:24 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:17:24 user nova-compute[72351]: DEBUG nova.network.neutron [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Successfully updated port: e5ba6e8d-49a5-4761-ab57-21cee6b0fbdd {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:17:24 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/50302d76-f6b9-4fe3-b55a-4a01e483e2e3/disk --force-share --output=json" returned: 0 in 0.188s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:17:24 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Cannot resize image /opt/stack/data/nova/instances/50302d76-f6b9-4fe3-b55a-4a01e483e2e3/disk to a smaller size. {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Aug 09 02:17:24 user nova-compute[72351]: DEBUG nova.objects.instance [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Lazy-loading 'migration_context' on Instance uuid 50302d76-f6b9-4fe3-b55a-4a01e483e2e3 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:17:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Acquiring lock "refresh_cache-50302d76-f6b9-4fe3-b55a-4a01e483e2e3" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:17:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Acquired lock "refresh_cache-50302d76-f6b9-4fe3-b55a-4a01e483e2e3" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:17:24 user nova-compute[72351]: DEBUG nova.network.neutron [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:17:24 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Created local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4851}}
Aug 09 02:17:24 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Ensure instance console log exists: /opt/stack/data/nova/instances/50302d76-f6b9-4fe3-b55a-4a01e483e2e3/console.log {{(pid=72351) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4603}}
Aug 09 02:17:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:17:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:17:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:17:24 user nova-compute[72351]: DEBUG nova.compute.manager [req-bde26e5d-26ac-48ea-ac60-b189989c5a97 req-35805b11-4c12-4aef-86d9-b84887a057bd service nova] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Received event network-changed-e5ba6e8d-49a5-4761-ab57-21cee6b0fbdd {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:17:24 user nova-compute[72351]: DEBUG nova.compute.manager [req-bde26e5d-26ac-48ea-ac60-b189989c5a97 req-35805b11-4c12-4aef-86d9-b84887a057bd service nova] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Refreshing instance network info cache due to event network-changed-e5ba6e8d-49a5-4761-ab57-21cee6b0fbdd. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:17:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-bde26e5d-26ac-48ea-ac60-b189989c5a97 req-35805b11-4c12-4aef-86d9-b84887a057bd service nova] Acquiring lock "refresh_cache-50302d76-f6b9-4fe3-b55a-4a01e483e2e3" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:17:24 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.187s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:17:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Acquiring lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:17:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:17:24 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:17:24 user nova-compute[72351]: DEBUG nova.network.neutron [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:17:24 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.179s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:17:24 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/7c5e633d-a32f-4c5e-bb9d-1e40da964bd0/disk 1073741824 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:17:24 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/7c5e633d-a32f-4c5e-bb9d-1e40da964bd0/disk 1073741824" returned: 0 in 0.061s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:17:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.248s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:17:24 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:17:24 user nova-compute[72351]: DEBUG nova.network.neutron [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Successfully updated port: 3718df01-0e23-424c-8ab1-bf0fd9c7e121 {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:17:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Acquiring lock "refresh_cache-74728ca8-34d4-45e6-a1f2-21c0952ff835" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:17:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Acquired lock "refresh_cache-74728ca8-34d4-45e6-a1f2-21c0952ff835" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:17:25 user nova-compute[72351]: DEBUG nova.network.neutron [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:17:25 user nova-compute[72351]: DEBUG nova.network.neutron [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Successfully created port: a309cd5b-1679-48c6-a596-168b3af2d284 {{(pid=72351) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Aug 09 02:17:25 user nova-compute[72351]: DEBUG nova.compute.manager [req-8e85c34c-e067-42f1-a0b0-873289446c22 req-0fb1605b-0aae-4a8a-b85e-6dbea574f951 service nova] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Received event network-changed-3718df01-0e23-424c-8ab1-bf0fd9c7e121 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:17:25 user nova-compute[72351]: DEBUG nova.compute.manager [req-8e85c34c-e067-42f1-a0b0-873289446c22 req-0fb1605b-0aae-4a8a-b85e-6dbea574f951 service nova] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Refreshing instance network info cache due to event network-changed-3718df01-0e23-424c-8ab1-bf0fd9c7e121. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:17:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-8e85c34c-e067-42f1-a0b0-873289446c22 req-0fb1605b-0aae-4a8a-b85e-6dbea574f951 service nova] Acquiring lock "refresh_cache-74728ca8-34d4-45e6-a1f2-21c0952ff835" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:17:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.203s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:17:25 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Checking if we can resize image /opt/stack/data/nova/instances/7c5e633d-a32f-4c5e-bb9d-1e40da964bd0/disk. size=1073741824 {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Aug 09 02:17:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7c5e633d-a32f-4c5e-bb9d-1e40da964bd0/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:17:25 user nova-compute[72351]: DEBUG nova.network.neutron [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Updating instance_info_cache with network_info: [{"id": "e5ba6e8d-49a5-4761-ab57-21cee6b0fbdd", "address": "fa:16:3e:6e:b8:a5", "network": {"id": "1acf710e-e626-43a0-a32f-389a87f9858e", "bridge": "br-int", "label": "tempest-AttachSCSIVolumeTestJSON-1052335889-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3f04dea5c1d84aaf94feb47b8ea84f0e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape5ba6e8d-49", "ovs_interfaceid": "e5ba6e8d-49a5-4761-ab57-21cee6b0fbdd", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:17:25 user nova-compute[72351]: DEBUG nova.network.neutron [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:17:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7c5e633d-a32f-4c5e-bb9d-1e40da964bd0/disk --force-share --output=json" returned: 0 in 0.213s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:17:25 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Cannot resize image /opt/stack/data/nova/instances/7c5e633d-a32f-4c5e-bb9d-1e40da964bd0/disk to a smaller size. {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Aug 09 02:17:25 user nova-compute[72351]: DEBUG nova.objects.instance [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Lazy-loading 'migration_context' on Instance uuid 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:17:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Releasing lock "refresh_cache-50302d76-f6b9-4fe3-b55a-4a01e483e2e3" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:17:25 user nova-compute[72351]: DEBUG nova.compute.manager [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Instance network_info: |[{"id": "e5ba6e8d-49a5-4761-ab57-21cee6b0fbdd", "address": "fa:16:3e:6e:b8:a5", "network": {"id": "1acf710e-e626-43a0-a32f-389a87f9858e", "bridge": "br-int", "label": "tempest-AttachSCSIVolumeTestJSON-1052335889-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3f04dea5c1d84aaf94feb47b8ea84f0e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape5ba6e8d-49", "ovs_interfaceid": "e5ba6e8d-49a5-4761-ab57-21cee6b0fbdd", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1971}}
Aug 09 02:17:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-bde26e5d-26ac-48ea-ac60-b189989c5a97 req-35805b11-4c12-4aef-86d9-b84887a057bd service nova] Acquired lock "refresh_cache-50302d76-f6b9-4fe3-b55a-4a01e483e2e3" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:17:25 user nova-compute[72351]: DEBUG nova.network.neutron [req-bde26e5d-26ac-48ea-ac60-b189989c5a97 req-35805b11-4c12-4aef-86d9-b84887a057bd service nova] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Refreshing network info cache for port e5ba6e8d-49a5-4761-ab57-21cee6b0fbdd {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:17:25 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Start _get_guest_xml network_info=[{"id": "e5ba6e8d-49a5-4761-ab57-21cee6b0fbdd", "address": "fa:16:3e:6e:b8:a5", "network": {"id": "1acf710e-e626-43a0-a32f-389a87f9858e", "bridge": "br-int", "label": "tempest-AttachSCSIVolumeTestJSON-1052335889-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3f04dea5c1d84aaf94feb47b8ea84f0e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape5ba6e8d-49", "ovs_interfaceid": "e5ba6e8d-49a5-4761-ab57-21cee6b0fbdd", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'scsi', 'cdrom_bus': 'scsi', 'mapping': {'root': {'bus': 'scsi', 'dev': 'sda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'scsi', 'dev': 'sda', 'type': 'disk', 'boot_index': '1'}, 'disk.config': {'bus': 'scsi', 'dev': 'sdb', 'type': 'cdrom'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:17:11Z,direct_url=<?>,disk_format='qcow2',id=62f1da90-2927-452b-831f-d34d21ab862a,min_disk=0,min_ram=0,name='',owner='5cdc423e82ee44aeba8d45db28b96e9e',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:17:14Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/sda', 'image': [{'size': 0, 'device_type': 'disk', 'encryption_secret_uuid': None, 'guest_format': None, 'encryption_options': None, 'disk_bus': 'scsi', 'encrypted': False, 'encryption_format': None, 'device_name': '/dev/sda', 'boot_index': 0, 'image_id': '62f1da90-2927-452b-831f-d34d21ab862a'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:17:25 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Created local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4851}}
Aug 09 02:17:25 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Ensure instance console log exists: /opt/stack/data/nova/instances/7c5e633d-a32f-4c5e-bb9d-1e40da964bd0/console.log {{(pid=72351) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4603}}
Aug 09 02:17:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:17:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:17:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:17:25 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:17:25 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:17:25 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:17:25 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:17:25 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:17:25 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:17:25 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:17:25 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:17:11Z,direct_url=<?>,disk_format='qcow2',id=62f1da90-2927-452b-831f-d34d21ab862a,min_disk=0,min_ram=0,name='',owner='5cdc423e82ee44aeba8d45db28b96e9e',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:17:14Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:17:25 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:17:25 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:17:25 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:17:25 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:17:25 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:17:25 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:17:25 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:17:25 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:17:25 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:17:25 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:17:25 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-08-09T02:17:18Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachSCSIVolumeTestJSON-server-1904871607',display_name='tempest-AttachSCSIVolumeTestJSON-server-1904871607',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachscsivolumetestjson-server-1904871607',id=3,image_ref='62f1da90-2927-452b-831f-d34d21ab862a',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJGHMVLR+W7uX3EHxXslcn5rNtsVUdS8noPRzLhTwfCXUymvnqG/yjZ6GlWujXjOHQqzGpQI8gBHB2Bq+KvBrqlYY9bJOEmcP7qmKR9OdnSd6Ve+rANep1lfbliuNjFHjg==',key_name='tempest-keypair-259400434',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='3f04dea5c1d84aaf94feb47b8ea84f0e',ramdisk_id='',reservation_id='r-tbsm9tz2',resources=None,root_device_name='/dev/sda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='62f1da90-2927-452b-831f-d34d21ab862a',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='scsi',image_hw_disk_bus='scsi',image_hw_machine_type='pc',image_hw_scsi_model='virtio-scsi',image_min_disk='1',image_min_ram='0',network_allocated='True',owner_project_name='tempest-AttachSCSIVolumeTestJSON-1890923628',owner_user_name='tempest-AttachSCSIVolumeTestJSON-1890923628-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:17:22Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='657c221a2b1840ad8e75165d89d779e8',uuid=50302d76-f6b9-4fe3-b55a-4a01e483e2e3,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "e5ba6e8d-49a5-4761-ab57-21cee6b0fbdd", "address": "fa:16:3e:6e:b8:a5", "network": {"id": "1acf710e-e626-43a0-a32f-389a87f9858e", "bridge": "br-int", "label": "tempest-AttachSCSIVolumeTestJSON-1052335889-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3f04dea5c1d84aaf94feb47b8ea84f0e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape5ba6e8d-49", "ovs_interfaceid": "e5ba6e8d-49a5-4761-ab57-21cee6b0fbdd", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:17:25 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Converting VIF {"id": "e5ba6e8d-49a5-4761-ab57-21cee6b0fbdd", "address": "fa:16:3e:6e:b8:a5", "network": {"id": "1acf710e-e626-43a0-a32f-389a87f9858e", "bridge": "br-int", "label": "tempest-AttachSCSIVolumeTestJSON-1052335889-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3f04dea5c1d84aaf94feb47b8ea84f0e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape5ba6e8d-49", "ovs_interfaceid": "e5ba6e8d-49a5-4761-ab57-21cee6b0fbdd", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:17:25 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:6e:b8:a5,bridge_name='br-int',has_traffic_filtering=True,id=e5ba6e8d-49a5-4761-ab57-21cee6b0fbdd,network=Network(1acf710e-e626-43a0-a32f-389a87f9858e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape5ba6e8d-49') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:17:25 user nova-compute[72351]: DEBUG nova.objects.instance [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Lazy-loading 'pci_devices' on Instance uuid 50302d76-f6b9-4fe3-b55a-4a01e483e2e3 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:17:25 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:17:25 user nova-compute[72351]:   <uuid>50302d76-f6b9-4fe3-b55a-4a01e483e2e3</uuid>
Aug 09 02:17:25 user nova-compute[72351]:   <name>instance-00000003</name>
Aug 09 02:17:25 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:17:25 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:17:25 user nova-compute[72351]:   <metadata>
Aug 09 02:17:25 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:17:25 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:17:25 user nova-compute[72351]:       <nova:name>tempest-AttachSCSIVolumeTestJSON-server-1904871607</nova:name>
Aug 09 02:17:25 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:17:25</nova:creationTime>
Aug 09 02:17:25 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:17:25 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:17:25 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:17:25 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:17:25 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:17:25 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:17:25 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:17:25 user nova-compute[72351]:       <nova:owner>
Aug 09 02:17:25 user nova-compute[72351]:         <nova:user uuid="657c221a2b1840ad8e75165d89d779e8">tempest-AttachSCSIVolumeTestJSON-1890923628-project-member</nova:user>
Aug 09 02:17:25 user nova-compute[72351]:         <nova:project uuid="3f04dea5c1d84aaf94feb47b8ea84f0e">tempest-AttachSCSIVolumeTestJSON-1890923628</nova:project>
Aug 09 02:17:25 user nova-compute[72351]:       </nova:owner>
Aug 09 02:17:25 user nova-compute[72351]:       <nova:root type="image" uuid="62f1da90-2927-452b-831f-d34d21ab862a"/>
Aug 09 02:17:25 user nova-compute[72351]:       <nova:ports>
Aug 09 02:17:25 user nova-compute[72351]:         <nova:port uuid="e5ba6e8d-49a5-4761-ab57-21cee6b0fbdd">
Aug 09 02:17:25 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.13" ipVersion="4"/>
Aug 09 02:17:25 user nova-compute[72351]:         </nova:port>
Aug 09 02:17:25 user nova-compute[72351]:       </nova:ports>
Aug 09 02:17:25 user nova-compute[72351]:     </nova:instance>
Aug 09 02:17:25 user nova-compute[72351]:   </metadata>
Aug 09 02:17:25 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:17:25 user nova-compute[72351]:     <system>
Aug 09 02:17:25 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:17:25 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:17:25 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:17:25 user nova-compute[72351]:       <entry name="serial">50302d76-f6b9-4fe3-b55a-4a01e483e2e3</entry>
Aug 09 02:17:25 user nova-compute[72351]:       <entry name="uuid">50302d76-f6b9-4fe3-b55a-4a01e483e2e3</entry>
Aug 09 02:17:25 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:17:25 user nova-compute[72351]:     </system>
Aug 09 02:17:25 user nova-compute[72351]:   </sysinfo>
Aug 09 02:17:25 user nova-compute[72351]:   <os>
Aug 09 02:17:25 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:17:25 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:17:25 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:17:25 user nova-compute[72351]:   </os>
Aug 09 02:17:25 user nova-compute[72351]:   <features>
Aug 09 02:17:25 user nova-compute[72351]:     <acpi/>
Aug 09 02:17:25 user nova-compute[72351]:     <apic/>
Aug 09 02:17:25 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:17:25 user nova-compute[72351]:   </features>
Aug 09 02:17:25 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:17:25 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:17:25 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:17:25 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:17:25 user nova-compute[72351]:   </clock>
Aug 09 02:17:25 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:17:25 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:17:25 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:17:25 user nova-compute[72351]:   </cpu>
Aug 09 02:17:25 user nova-compute[72351]:   <devices>
Aug 09 02:17:25 user nova-compute[72351]:     <disk type="file" device="disk">
Aug 09 02:17:25 user nova-compute[72351]:       <driver name="qemu" type="qcow2" cache="none"/>
Aug 09 02:17:25 user nova-compute[72351]:       <source file="/opt/stack/data/nova/instances/50302d76-f6b9-4fe3-b55a-4a01e483e2e3/disk"/>
Aug 09 02:17:25 user nova-compute[72351]:       <target dev="sda" bus="scsi"/>
Aug 09 02:17:25 user nova-compute[72351]:       <address type="drive" controller="0" unit="0"/>
Aug 09 02:17:25 user nova-compute[72351]:     </disk>
Aug 09 02:17:25 user nova-compute[72351]:     <disk type="file" device="cdrom">
Aug 09 02:17:25 user nova-compute[72351]:       <driver name="qemu" type="raw" cache="none"/>
Aug 09 02:17:25 user nova-compute[72351]:       <source file="/opt/stack/data/nova/instances/50302d76-f6b9-4fe3-b55a-4a01e483e2e3/disk.config"/>
Aug 09 02:17:25 user nova-compute[72351]:       <target dev="sdb" bus="scsi"/>
Aug 09 02:17:25 user nova-compute[72351]:       <address type="drive" controller="0" unit="1"/>
Aug 09 02:17:25 user nova-compute[72351]:     </disk>
Aug 09 02:17:25 user nova-compute[72351]:     <controller type="scsi" index="0" model="virtio-scsi"/>
Aug 09 02:17:25 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:17:25 user nova-compute[72351]:       <mac address="fa:16:3e:6e:b8:a5"/>
Aug 09 02:17:25 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:17:25 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:17:25 user nova-compute[72351]:       <target dev="tape5ba6e8d-49"/>
Aug 09 02:17:25 user nova-compute[72351]:     </interface>
Aug 09 02:17:25 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:17:25 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/50302d76-f6b9-4fe3-b55a-4a01e483e2e3/console.log" append="off"/>
Aug 09 02:17:25 user nova-compute[72351]:     </serial>
Aug 09 02:17:25 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:17:25 user nova-compute[72351]:     <video>
Aug 09 02:17:25 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:17:25 user nova-compute[72351]:     </video>
Aug 09 02:17:25 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:17:25 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:17:25 user nova-compute[72351]:     </rng>
Aug 09 02:17:25 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:17:25 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:17:25 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:17:25 user nova-compute[72351]:     </memballoon>
Aug 09 02:17:25 user nova-compute[72351]:   </devices>
Aug 09 02:17:25 user nova-compute[72351]: </domain>
Aug 09 02:17:25 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:17:25 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-08-09T02:17:18Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachSCSIVolumeTestJSON-server-1904871607',display_name='tempest-AttachSCSIVolumeTestJSON-server-1904871607',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachscsivolumetestjson-server-1904871607',id=3,image_ref='62f1da90-2927-452b-831f-d34d21ab862a',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJGHMVLR+W7uX3EHxXslcn5rNtsVUdS8noPRzLhTwfCXUymvnqG/yjZ6GlWujXjOHQqzGpQI8gBHB2Bq+KvBrqlYY9bJOEmcP7qmKR9OdnSd6Ve+rANep1lfbliuNjFHjg==',key_name='tempest-keypair-259400434',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='3f04dea5c1d84aaf94feb47b8ea84f0e',ramdisk_id='',reservation_id='r-tbsm9tz2',resources=None,root_device_name='/dev/sda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='62f1da90-2927-452b-831f-d34d21ab862a',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='scsi',image_hw_disk_bus='scsi',image_hw_machine_type='pc',image_hw_scsi_model='virtio-scsi',image_min_disk='1',image_min_ram='0',network_allocated='True',owner_project_name='tempest-AttachSCSIVolumeTestJSON-1890923628',owner_user_name='tempest-AttachSCSIVolumeTestJSON-1890923628-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:17:22Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='657c221a2b1840ad8e75165d89d779e8',uuid=50302d76-f6b9-4fe3-b55a-4a01e483e2e3,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "e5ba6e8d-49a5-4761-ab57-21cee6b0fbdd", "address": "fa:16:3e:6e:b8:a5", "network": {"id": "1acf710e-e626-43a0-a32f-389a87f9858e", "bridge": "br-int", "label": "tempest-AttachSCSIVolumeTestJSON-1052335889-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3f04dea5c1d84aaf94feb47b8ea84f0e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape5ba6e8d-49", "ovs_interfaceid": "e5ba6e8d-49a5-4761-ab57-21cee6b0fbdd", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:17:25 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Converting VIF {"id": "e5ba6e8d-49a5-4761-ab57-21cee6b0fbdd", "address": "fa:16:3e:6e:b8:a5", "network": {"id": "1acf710e-e626-43a0-a32f-389a87f9858e", "bridge": "br-int", "label": "tempest-AttachSCSIVolumeTestJSON-1052335889-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3f04dea5c1d84aaf94feb47b8ea84f0e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape5ba6e8d-49", "ovs_interfaceid": "e5ba6e8d-49a5-4761-ab57-21cee6b0fbdd", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:17:25 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:6e:b8:a5,bridge_name='br-int',has_traffic_filtering=True,id=e5ba6e8d-49a5-4761-ab57-21cee6b0fbdd,network=Network(1acf710e-e626-43a0-a32f-389a87f9858e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape5ba6e8d-49') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:17:25 user nova-compute[72351]: DEBUG os_vif [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:6e:b8:a5,bridge_name='br-int',has_traffic_filtering=True,id=e5ba6e8d-49a5-4761-ab57-21cee6b0fbdd,network=Network(1acf710e-e626-43a0-a32f-389a87f9858e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape5ba6e8d-49') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:17:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:17:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:17:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tape5ba6e8d-49, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:17:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tape5ba6e8d-49, col_values=(('external_ids', {'iface-id': 'e5ba6e8d-49a5-4761-ab57-21cee6b0fbdd', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:6e:b8:a5', 'vm-uuid': '50302d76-f6b9-4fe3-b55a-4a01e483e2e3'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:17:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:25 user nova-compute[72351]: INFO os_vif [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:6e:b8:a5,bridge_name='br-int',has_traffic_filtering=True,id=e5ba6e8d-49a5-4761-ab57-21cee6b0fbdd,network=Network(1acf710e-e626-43a0-a32f-389a87f9858e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape5ba6e8d-49')
Aug 09 02:17:25 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] No BDM found with device name sda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:17:25 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] No BDM found with device name sdb, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:17:25 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] No VIF found with MAC fa:16:3e:6e:b8:a5, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:17:25 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Using config drive
Aug 09 02:17:25 user nova-compute[72351]: DEBUG nova.network.neutron [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Successfully created port: 7e7b4229-7c23-4a73-9294-4c4fc9f6a01a {{(pid=72351) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Aug 09 02:17:26 user nova-compute[72351]: DEBUG nova.network.neutron [req-bde26e5d-26ac-48ea-ac60-b189989c5a97 req-35805b11-4c12-4aef-86d9-b84887a057bd service nova] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Updated VIF entry in instance network info cache for port e5ba6e8d-49a5-4761-ab57-21cee6b0fbdd. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:17:26 user nova-compute[72351]: DEBUG nova.network.neutron [req-bde26e5d-26ac-48ea-ac60-b189989c5a97 req-35805b11-4c12-4aef-86d9-b84887a057bd service nova] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Updating instance_info_cache with network_info: [{"id": "e5ba6e8d-49a5-4761-ab57-21cee6b0fbdd", "address": "fa:16:3e:6e:b8:a5", "network": {"id": "1acf710e-e626-43a0-a32f-389a87f9858e", "bridge": "br-int", "label": "tempest-AttachSCSIVolumeTestJSON-1052335889-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3f04dea5c1d84aaf94feb47b8ea84f0e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape5ba6e8d-49", "ovs_interfaceid": "e5ba6e8d-49a5-4761-ab57-21cee6b0fbdd", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:17:26 user nova-compute[72351]: DEBUG nova.network.neutron [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Updating instance_info_cache with network_info: [{"id": "3718df01-0e23-424c-8ab1-bf0fd9c7e121", "address": "fa:16:3e:7c:0b:0d", "network": {"id": "21efbfa4-f3a2-4317-89fc-3cc7b60fde97", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1919568535-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bc5e05fbd7f44f85bc3b1cc169dd03a9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3718df01-0e", "ovs_interfaceid": "3718df01-0e23-424c-8ab1-bf0fd9c7e121", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:17:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-bde26e5d-26ac-48ea-ac60-b189989c5a97 req-35805b11-4c12-4aef-86d9-b84887a057bd service nova] Releasing lock "refresh_cache-50302d76-f6b9-4fe3-b55a-4a01e483e2e3" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:17:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Releasing lock "refresh_cache-74728ca8-34d4-45e6-a1f2-21c0952ff835" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:17:26 user nova-compute[72351]: DEBUG nova.compute.manager [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Instance network_info: |[{"id": "3718df01-0e23-424c-8ab1-bf0fd9c7e121", "address": "fa:16:3e:7c:0b:0d", "network": {"id": "21efbfa4-f3a2-4317-89fc-3cc7b60fde97", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1919568535-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bc5e05fbd7f44f85bc3b1cc169dd03a9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3718df01-0e", "ovs_interfaceid": "3718df01-0e23-424c-8ab1-bf0fd9c7e121", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1971}}
Aug 09 02:17:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-8e85c34c-e067-42f1-a0b0-873289446c22 req-0fb1605b-0aae-4a8a-b85e-6dbea574f951 service nova] Acquired lock "refresh_cache-74728ca8-34d4-45e6-a1f2-21c0952ff835" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:17:26 user nova-compute[72351]: DEBUG nova.network.neutron [req-8e85c34c-e067-42f1-a0b0-873289446c22 req-0fb1605b-0aae-4a8a-b85e-6dbea574f951 service nova] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Refreshing network info cache for port 3718df01-0e23-424c-8ab1-bf0fd9c7e121 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:17:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Start _get_guest_xml network_info=[{"id": "3718df01-0e23-424c-8ab1-bf0fd9c7e121", "address": "fa:16:3e:7c:0b:0d", "network": {"id": "21efbfa4-f3a2-4317-89fc-3cc7b60fde97", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1919568535-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bc5e05fbd7f44f85bc3b1cc169dd03a9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3718df01-0e", "ovs_interfaceid": "3718df01-0e23-424c-8ab1-bf0fd9c7e121", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'size': 0, 'device_type': 'disk', 'encryption_secret_uuid': None, 'guest_format': None, 'encryption_options': None, 'disk_bus': 'virtio', 'encrypted': False, 'encryption_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'image_id': '18ccda7f-31d3-4978-ab9d-4ed2f74eff94'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:17:26 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:17:26 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:17:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:17:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:17:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:17:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:17:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:17:26 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:17:26 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:17:26 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:17:26 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:17:26 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:17:26 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:17:26 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:17:26 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:17:26 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:17:26 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:17:26 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:17:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:17:19Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesAdminNegativeTest-server-342268075',display_name='tempest-VolumesAdminNegativeTest-server-342268075',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-volumesadminnegativetest-server-342268075',id=4,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOAriZVJ2fue+G5VkQcpwuO1FraVLGHA1whj7NY2xEK0SFC2eroBPurQgFtB5AxEfAlnwcHpzvvI/FlO3wNg0bKdfmp7QzjOzEDrSqhn/OdHVQ7YHqei4WWY5uwxumXZ+g==',key_name='tempest-keypair-404137544',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='bc5e05fbd7f44f85bc3b1cc169dd03a9',ramdisk_id='',reservation_id='r-1g3ssrep',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesAdminNegativeTest-35684003',owner_user_name='tempest-VolumesAdminNegativeTest-35684003-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:17:21Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='fb9f73926e0d42f586e7c0055764d611',uuid=74728ca8-34d4-45e6-a1f2-21c0952ff835,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "3718df01-0e23-424c-8ab1-bf0fd9c7e121", "address": "fa:16:3e:7c:0b:0d", "network": {"id": "21efbfa4-f3a2-4317-89fc-3cc7b60fde97", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1919568535-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bc5e05fbd7f44f85bc3b1cc169dd03a9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3718df01-0e", "ovs_interfaceid": "3718df01-0e23-424c-8ab1-bf0fd9c7e121", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:17:26 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Converting VIF {"id": "3718df01-0e23-424c-8ab1-bf0fd9c7e121", "address": "fa:16:3e:7c:0b:0d", "network": {"id": "21efbfa4-f3a2-4317-89fc-3cc7b60fde97", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1919568535-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bc5e05fbd7f44f85bc3b1cc169dd03a9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3718df01-0e", "ovs_interfaceid": "3718df01-0e23-424c-8ab1-bf0fd9c7e121", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:17:26 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:7c:0b:0d,bridge_name='br-int',has_traffic_filtering=True,id=3718df01-0e23-424c-8ab1-bf0fd9c7e121,network=Network(21efbfa4-f3a2-4317-89fc-3cc7b60fde97),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3718df01-0e') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:17:26 user nova-compute[72351]: DEBUG nova.objects.instance [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Lazy-loading 'pci_devices' on Instance uuid 74728ca8-34d4-45e6-a1f2-21c0952ff835 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:17:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:17:26 user nova-compute[72351]:   <uuid>74728ca8-34d4-45e6-a1f2-21c0952ff835</uuid>
Aug 09 02:17:26 user nova-compute[72351]:   <name>instance-00000004</name>
Aug 09 02:17:26 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:17:26 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:17:26 user nova-compute[72351]:   <metadata>
Aug 09 02:17:26 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:17:26 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:17:26 user nova-compute[72351]:       <nova:name>tempest-VolumesAdminNegativeTest-server-342268075</nova:name>
Aug 09 02:17:26 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:17:26</nova:creationTime>
Aug 09 02:17:26 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:17:26 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:17:26 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:17:26 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:17:26 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:17:26 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:17:26 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:17:26 user nova-compute[72351]:       <nova:owner>
Aug 09 02:17:26 user nova-compute[72351]:         <nova:user uuid="fb9f73926e0d42f586e7c0055764d611">tempest-VolumesAdminNegativeTest-35684003-project-member</nova:user>
Aug 09 02:17:26 user nova-compute[72351]:         <nova:project uuid="bc5e05fbd7f44f85bc3b1cc169dd03a9">tempest-VolumesAdminNegativeTest-35684003</nova:project>
Aug 09 02:17:26 user nova-compute[72351]:       </nova:owner>
Aug 09 02:17:26 user nova-compute[72351]:       <nova:root type="image" uuid="18ccda7f-31d3-4978-ab9d-4ed2f74eff94"/>
Aug 09 02:17:26 user nova-compute[72351]:       <nova:ports>
Aug 09 02:17:26 user nova-compute[72351]:         <nova:port uuid="3718df01-0e23-424c-8ab1-bf0fd9c7e121">
Aug 09 02:17:26 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.13" ipVersion="4"/>
Aug 09 02:17:26 user nova-compute[72351]:         </nova:port>
Aug 09 02:17:26 user nova-compute[72351]:       </nova:ports>
Aug 09 02:17:26 user nova-compute[72351]:     </nova:instance>
Aug 09 02:17:26 user nova-compute[72351]:   </metadata>
Aug 09 02:17:26 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:17:26 user nova-compute[72351]:     <system>
Aug 09 02:17:26 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:17:26 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:17:26 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:17:26 user nova-compute[72351]:       <entry name="serial">74728ca8-34d4-45e6-a1f2-21c0952ff835</entry>
Aug 09 02:17:26 user nova-compute[72351]:       <entry name="uuid">74728ca8-34d4-45e6-a1f2-21c0952ff835</entry>
Aug 09 02:17:26 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:17:26 user nova-compute[72351]:     </system>
Aug 09 02:17:26 user nova-compute[72351]:   </sysinfo>
Aug 09 02:17:26 user nova-compute[72351]:   <os>
Aug 09 02:17:26 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:17:26 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:17:26 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:17:26 user nova-compute[72351]:   </os>
Aug 09 02:17:26 user nova-compute[72351]:   <features>
Aug 09 02:17:26 user nova-compute[72351]:     <acpi/>
Aug 09 02:17:26 user nova-compute[72351]:     <apic/>
Aug 09 02:17:26 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:17:26 user nova-compute[72351]:   </features>
Aug 09 02:17:26 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:17:26 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:17:26 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:17:26 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:17:26 user nova-compute[72351]:   </clock>
Aug 09 02:17:26 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:17:26 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:17:26 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:17:26 user nova-compute[72351]:   </cpu>
Aug 09 02:17:26 user nova-compute[72351]:   <devices>
Aug 09 02:17:26 user nova-compute[72351]:     <disk type="file" device="disk">
Aug 09 02:17:26 user nova-compute[72351]:       <driver name="qemu" type="qcow2" cache="none"/>
Aug 09 02:17:26 user nova-compute[72351]:       <source file="/opt/stack/data/nova/instances/74728ca8-34d4-45e6-a1f2-21c0952ff835/disk"/>
Aug 09 02:17:26 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:17:26 user nova-compute[72351]:     </disk>
Aug 09 02:17:26 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:17:26 user nova-compute[72351]:       <mac address="fa:16:3e:7c:0b:0d"/>
Aug 09 02:17:26 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:17:26 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:17:26 user nova-compute[72351]:       <target dev="tap3718df01-0e"/>
Aug 09 02:17:26 user nova-compute[72351]:     </interface>
Aug 09 02:17:26 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:17:26 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/74728ca8-34d4-45e6-a1f2-21c0952ff835/console.log" append="off"/>
Aug 09 02:17:26 user nova-compute[72351]:     </serial>
Aug 09 02:17:26 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:17:26 user nova-compute[72351]:     <video>
Aug 09 02:17:26 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:17:26 user nova-compute[72351]:     </video>
Aug 09 02:17:26 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:17:26 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:17:26 user nova-compute[72351]:     </rng>
Aug 09 02:17:26 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:17:26 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:17:26 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:17:26 user nova-compute[72351]:     </memballoon>
Aug 09 02:17:26 user nova-compute[72351]:   </devices>
Aug 09 02:17:26 user nova-compute[72351]: </domain>
Aug 09 02:17:26 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:17:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:17:19Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesAdminNegativeTest-server-342268075',display_name='tempest-VolumesAdminNegativeTest-server-342268075',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-volumesadminnegativetest-server-342268075',id=4,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOAriZVJ2fue+G5VkQcpwuO1FraVLGHA1whj7NY2xEK0SFC2eroBPurQgFtB5AxEfAlnwcHpzvvI/FlO3wNg0bKdfmp7QzjOzEDrSqhn/OdHVQ7YHqei4WWY5uwxumXZ+g==',key_name='tempest-keypair-404137544',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='bc5e05fbd7f44f85bc3b1cc169dd03a9',ramdisk_id='',reservation_id='r-1g3ssrep',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesAdminNegativeTest-35684003',owner_user_name='tempest-VolumesAdminNegativeTest-35684003-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:17:21Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='fb9f73926e0d42f586e7c0055764d611',uuid=74728ca8-34d4-45e6-a1f2-21c0952ff835,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "3718df01-0e23-424c-8ab1-bf0fd9c7e121", "address": "fa:16:3e:7c:0b:0d", "network": {"id": "21efbfa4-f3a2-4317-89fc-3cc7b60fde97", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1919568535-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bc5e05fbd7f44f85bc3b1cc169dd03a9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3718df01-0e", "ovs_interfaceid": "3718df01-0e23-424c-8ab1-bf0fd9c7e121", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:17:26 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Converting VIF {"id": "3718df01-0e23-424c-8ab1-bf0fd9c7e121", "address": "fa:16:3e:7c:0b:0d", "network": {"id": "21efbfa4-f3a2-4317-89fc-3cc7b60fde97", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1919568535-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bc5e05fbd7f44f85bc3b1cc169dd03a9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3718df01-0e", "ovs_interfaceid": "3718df01-0e23-424c-8ab1-bf0fd9c7e121", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:17:26 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:7c:0b:0d,bridge_name='br-int',has_traffic_filtering=True,id=3718df01-0e23-424c-8ab1-bf0fd9c7e121,network=Network(21efbfa4-f3a2-4317-89fc-3cc7b60fde97),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3718df01-0e') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:17:26 user nova-compute[72351]: DEBUG os_vif [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:7c:0b:0d,bridge_name='br-int',has_traffic_filtering=True,id=3718df01-0e23-424c-8ab1-bf0fd9c7e121,network=Network(21efbfa4-f3a2-4317-89fc-3cc7b60fde97),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3718df01-0e') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:17:26 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:26 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:17:26 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:17:26 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:26 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap3718df01-0e, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:17:26 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap3718df01-0e, col_values=(('external_ids', {'iface-id': '3718df01-0e23-424c-8ab1-bf0fd9c7e121', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:7c:0b:0d', 'vm-uuid': '74728ca8-34d4-45e6-a1f2-21c0952ff835'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:17:26 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:26 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:17:26 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:26 user nova-compute[72351]: INFO os_vif [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:7c:0b:0d,bridge_name='br-int',has_traffic_filtering=True,id=3718df01-0e23-424c-8ab1-bf0fd9c7e121,network=Network(21efbfa4-f3a2-4317-89fc-3cc7b60fde97),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3718df01-0e')
Aug 09 02:17:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:17:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] No VIF found with MAC fa:16:3e:7c:0b:0d, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:17:26 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Creating config drive at /opt/stack/data/nova/instances/50302d76-f6b9-4fe3-b55a-4a01e483e2e3/disk.config
Aug 09 02:17:26 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Running cmd (subprocess): genisoimage -o /opt/stack/data/nova/instances/50302d76-f6b9-4fe3-b55a-4a01e483e2e3/disk.config -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 0.0.0 -quiet -J -r -V config-2 /tmp/tmp64kqrjnj {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:17:26 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] CMD "genisoimage -o /opt/stack/data/nova/instances/50302d76-f6b9-4fe3-b55a-4a01e483e2e3/disk.config -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 0.0.0 -quiet -J -r -V config-2 /tmp/tmp64kqrjnj" returned: 0 in 0.063s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:17:26 user nova-compute[72351]: DEBUG nova.network.neutron [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Successfully updated port: 7e7b4229-7c23-4a73-9294-4c4fc9f6a01a {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:17:27 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Acquiring lock "refresh_cache-7c5e633d-a32f-4c5e-bb9d-1e40da964bd0" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:17:27 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Acquired lock "refresh_cache-7c5e633d-a32f-4c5e-bb9d-1e40da964bd0" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:17:27 user nova-compute[72351]: DEBUG nova.network.neutron [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:17:27 user nova-compute[72351]: DEBUG nova.network.neutron [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:17:27 user nova-compute[72351]: DEBUG nova.network.neutron [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Successfully updated port: a309cd5b-1679-48c6-a596-168b3af2d284 {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:17:27 user nova-compute[72351]: DEBUG nova.network.neutron [req-8e85c34c-e067-42f1-a0b0-873289446c22 req-0fb1605b-0aae-4a8a-b85e-6dbea574f951 service nova] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Updated VIF entry in instance network info cache for port 3718df01-0e23-424c-8ab1-bf0fd9c7e121. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:17:27 user nova-compute[72351]: DEBUG nova.network.neutron [req-8e85c34c-e067-42f1-a0b0-873289446c22 req-0fb1605b-0aae-4a8a-b85e-6dbea574f951 service nova] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Updating instance_info_cache with network_info: [{"id": "3718df01-0e23-424c-8ab1-bf0fd9c7e121", "address": "fa:16:3e:7c:0b:0d", "network": {"id": "21efbfa4-f3a2-4317-89fc-3cc7b60fde97", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1919568535-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bc5e05fbd7f44f85bc3b1cc169dd03a9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3718df01-0e", "ovs_interfaceid": "3718df01-0e23-424c-8ab1-bf0fd9c7e121", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:17:27 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Acquiring lock "refresh_cache-11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:17:27 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Acquired lock "refresh_cache-11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:17:27 user nova-compute[72351]: DEBUG nova.network.neutron [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:17:27 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-8e85c34c-e067-42f1-a0b0-873289446c22 req-0fb1605b-0aae-4a8a-b85e-6dbea574f951 service nova] Releasing lock "refresh_cache-74728ca8-34d4-45e6-a1f2-21c0952ff835" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:17:27 user nova-compute[72351]: DEBUG nova.compute.manager [req-43d9f230-d4f3-4cb5-8fcd-822f7bced57c req-bc50f20d-8fb2-4143-9b59-c357d94bc55f service nova] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Received event network-changed-7e7b4229-7c23-4a73-9294-4c4fc9f6a01a {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:17:27 user nova-compute[72351]: DEBUG nova.compute.manager [req-43d9f230-d4f3-4cb5-8fcd-822f7bced57c req-bc50f20d-8fb2-4143-9b59-c357d94bc55f service nova] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Refreshing instance network info cache due to event network-changed-7e7b4229-7c23-4a73-9294-4c4fc9f6a01a. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:17:27 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-43d9f230-d4f3-4cb5-8fcd-822f7bced57c req-bc50f20d-8fb2-4143-9b59-c357d94bc55f service nova] Acquiring lock "refresh_cache-7c5e633d-a32f-4c5e-bb9d-1e40da964bd0" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:17:27 user nova-compute[72351]: DEBUG nova.compute.manager [req-43196e6c-db94-43b3-a15b-12d850919f9a req-9678b041-dc42-4c4e-8de8-3d0689cf0807 service nova] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Received event network-changed-a309cd5b-1679-48c6-a596-168b3af2d284 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:17:27 user nova-compute[72351]: DEBUG nova.compute.manager [req-43196e6c-db94-43b3-a15b-12d850919f9a req-9678b041-dc42-4c4e-8de8-3d0689cf0807 service nova] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Refreshing instance network info cache due to event network-changed-a309cd5b-1679-48c6-a596-168b3af2d284. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:17:27 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-43196e6c-db94-43b3-a15b-12d850919f9a req-9678b041-dc42-4c4e-8de8-3d0689cf0807 service nova] Acquiring lock "refresh_cache-11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:17:27 user nova-compute[72351]: DEBUG nova.network.neutron [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:17:27 user nova-compute[72351]: DEBUG nova.network.neutron [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Updating instance_info_cache with network_info: [{"id": "a309cd5b-1679-48c6-a596-168b3af2d284", "address": "fa:16:3e:dc:f0:3c", "network": {"id": "da9c66d8-08c8-4c9c-aec7-c9ad980b89e4", "bridge": "br-int", "label": "tempest-TestVolumeSwap-1327534036-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "1277988bc35041249b8e315df5259f98", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa309cd5b-16", "ovs_interfaceid": "a309cd5b-1679-48c6-a596-168b3af2d284", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:17:27 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Releasing lock "refresh_cache-11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:17:27 user nova-compute[72351]: DEBUG nova.compute.manager [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Instance network_info: |[{"id": "a309cd5b-1679-48c6-a596-168b3af2d284", "address": "fa:16:3e:dc:f0:3c", "network": {"id": "da9c66d8-08c8-4c9c-aec7-c9ad980b89e4", "bridge": "br-int", "label": "tempest-TestVolumeSwap-1327534036-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "1277988bc35041249b8e315df5259f98", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa309cd5b-16", "ovs_interfaceid": "a309cd5b-1679-48c6-a596-168b3af2d284", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1971}}
Aug 09 02:17:27 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-43196e6c-db94-43b3-a15b-12d850919f9a req-9678b041-dc42-4c4e-8de8-3d0689cf0807 service nova] Acquired lock "refresh_cache-11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:17:27 user nova-compute[72351]: DEBUG nova.network.neutron [req-43196e6c-db94-43b3-a15b-12d850919f9a req-9678b041-dc42-4c4e-8de8-3d0689cf0807 service nova] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Refreshing network info cache for port a309cd5b-1679-48c6-a596-168b3af2d284 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:17:27 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Start _get_guest_xml network_info=[{"id": "a309cd5b-1679-48c6-a596-168b3af2d284", "address": "fa:16:3e:dc:f0:3c", "network": {"id": "da9c66d8-08c8-4c9c-aec7-c9ad980b89e4", "bridge": "br-int", "label": "tempest-TestVolumeSwap-1327534036-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "1277988bc35041249b8e315df5259f98", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa309cd5b-16", "ovs_interfaceid": "a309cd5b-1679-48c6-a596-168b3af2d284", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'size': 0, 'device_type': 'disk', 'encryption_secret_uuid': None, 'guest_format': None, 'encryption_options': None, 'disk_bus': 'virtio', 'encrypted': False, 'encryption_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'image_id': '18ccda7f-31d3-4978-ab9d-4ed2f74eff94'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:17:27 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:17:27 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:17:27 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:17:27 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:17:19Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeSwap-server-119194024',display_name='tempest-TestVolumeSwap-server-119194024',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testvolumeswap-server-119194024',id=5,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBARbMYIq63VHbJ/MdcixbKY0L/h+74ffTkbUbHAN5B89Y/PUcG1NPaAWv1yx/WtztC1ALDJHBziTUBRaQrA68gE233M+Zzf5mD/gaihU2S129LWBcEuS8kxOAPQEOccAWw==',key_name='tempest-keypair-1608513669',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='1277988bc35041249b8e315df5259f98',ramdisk_id='',reservation_id='r-iosy2zx7',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestVolumeSwap-646330225',owner_user_name='tempest-TestVolumeSwap-646330225-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:17:23Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='876134724b18462aba75896ddfad5705',uuid=11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "a309cd5b-1679-48c6-a596-168b3af2d284", "address": "fa:16:3e:dc:f0:3c", "network": {"id": "da9c66d8-08c8-4c9c-aec7-c9ad980b89e4", "bridge": "br-int", "label": "tempest-TestVolumeSwap-1327534036-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "1277988bc35041249b8e315df5259f98", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa309cd5b-16", "ovs_interfaceid": "a309cd5b-1679-48c6-a596-168b3af2d284", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Converting VIF {"id": "a309cd5b-1679-48c6-a596-168b3af2d284", "address": "fa:16:3e:dc:f0:3c", "network": {"id": "da9c66d8-08c8-4c9c-aec7-c9ad980b89e4", "bridge": "br-int", "label": "tempest-TestVolumeSwap-1327534036-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "1277988bc35041249b8e315df5259f98", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa309cd5b-16", "ovs_interfaceid": "a309cd5b-1679-48c6-a596-168b3af2d284", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:dc:f0:3c,bridge_name='br-int',has_traffic_filtering=True,id=a309cd5b-1679-48c6-a596-168b3af2d284,network=Network(da9c66d8-08c8-4c9c-aec7-c9ad980b89e4),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa309cd5b-16') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG nova.objects.instance [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Lazy-loading 'pci_devices' on Instance uuid 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG nova.network.neutron [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Updating instance_info_cache with network_info: [{"id": "7e7b4229-7c23-4a73-9294-4c4fc9f6a01a", "address": "fa:16:3e:d9:0b:0f", "network": {"id": "49cd7d7c-b635-484c-9ef3-268529d7344f", "bridge": "br-int", "label": "tempest-VolumesAssistedSnapshotsTest-24910214-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "26827058be4b48a3a7aed21aad7c5e77", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7e7b4229-7c", "ovs_interfaceid": "7e7b4229-7c23-4a73-9294-4c4fc9f6a01a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:17:28 user nova-compute[72351]:   <uuid>11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b</uuid>
Aug 09 02:17:28 user nova-compute[72351]:   <name>instance-00000005</name>
Aug 09 02:17:28 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:17:28 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:17:28 user nova-compute[72351]:   <metadata>
Aug 09 02:17:28 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:17:28 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:17:28 user nova-compute[72351]:       <nova:name>tempest-TestVolumeSwap-server-119194024</nova:name>
Aug 09 02:17:28 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:17:27</nova:creationTime>
Aug 09 02:17:28 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:17:28 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:17:28 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:17:28 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:17:28 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:17:28 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:17:28 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:17:28 user nova-compute[72351]:       <nova:owner>
Aug 09 02:17:28 user nova-compute[72351]:         <nova:user uuid="876134724b18462aba75896ddfad5705">tempest-TestVolumeSwap-646330225-project-member</nova:user>
Aug 09 02:17:28 user nova-compute[72351]:         <nova:project uuid="1277988bc35041249b8e315df5259f98">tempest-TestVolumeSwap-646330225</nova:project>
Aug 09 02:17:28 user nova-compute[72351]:       </nova:owner>
Aug 09 02:17:28 user nova-compute[72351]:       <nova:root type="image" uuid="18ccda7f-31d3-4978-ab9d-4ed2f74eff94"/>
Aug 09 02:17:28 user nova-compute[72351]:       <nova:ports>
Aug 09 02:17:28 user nova-compute[72351]:         <nova:port uuid="a309cd5b-1679-48c6-a596-168b3af2d284">
Aug 09 02:17:28 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.11" ipVersion="4"/>
Aug 09 02:17:28 user nova-compute[72351]:         </nova:port>
Aug 09 02:17:28 user nova-compute[72351]:       </nova:ports>
Aug 09 02:17:28 user nova-compute[72351]:     </nova:instance>
Aug 09 02:17:28 user nova-compute[72351]:   </metadata>
Aug 09 02:17:28 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:17:28 user nova-compute[72351]:     <system>
Aug 09 02:17:28 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:17:28 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:17:28 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:17:28 user nova-compute[72351]:       <entry name="serial">11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b</entry>
Aug 09 02:17:28 user nova-compute[72351]:       <entry name="uuid">11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b</entry>
Aug 09 02:17:28 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:17:28 user nova-compute[72351]:     </system>
Aug 09 02:17:28 user nova-compute[72351]:   </sysinfo>
Aug 09 02:17:28 user nova-compute[72351]:   <os>
Aug 09 02:17:28 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:17:28 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:17:28 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:17:28 user nova-compute[72351]:   </os>
Aug 09 02:17:28 user nova-compute[72351]:   <features>
Aug 09 02:17:28 user nova-compute[72351]:     <acpi/>
Aug 09 02:17:28 user nova-compute[72351]:     <apic/>
Aug 09 02:17:28 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:17:28 user nova-compute[72351]:   </features>
Aug 09 02:17:28 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:17:28 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:17:28 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:17:28 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:17:28 user nova-compute[72351]:   </clock>
Aug 09 02:17:28 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:17:28 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:17:28 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:17:28 user nova-compute[72351]:   </cpu>
Aug 09 02:17:28 user nova-compute[72351]:   <devices>
Aug 09 02:17:28 user nova-compute[72351]:     <disk type="file" device="disk">
Aug 09 02:17:28 user nova-compute[72351]:       <driver name="qemu" type="qcow2" cache="none"/>
Aug 09 02:17:28 user nova-compute[72351]:       <source file="/opt/stack/data/nova/instances/11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b/disk"/>
Aug 09 02:17:28 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:17:28 user nova-compute[72351]:     </disk>
Aug 09 02:17:28 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:17:28 user nova-compute[72351]:       <mac address="fa:16:3e:dc:f0:3c"/>
Aug 09 02:17:28 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:17:28 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:17:28 user nova-compute[72351]:       <target dev="tapa309cd5b-16"/>
Aug 09 02:17:28 user nova-compute[72351]:     </interface>
Aug 09 02:17:28 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:17:28 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b/console.log" append="off"/>
Aug 09 02:17:28 user nova-compute[72351]:     </serial>
Aug 09 02:17:28 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:17:28 user nova-compute[72351]:     <video>
Aug 09 02:17:28 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:17:28 user nova-compute[72351]:     </video>
Aug 09 02:17:28 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:17:28 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:17:28 user nova-compute[72351]:     </rng>
Aug 09 02:17:28 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:17:28 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:17:28 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:17:28 user nova-compute[72351]:     </memballoon>
Aug 09 02:17:28 user nova-compute[72351]:   </devices>
Aug 09 02:17:28 user nova-compute[72351]: </domain>
Aug 09 02:17:28 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:17:19Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeSwap-server-119194024',display_name='tempest-TestVolumeSwap-server-119194024',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testvolumeswap-server-119194024',id=5,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBARbMYIq63VHbJ/MdcixbKY0L/h+74ffTkbUbHAN5B89Y/PUcG1NPaAWv1yx/WtztC1ALDJHBziTUBRaQrA68gE233M+Zzf5mD/gaihU2S129LWBcEuS8kxOAPQEOccAWw==',key_name='tempest-keypair-1608513669',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='1277988bc35041249b8e315df5259f98',ramdisk_id='',reservation_id='r-iosy2zx7',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestVolumeSwap-646330225',owner_user_name='tempest-TestVolumeSwap-646330225-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:17:23Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='876134724b18462aba75896ddfad5705',uuid=11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "a309cd5b-1679-48c6-a596-168b3af2d284", "address": "fa:16:3e:dc:f0:3c", "network": {"id": "da9c66d8-08c8-4c9c-aec7-c9ad980b89e4", "bridge": "br-int", "label": "tempest-TestVolumeSwap-1327534036-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "1277988bc35041249b8e315df5259f98", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa309cd5b-16", "ovs_interfaceid": "a309cd5b-1679-48c6-a596-168b3af2d284", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Converting VIF {"id": "a309cd5b-1679-48c6-a596-168b3af2d284", "address": "fa:16:3e:dc:f0:3c", "network": {"id": "da9c66d8-08c8-4c9c-aec7-c9ad980b89e4", "bridge": "br-int", "label": "tempest-TestVolumeSwap-1327534036-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "1277988bc35041249b8e315df5259f98", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa309cd5b-16", "ovs_interfaceid": "a309cd5b-1679-48c6-a596-168b3af2d284", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:dc:f0:3c,bridge_name='br-int',has_traffic_filtering=True,id=a309cd5b-1679-48c6-a596-168b3af2d284,network=Network(da9c66d8-08c8-4c9c-aec7-c9ad980b89e4),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa309cd5b-16') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG os_vif [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:dc:f0:3c,bridge_name='br-int',has_traffic_filtering=True,id=a309cd5b-1679-48c6-a596-168b3af2d284,network=Network(da9c66d8-08c8-4c9c-aec7-c9ad980b89e4),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa309cd5b-16') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Releasing lock "refresh_cache-7c5e633d-a32f-4c5e-bb9d-1e40da964bd0" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG nova.compute.manager [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Instance network_info: |[{"id": "7e7b4229-7c23-4a73-9294-4c4fc9f6a01a", "address": "fa:16:3e:d9:0b:0f", "network": {"id": "49cd7d7c-b635-484c-9ef3-268529d7344f", "bridge": "br-int", "label": "tempest-VolumesAssistedSnapshotsTest-24910214-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "26827058be4b48a3a7aed21aad7c5e77", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7e7b4229-7c", "ovs_interfaceid": "7e7b4229-7c23-4a73-9294-4c4fc9f6a01a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1971}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-43d9f230-d4f3-4cb5-8fcd-822f7bced57c req-bc50f20d-8fb2-4143-9b59-c357d94bc55f service nova] Acquired lock "refresh_cache-7c5e633d-a32f-4c5e-bb9d-1e40da964bd0" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG nova.network.neutron [req-43d9f230-d4f3-4cb5-8fcd-822f7bced57c req-bc50f20d-8fb2-4143-9b59-c357d94bc55f service nova] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Refreshing network info cache for port 7e7b4229-7c23-4a73-9294-4c4fc9f6a01a {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Start _get_guest_xml network_info=[{"id": "7e7b4229-7c23-4a73-9294-4c4fc9f6a01a", "address": "fa:16:3e:d9:0b:0f", "network": {"id": "49cd7d7c-b635-484c-9ef3-268529d7344f", "bridge": "br-int", "label": "tempest-VolumesAssistedSnapshotsTest-24910214-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "26827058be4b48a3a7aed21aad7c5e77", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7e7b4229-7c", "ovs_interfaceid": "7e7b4229-7c23-4a73-9294-4c4fc9f6a01a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'size': 0, 'device_type': 'disk', 'encryption_secret_uuid': None, 'guest_format': None, 'encryption_options': None, 'disk_bus': 'virtio', 'encrypted': False, 'encryption_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'image_id': '18ccda7f-31d3-4978-ab9d-4ed2f74eff94'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapa309cd5b-16, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapa309cd5b-16, col_values=(('external_ids', {'iface-id': 'a309cd5b-1679-48c6-a596-168b3af2d284', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:dc:f0:3c', 'vm-uuid': '11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:17:28 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:17:28 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:17:28 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:28 user nova-compute[72351]: INFO os_vif [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:dc:f0:3c,bridge_name='br-int',has_traffic_filtering=True,id=a309cd5b-1679-48c6-a596-168b3af2d284,network=Network(da9c66d8-08c8-4c9c-aec7-c9ad980b89e4),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa309cd5b-16')
Aug 09 02:17:28 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:17:21Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesAssistedSnapshotsTest-server-340980583',display_name='tempest-VolumesAssistedSnapshotsTest-server-340980583',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-volumesassistedsnapshotstest-server-340980583',id=6,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBAHay+TchlGaaKhLpYV0vC5z6GvCYSIp4aglFVFWN37IvZ4glkz7z31jiUXDM9oqyU3JNQYda4Z7n5Vj2jgZz/+uvC2MLvz6lGYCHuZO0g1H0LqpN/JGwocv3XJU6QV9kA==',key_name='tempest-keypair-1914691164',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='26827058be4b48a3a7aed21aad7c5e77',ramdisk_id='',reservation_id='r-pij98zku',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesAssistedSnapshotsTest-1116923063',owner_user_name='tempest-VolumesAssistedSnapshotsTest-1116923063-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:17:24Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='52bc0e2d81424ca19722e8af947d399b',uuid=7c5e633d-a32f-4c5e-bb9d-1e40da964bd0,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "7e7b4229-7c23-4a73-9294-4c4fc9f6a01a", "address": "fa:16:3e:d9:0b:0f", "network": {"id": "49cd7d7c-b635-484c-9ef3-268529d7344f", "bridge": "br-int", "label": "tempest-VolumesAssistedSnapshotsTest-24910214-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "26827058be4b48a3a7aed21aad7c5e77", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7e7b4229-7c", "ovs_interfaceid": "7e7b4229-7c23-4a73-9294-4c4fc9f6a01a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Converting VIF {"id": "7e7b4229-7c23-4a73-9294-4c4fc9f6a01a", "address": "fa:16:3e:d9:0b:0f", "network": {"id": "49cd7d7c-b635-484c-9ef3-268529d7344f", "bridge": "br-int", "label": "tempest-VolumesAssistedSnapshotsTest-24910214-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "26827058be4b48a3a7aed21aad7c5e77", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7e7b4229-7c", "ovs_interfaceid": "7e7b4229-7c23-4a73-9294-4c4fc9f6a01a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:d9:0b:0f,bridge_name='br-int',has_traffic_filtering=True,id=7e7b4229-7c23-4a73-9294-4c4fc9f6a01a,network=Network(49cd7d7c-b635-484c-9ef3-268529d7344f),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7e7b4229-7c') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG nova.objects.instance [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Lazy-loading 'pci_devices' on Instance uuid 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:17:28 user nova-compute[72351]:   <uuid>7c5e633d-a32f-4c5e-bb9d-1e40da964bd0</uuid>
Aug 09 02:17:28 user nova-compute[72351]:   <name>instance-00000006</name>
Aug 09 02:17:28 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:17:28 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:17:28 user nova-compute[72351]:   <metadata>
Aug 09 02:17:28 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:17:28 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:17:28 user nova-compute[72351]:       <nova:name>tempest-VolumesAssistedSnapshotsTest-server-340980583</nova:name>
Aug 09 02:17:28 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:17:28</nova:creationTime>
Aug 09 02:17:28 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:17:28 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:17:28 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:17:28 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:17:28 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:17:28 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:17:28 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:17:28 user nova-compute[72351]:       <nova:owner>
Aug 09 02:17:28 user nova-compute[72351]:         <nova:user uuid="52bc0e2d81424ca19722e8af947d399b">tempest-VolumesAssistedSnapshotsTest-1116923063-project-member</nova:user>
Aug 09 02:17:28 user nova-compute[72351]:         <nova:project uuid="26827058be4b48a3a7aed21aad7c5e77">tempest-VolumesAssistedSnapshotsTest-1116923063</nova:project>
Aug 09 02:17:28 user nova-compute[72351]:       </nova:owner>
Aug 09 02:17:28 user nova-compute[72351]:       <nova:root type="image" uuid="18ccda7f-31d3-4978-ab9d-4ed2f74eff94"/>
Aug 09 02:17:28 user nova-compute[72351]:       <nova:ports>
Aug 09 02:17:28 user nova-compute[72351]:         <nova:port uuid="7e7b4229-7c23-4a73-9294-4c4fc9f6a01a">
Aug 09 02:17:28 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.11" ipVersion="4"/>
Aug 09 02:17:28 user nova-compute[72351]:         </nova:port>
Aug 09 02:17:28 user nova-compute[72351]:       </nova:ports>
Aug 09 02:17:28 user nova-compute[72351]:     </nova:instance>
Aug 09 02:17:28 user nova-compute[72351]:   </metadata>
Aug 09 02:17:28 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:17:28 user nova-compute[72351]:     <system>
Aug 09 02:17:28 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:17:28 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:17:28 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:17:28 user nova-compute[72351]:       <entry name="serial">7c5e633d-a32f-4c5e-bb9d-1e40da964bd0</entry>
Aug 09 02:17:28 user nova-compute[72351]:       <entry name="uuid">7c5e633d-a32f-4c5e-bb9d-1e40da964bd0</entry>
Aug 09 02:17:28 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:17:28 user nova-compute[72351]:     </system>
Aug 09 02:17:28 user nova-compute[72351]:   </sysinfo>
Aug 09 02:17:28 user nova-compute[72351]:   <os>
Aug 09 02:17:28 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:17:28 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:17:28 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:17:28 user nova-compute[72351]:   </os>
Aug 09 02:17:28 user nova-compute[72351]:   <features>
Aug 09 02:17:28 user nova-compute[72351]:     <acpi/>
Aug 09 02:17:28 user nova-compute[72351]:     <apic/>
Aug 09 02:17:28 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:17:28 user nova-compute[72351]:   </features>
Aug 09 02:17:28 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:17:28 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:17:28 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:17:28 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:17:28 user nova-compute[72351]:   </clock>
Aug 09 02:17:28 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:17:28 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:17:28 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:17:28 user nova-compute[72351]:   </cpu>
Aug 09 02:17:28 user nova-compute[72351]:   <devices>
Aug 09 02:17:28 user nova-compute[72351]:     <disk type="file" device="disk">
Aug 09 02:17:28 user nova-compute[72351]:       <driver name="qemu" type="qcow2" cache="none"/>
Aug 09 02:17:28 user nova-compute[72351]:       <source file="/opt/stack/data/nova/instances/7c5e633d-a32f-4c5e-bb9d-1e40da964bd0/disk"/>
Aug 09 02:17:28 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:17:28 user nova-compute[72351]:     </disk>
Aug 09 02:17:28 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:17:28 user nova-compute[72351]:       <mac address="fa:16:3e:d9:0b:0f"/>
Aug 09 02:17:28 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:17:28 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:17:28 user nova-compute[72351]:       <target dev="tap7e7b4229-7c"/>
Aug 09 02:17:28 user nova-compute[72351]:     </interface>
Aug 09 02:17:28 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:17:28 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/7c5e633d-a32f-4c5e-bb9d-1e40da964bd0/console.log" append="off"/>
Aug 09 02:17:28 user nova-compute[72351]:     </serial>
Aug 09 02:17:28 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:17:28 user nova-compute[72351]:     <video>
Aug 09 02:17:28 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:17:28 user nova-compute[72351]:     </video>
Aug 09 02:17:28 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:17:28 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:17:28 user nova-compute[72351]:     </rng>
Aug 09 02:17:28 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:17:28 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:17:28 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:17:28 user nova-compute[72351]:     </memballoon>
Aug 09 02:17:28 user nova-compute[72351]:   </devices>
Aug 09 02:17:28 user nova-compute[72351]: </domain>
Aug 09 02:17:28 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:17:21Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesAssistedSnapshotsTest-server-340980583',display_name='tempest-VolumesAssistedSnapshotsTest-server-340980583',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-volumesassistedsnapshotstest-server-340980583',id=6,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBAHay+TchlGaaKhLpYV0vC5z6GvCYSIp4aglFVFWN37IvZ4glkz7z31jiUXDM9oqyU3JNQYda4Z7n5Vj2jgZz/+uvC2MLvz6lGYCHuZO0g1H0LqpN/JGwocv3XJU6QV9kA==',key_name='tempest-keypair-1914691164',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='26827058be4b48a3a7aed21aad7c5e77',ramdisk_id='',reservation_id='r-pij98zku',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesAssistedSnapshotsTest-1116923063',owner_user_name='tempest-VolumesAssistedSnapshotsTest-1116923063-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:17:24Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='52bc0e2d81424ca19722e8af947d399b',uuid=7c5e633d-a32f-4c5e-bb9d-1e40da964bd0,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "7e7b4229-7c23-4a73-9294-4c4fc9f6a01a", "address": "fa:16:3e:d9:0b:0f", "network": {"id": "49cd7d7c-b635-484c-9ef3-268529d7344f", "bridge": "br-int", "label": "tempest-VolumesAssistedSnapshotsTest-24910214-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "26827058be4b48a3a7aed21aad7c5e77", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7e7b4229-7c", "ovs_interfaceid": "7e7b4229-7c23-4a73-9294-4c4fc9f6a01a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Converting VIF {"id": "7e7b4229-7c23-4a73-9294-4c4fc9f6a01a", "address": "fa:16:3e:d9:0b:0f", "network": {"id": "49cd7d7c-b635-484c-9ef3-268529d7344f", "bridge": "br-int", "label": "tempest-VolumesAssistedSnapshotsTest-24910214-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "26827058be4b48a3a7aed21aad7c5e77", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7e7b4229-7c", "ovs_interfaceid": "7e7b4229-7c23-4a73-9294-4c4fc9f6a01a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:d9:0b:0f,bridge_name='br-int',has_traffic_filtering=True,id=7e7b4229-7c23-4a73-9294-4c4fc9f6a01a,network=Network(49cd7d7c-b635-484c-9ef3-268529d7344f),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7e7b4229-7c') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG os_vif [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:d9:0b:0f,bridge_name='br-int',has_traffic_filtering=True,id=7e7b4229-7c23-4a73-9294-4c4fc9f6a01a,network=Network(49cd7d7c-b635-484c-9ef3-268529d7344f),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7e7b4229-7c') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap7e7b4229-7c, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap7e7b4229-7c, col_values=(('external_ids', {'iface-id': '7e7b4229-7c23-4a73-9294-4c4fc9f6a01a', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:d9:0b:0f', 'vm-uuid': '7c5e633d-a32f-4c5e-bb9d-1e40da964bd0'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] No VIF found with MAC fa:16:3e:dc:f0:3c, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:28 user nova-compute[72351]: INFO os_vif [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:d9:0b:0f,bridge_name='br-int',has_traffic_filtering=True,id=7e7b4229-7c23-4a73-9294-4c4fc9f6a01a,network=Network(49cd7d7c-b635-484c-9ef3-268529d7344f),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7e7b4229-7c')
Aug 09 02:17:28 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] No VIF found with MAC fa:16:3e:d9:0b:0f, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG nova.network.neutron [req-43196e6c-db94-43b3-a15b-12d850919f9a req-9678b041-dc42-4c4e-8de8-3d0689cf0807 service nova] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Updated VIF entry in instance network info cache for port a309cd5b-1679-48c6-a596-168b3af2d284. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG nova.network.neutron [req-43196e6c-db94-43b3-a15b-12d850919f9a req-9678b041-dc42-4c4e-8de8-3d0689cf0807 service nova] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Updating instance_info_cache with network_info: [{"id": "a309cd5b-1679-48c6-a596-168b3af2d284", "address": "fa:16:3e:dc:f0:3c", "network": {"id": "da9c66d8-08c8-4c9c-aec7-c9ad980b89e4", "bridge": "br-int", "label": "tempest-TestVolumeSwap-1327534036-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "1277988bc35041249b8e315df5259f98", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa309cd5b-16", "ovs_interfaceid": "a309cd5b-1679-48c6-a596-168b3af2d284", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-43196e6c-db94-43b3-a15b-12d850919f9a req-9678b041-dc42-4c4e-8de8-3d0689cf0807 service nova] Releasing lock "refresh_cache-11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG nova.network.neutron [req-43d9f230-d4f3-4cb5-8fcd-822f7bced57c req-bc50f20d-8fb2-4143-9b59-c357d94bc55f service nova] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Updated VIF entry in instance network info cache for port 7e7b4229-7c23-4a73-9294-4c4fc9f6a01a. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG nova.network.neutron [req-43d9f230-d4f3-4cb5-8fcd-822f7bced57c req-bc50f20d-8fb2-4143-9b59-c357d94bc55f service nova] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Updating instance_info_cache with network_info: [{"id": "7e7b4229-7c23-4a73-9294-4c4fc9f6a01a", "address": "fa:16:3e:d9:0b:0f", "network": {"id": "49cd7d7c-b635-484c-9ef3-268529d7344f", "bridge": "br-int", "label": "tempest-VolumesAssistedSnapshotsTest-24910214-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "26827058be4b48a3a7aed21aad7c5e77", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7e7b4229-7c", "ovs_interfaceid": "7e7b4229-7c23-4a73-9294-4c4fc9f6a01a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:17:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-43d9f230-d4f3-4cb5-8fcd-822f7bced57c req-bc50f20d-8fb2-4143-9b59-c357d94bc55f service nova] Releasing lock "refresh_cache-7c5e633d-a32f-4c5e-bb9d-1e40da964bd0" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:17:30 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:30 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:30 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:30 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:30 user nova-compute[72351]: DEBUG nova.compute.manager [req-101f9876-1a10-42eb-ae65-7f280502d06c req-8ed098fd-dc8e-4c79-a2eb-a8b3acb52b5a service nova] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Received event network-vif-plugged-e5ba6e8d-49a5-4761-ab57-21cee6b0fbdd {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:17:30 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-101f9876-1a10-42eb-ae65-7f280502d06c req-8ed098fd-dc8e-4c79-a2eb-a8b3acb52b5a service nova] Acquiring lock "50302d76-f6b9-4fe3-b55a-4a01e483e2e3-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:17:30 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-101f9876-1a10-42eb-ae65-7f280502d06c req-8ed098fd-dc8e-4c79-a2eb-a8b3acb52b5a service nova] Lock "50302d76-f6b9-4fe3-b55a-4a01e483e2e3-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:17:30 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-101f9876-1a10-42eb-ae65-7f280502d06c req-8ed098fd-dc8e-4c79-a2eb-a8b3acb52b5a service nova] Lock "50302d76-f6b9-4fe3-b55a-4a01e483e2e3-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:17:30 user nova-compute[72351]: DEBUG nova.compute.manager [req-101f9876-1a10-42eb-ae65-7f280502d06c req-8ed098fd-dc8e-4c79-a2eb-a8b3acb52b5a service nova] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] No waiting events found dispatching network-vif-plugged-e5ba6e8d-49a5-4761-ab57-21cee6b0fbdd {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:17:30 user nova-compute[72351]: WARNING nova.compute.manager [req-101f9876-1a10-42eb-ae65-7f280502d06c req-8ed098fd-dc8e-4c79-a2eb-a8b3acb52b5a service nova] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Received unexpected event network-vif-plugged-e5ba6e8d-49a5-4761-ab57-21cee6b0fbdd for instance with vm_state building and task_state spawning.
Aug 09 02:17:30 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:30 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:30 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:30 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:30 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:30 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:31 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:31 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:31 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:31 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:31 user nova-compute[72351]: DEBUG nova.compute.manager [req-6b50c862-9465-453a-9db1-9fbf9543e6f1 req-54f2ab06-fa7a-4a56-85d0-1d3903b86cf5 service nova] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Received event network-vif-plugged-3718df01-0e23-424c-8ab1-bf0fd9c7e121 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:17:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-6b50c862-9465-453a-9db1-9fbf9543e6f1 req-54f2ab06-fa7a-4a56-85d0-1d3903b86cf5 service nova] Acquiring lock "74728ca8-34d4-45e6-a1f2-21c0952ff835-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:17:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-6b50c862-9465-453a-9db1-9fbf9543e6f1 req-54f2ab06-fa7a-4a56-85d0-1d3903b86cf5 service nova] Lock "74728ca8-34d4-45e6-a1f2-21c0952ff835-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:17:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-6b50c862-9465-453a-9db1-9fbf9543e6f1 req-54f2ab06-fa7a-4a56-85d0-1d3903b86cf5 service nova] Lock "74728ca8-34d4-45e6-a1f2-21c0952ff835-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:17:31 user nova-compute[72351]: DEBUG nova.compute.manager [req-6b50c862-9465-453a-9db1-9fbf9543e6f1 req-54f2ab06-fa7a-4a56-85d0-1d3903b86cf5 service nova] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] No waiting events found dispatching network-vif-plugged-3718df01-0e23-424c-8ab1-bf0fd9c7e121 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:17:31 user nova-compute[72351]: WARNING nova.compute.manager [req-6b50c862-9465-453a-9db1-9fbf9543e6f1 req-54f2ab06-fa7a-4a56-85d0-1d3903b86cf5 service nova] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Received unexpected event network-vif-plugged-3718df01-0e23-424c-8ab1-bf0fd9c7e121 for instance with vm_state building and task_state spawning.
Aug 09 02:17:31 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:31 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:31 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:31 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:31 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:32 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:32 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:32 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:32 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:32 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:32 user nova-compute[72351]: DEBUG nova.compute.manager [req-803675ab-b6e8-4db4-9482-1886ca7de883 req-19eec826-9d36-4b0f-9ff8-07ab11e26eff service nova] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Received event network-vif-plugged-e5ba6e8d-49a5-4761-ab57-21cee6b0fbdd {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:17:32 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-803675ab-b6e8-4db4-9482-1886ca7de883 req-19eec826-9d36-4b0f-9ff8-07ab11e26eff service nova] Acquiring lock "50302d76-f6b9-4fe3-b55a-4a01e483e2e3-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:17:32 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-803675ab-b6e8-4db4-9482-1886ca7de883 req-19eec826-9d36-4b0f-9ff8-07ab11e26eff service nova] Lock "50302d76-f6b9-4fe3-b55a-4a01e483e2e3-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:17:32 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-803675ab-b6e8-4db4-9482-1886ca7de883 req-19eec826-9d36-4b0f-9ff8-07ab11e26eff service nova] Lock "50302d76-f6b9-4fe3-b55a-4a01e483e2e3-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:17:32 user nova-compute[72351]: DEBUG nova.compute.manager [req-803675ab-b6e8-4db4-9482-1886ca7de883 req-19eec826-9d36-4b0f-9ff8-07ab11e26eff service nova] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] No waiting events found dispatching network-vif-plugged-e5ba6e8d-49a5-4761-ab57-21cee6b0fbdd {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:17:32 user nova-compute[72351]: WARNING nova.compute.manager [req-803675ab-b6e8-4db4-9482-1886ca7de883 req-19eec826-9d36-4b0f-9ff8-07ab11e26eff service nova] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Received unexpected event network-vif-plugged-e5ba6e8d-49a5-4761-ab57-21cee6b0fbdd for instance with vm_state building and task_state spawning.
Aug 09 02:17:32 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:32 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:32 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:32 user nova-compute[72351]: DEBUG nova.compute.manager [req-ab1f8883-9350-47f4-98e7-52e182f8f269 req-fcab40e4-435c-4db2-a5c3-f77d3b919c24 service nova] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Received event network-vif-plugged-a309cd5b-1679-48c6-a596-168b3af2d284 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:17:32 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ab1f8883-9350-47f4-98e7-52e182f8f269 req-fcab40e4-435c-4db2-a5c3-f77d3b919c24 service nova] Acquiring lock "11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:17:32 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ab1f8883-9350-47f4-98e7-52e182f8f269 req-fcab40e4-435c-4db2-a5c3-f77d3b919c24 service nova] Lock "11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:17:32 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ab1f8883-9350-47f4-98e7-52e182f8f269 req-fcab40e4-435c-4db2-a5c3-f77d3b919c24 service nova] Lock "11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:17:32 user nova-compute[72351]: DEBUG nova.compute.manager [req-ab1f8883-9350-47f4-98e7-52e182f8f269 req-fcab40e4-435c-4db2-a5c3-f77d3b919c24 service nova] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] No waiting events found dispatching network-vif-plugged-a309cd5b-1679-48c6-a596-168b3af2d284 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:17:32 user nova-compute[72351]: WARNING nova.compute.manager [req-ab1f8883-9350-47f4-98e7-52e182f8f269 req-fcab40e4-435c-4db2-a5c3-f77d3b919c24 service nova] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Received unexpected event network-vif-plugged-a309cd5b-1679-48c6-a596-168b3af2d284 for instance with vm_state building and task_state spawning.
Aug 09 02:17:33 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:33 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:33 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:33 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:33 user nova-compute[72351]: DEBUG nova.compute.manager [req-7cf2cb19-0080-4ca9-abff-bb83151a92ea req-9b04c10f-f8c7-4588-976b-a694dde4755c service nova] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Received event network-vif-plugged-3718df01-0e23-424c-8ab1-bf0fd9c7e121 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:17:33 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-7cf2cb19-0080-4ca9-abff-bb83151a92ea req-9b04c10f-f8c7-4588-976b-a694dde4755c service nova] Acquiring lock "74728ca8-34d4-45e6-a1f2-21c0952ff835-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:17:33 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-7cf2cb19-0080-4ca9-abff-bb83151a92ea req-9b04c10f-f8c7-4588-976b-a694dde4755c service nova] Lock "74728ca8-34d4-45e6-a1f2-21c0952ff835-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:17:33 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-7cf2cb19-0080-4ca9-abff-bb83151a92ea req-9b04c10f-f8c7-4588-976b-a694dde4755c service nova] Lock "74728ca8-34d4-45e6-a1f2-21c0952ff835-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:17:33 user nova-compute[72351]: DEBUG nova.compute.manager [req-7cf2cb19-0080-4ca9-abff-bb83151a92ea req-9b04c10f-f8c7-4588-976b-a694dde4755c service nova] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] No waiting events found dispatching network-vif-plugged-3718df01-0e23-424c-8ab1-bf0fd9c7e121 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:17:33 user nova-compute[72351]: WARNING nova.compute.manager [req-7cf2cb19-0080-4ca9-abff-bb83151a92ea req-9b04c10f-f8c7-4588-976b-a694dde4755c service nova] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Received unexpected event network-vif-plugged-3718df01-0e23-424c-8ab1-bf0fd9c7e121 for instance with vm_state building and task_state spawning.
Aug 09 02:17:33 user nova-compute[72351]: DEBUG nova.compute.manager [req-7cf2cb19-0080-4ca9-abff-bb83151a92ea req-9b04c10f-f8c7-4588-976b-a694dde4755c service nova] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Received event network-vif-plugged-7e7b4229-7c23-4a73-9294-4c4fc9f6a01a {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:17:33 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-7cf2cb19-0080-4ca9-abff-bb83151a92ea req-9b04c10f-f8c7-4588-976b-a694dde4755c service nova] Acquiring lock "7c5e633d-a32f-4c5e-bb9d-1e40da964bd0-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:17:33 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-7cf2cb19-0080-4ca9-abff-bb83151a92ea req-9b04c10f-f8c7-4588-976b-a694dde4755c service nova] Lock "7c5e633d-a32f-4c5e-bb9d-1e40da964bd0-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:17:33 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-7cf2cb19-0080-4ca9-abff-bb83151a92ea req-9b04c10f-f8c7-4588-976b-a694dde4755c service nova] Lock "7c5e633d-a32f-4c5e-bb9d-1e40da964bd0-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:17:33 user nova-compute[72351]: DEBUG nova.compute.manager [req-7cf2cb19-0080-4ca9-abff-bb83151a92ea req-9b04c10f-f8c7-4588-976b-a694dde4755c service nova] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] No waiting events found dispatching network-vif-plugged-7e7b4229-7c23-4a73-9294-4c4fc9f6a01a {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:17:33 user nova-compute[72351]: WARNING nova.compute.manager [req-7cf2cb19-0080-4ca9-abff-bb83151a92ea req-9b04c10f-f8c7-4588-976b-a694dde4755c service nova] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Received unexpected event network-vif-plugged-7e7b4229-7c23-4a73-9294-4c4fc9f6a01a for instance with vm_state building and task_state spawning.
Aug 09 02:17:33 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:34 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:34 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:34 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:34 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:34 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:34 user nova-compute[72351]: DEBUG nova.compute.manager [req-15a2acf1-64b1-48d5-8057-77df491930da req-22c1659a-7b0e-4634-b072-a6f0e8ae9e72 service nova] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Received event network-vif-plugged-a309cd5b-1679-48c6-a596-168b3af2d284 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:17:34 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-15a2acf1-64b1-48d5-8057-77df491930da req-22c1659a-7b0e-4634-b072-a6f0e8ae9e72 service nova] Acquiring lock "11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:17:34 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-15a2acf1-64b1-48d5-8057-77df491930da req-22c1659a-7b0e-4634-b072-a6f0e8ae9e72 service nova] Lock "11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:17:34 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-15a2acf1-64b1-48d5-8057-77df491930da req-22c1659a-7b0e-4634-b072-a6f0e8ae9e72 service nova] Lock "11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:17:34 user nova-compute[72351]: DEBUG nova.compute.manager [req-15a2acf1-64b1-48d5-8057-77df491930da req-22c1659a-7b0e-4634-b072-a6f0e8ae9e72 service nova] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] No waiting events found dispatching network-vif-plugged-a309cd5b-1679-48c6-a596-168b3af2d284 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:17:34 user nova-compute[72351]: WARNING nova.compute.manager [req-15a2acf1-64b1-48d5-8057-77df491930da req-22c1659a-7b0e-4634-b072-a6f0e8ae9e72 service nova] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Received unexpected event network-vif-plugged-a309cd5b-1679-48c6-a596-168b3af2d284 for instance with vm_state building and task_state spawning.
Aug 09 02:17:35 user nova-compute[72351]: DEBUG nova.compute.manager [req-15f9ea87-7562-4d81-a9c5-14df8e979de7 req-ab4dc994-60c7-4cc9-ab05-98c18cffd507 service nova] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Received event network-vif-plugged-7e7b4229-7c23-4a73-9294-4c4fc9f6a01a {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:17:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-15f9ea87-7562-4d81-a9c5-14df8e979de7 req-ab4dc994-60c7-4cc9-ab05-98c18cffd507 service nova] Acquiring lock "7c5e633d-a32f-4c5e-bb9d-1e40da964bd0-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:17:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-15f9ea87-7562-4d81-a9c5-14df8e979de7 req-ab4dc994-60c7-4cc9-ab05-98c18cffd507 service nova] Lock "7c5e633d-a32f-4c5e-bb9d-1e40da964bd0-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:17:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-15f9ea87-7562-4d81-a9c5-14df8e979de7 req-ab4dc994-60c7-4cc9-ab05-98c18cffd507 service nova] Lock "7c5e633d-a32f-4c5e-bb9d-1e40da964bd0-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:17:35 user nova-compute[72351]: DEBUG nova.compute.manager [req-15f9ea87-7562-4d81-a9c5-14df8e979de7 req-ab4dc994-60c7-4cc9-ab05-98c18cffd507 service nova] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] No waiting events found dispatching network-vif-plugged-7e7b4229-7c23-4a73-9294-4c4fc9f6a01a {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:17:35 user nova-compute[72351]: WARNING nova.compute.manager [req-15f9ea87-7562-4d81-a9c5-14df8e979de7 req-ab4dc994-60c7-4cc9-ab05-98c18cffd507 service nova] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Received unexpected event network-vif-plugged-7e7b4229-7c23-4a73-9294-4c4fc9f6a01a for instance with vm_state building and task_state spawning.
Aug 09 02:17:35 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547455.7895112, 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:17:35 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] VM Resumed (Lifecycle Event)
Aug 09 02:17:35 user nova-compute[72351]: DEBUG nova.compute.manager [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:17:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Guest created on hypervisor {{(pid=72351) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4411}}
Aug 09 02:17:35 user nova-compute[72351]: DEBUG nova.compute.manager [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:17:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Guest created on hypervisor {{(pid=72351) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4411}}
Aug 09 02:17:35 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Instance spawned successfully.
Aug 09 02:17:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:894}}
Aug 09 02:17:35 user nova-compute[72351]: DEBUG nova.compute.manager [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:17:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Guest created on hypervisor {{(pid=72351) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4411}}
Aug 09 02:17:35 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:17:35 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Instance spawned successfully.
Aug 09 02:17:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:894}}
Aug 09 02:17:35 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Instance spawned successfully.
Aug 09 02:17:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:894}}
Aug 09 02:17:35 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:17:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Found default for hw_cdrom_bus of ide {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:17:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Found default for hw_disk_bus of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:17:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Found default for hw_input_bus of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:17:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Found default for hw_pointer_model of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:17:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Found default for hw_video_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:17:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Found default for hw_vif_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:17:35 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:17:35 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547455.7896936, 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:17:35 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] VM Started (Lifecycle Event)
Aug 09 02:17:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Found default for hw_cdrom_bus of ide {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:17:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Found default for hw_disk_bus of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:17:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Found default for hw_input_bus of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:17:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Found default for hw_pointer_model of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:17:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Found default for hw_video_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:17:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Found default for hw_vif_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:17:35 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:17:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Found default for hw_cdrom_bus of ide {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:17:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Found default for hw_disk_bus of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:17:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Found default for hw_input_bus of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:17:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Found default for hw_pointer_model of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:17:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Found default for hw_video_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:17:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Found default for hw_vif_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:17:35 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:17:35 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:17:35 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547455.7985525, 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0 => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:17:35 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] VM Resumed (Lifecycle Event)
Aug 09 02:17:35 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:17:35 user nova-compute[72351]: DEBUG nova.compute.manager [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:17:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Guest created on hypervisor {{(pid=72351) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4411}}
Aug 09 02:17:35 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:17:36 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Instance spawned successfully.
Aug 09 02:17:36 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Attempting to register defaults for the following image properties: ['hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:894}}
Aug 09 02:17:36 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Found default for hw_input_bus of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:17:36 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Found default for hw_pointer_model of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:17:36 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Found default for hw_video_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:17:36 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Found default for hw_vif_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:17:36 user nova-compute[72351]: INFO nova.compute.manager [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Took 14.86 seconds to spawn the instance on the hypervisor.
Aug 09 02:17:36 user nova-compute[72351]: DEBUG nova.compute.manager [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:17:36 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:17:36 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547455.8004363, 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0 => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:17:36 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] VM Started (Lifecycle Event)
Aug 09 02:17:36 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:17:36 user nova-compute[72351]: INFO nova.compute.manager [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Took 13.14 seconds to spawn the instance on the hypervisor.
Aug 09 02:17:36 user nova-compute[72351]: DEBUG nova.compute.manager [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:17:36 user nova-compute[72351]: INFO nova.compute.manager [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Took 14.36 seconds to spawn the instance on the hypervisor.
Aug 09 02:17:36 user nova-compute[72351]: DEBUG nova.compute.manager [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:17:36 user nova-compute[72351]: INFO nova.compute.manager [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Took 11.73 seconds to spawn the instance on the hypervisor.
Aug 09 02:17:36 user nova-compute[72351]: DEBUG nova.compute.manager [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:17:36 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:17:36 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:17:36 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547455.8163507, 74728ca8-34d4-45e6-a1f2-21c0952ff835 => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:17:36 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] VM Resumed (Lifecycle Event)
Aug 09 02:17:36 user nova-compute[72351]: INFO nova.compute.manager [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Took 16.09 seconds to build instance.
Aug 09 02:17:36 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:17:36 user nova-compute[72351]: INFO nova.compute.manager [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Took 13.39 seconds to build instance.
Aug 09 02:17:36 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: active, current task_state: None, current DB power_state: 1, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:17:36 user nova-compute[72351]: INFO nova.compute.manager [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Took 16.05 seconds to build instance.
Aug 09 02:17:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-d1301160-dc1f-4437-a191-1275813da1fc tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Lock "74728ca8-34d4-45e6-a1f2-21c0952ff835" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 16.287s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:17:36 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547455.8235724, 74728ca8-34d4-45e6-a1f2-21c0952ff835 => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:17:36 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] VM Started (Lifecycle Event)
Aug 09 02:17:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c24e9868-dfb9-45b3-a9c7-17c7303aaa8c tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Lock "50302d76-f6b9-4fe3-b55a-4a01e483e2e3" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 16.228s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:17:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f809e8bd-e907-4ab1-9e6a-89a96aa9de06 tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Lock "7c5e633d-a32f-4c5e-bb9d-1e40da964bd0" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 13.593s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:17:36 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:17:36 user nova-compute[72351]: INFO nova.compute.manager [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Took 15.68 seconds to build instance.
Aug 09 02:17:36 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Synchronizing instance power state after lifecycle event "Started"; current vm_state: active, current task_state: None, current DB power_state: 1, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:17:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-42197e07-03b5-4bdc-8e59-7ffd12d8c67b tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Lock "11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 15.810s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:17:36 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547455.994753, 50302d76-f6b9-4fe3-b55a-4a01e483e2e3 => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:17:36 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] VM Resumed (Lifecycle Event)
Aug 09 02:17:36 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:17:36 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: active, current task_state: None, current DB power_state: 1, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:17:36 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547455.9949315, 50302d76-f6b9-4fe3-b55a-4a01e483e2e3 => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:17:36 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] VM Started (Lifecycle Event)
Aug 09 02:17:36 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:17:36 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Synchronizing instance power state after lifecycle event "Started"; current vm_state: active, current task_state: None, current DB power_state: 1, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:17:37 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:38 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:42 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:43 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Acquiring lock "d0c7002c-8a3a-4bc5-940c-63cf4264668b" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:17:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Lock "d0c7002c-8a3a-4bc5-940c-63cf4264668b" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:17:44 user nova-compute[72351]: DEBUG nova.compute.manager [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Starting instance... {{(pid=72351) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2406}}
Aug 09 02:17:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:17:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:17:44 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72351) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Aug 09 02:17:44 user nova-compute[72351]: INFO nova.compute.claims [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Claim successful on node user
Aug 09 02:17:45 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:17:45 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:17:45 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.818s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:17:45 user nova-compute[72351]: DEBUG nova.compute.manager [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Start building networks asynchronously for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2803}}
Aug 09 02:17:45 user nova-compute[72351]: DEBUG nova.compute.manager [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Allocating IP information in the background. {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1956}}
Aug 09 02:17:45 user nova-compute[72351]: DEBUG nova.network.neutron [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] allocate_for_instance() {{(pid=72351) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Aug 09 02:17:45 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Aug 09 02:17:45 user nova-compute[72351]: DEBUG nova.compute.manager [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Start building block device mappings for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2838}}
Aug 09 02:17:45 user nova-compute[72351]: DEBUG nova.policy [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '00a45ab3cb2d43028418af1ae209da51', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5209610aaaa34f7a8ffb7757dbfac85d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72351) authorize /opt/stack/nova/nova/policy.py:203}}
Aug 09 02:17:45 user nova-compute[72351]: DEBUG nova.compute.manager [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Start spawning the instance on the hypervisor. {{(pid=72351) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2612}}
Aug 09 02:17:45 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Creating instance directory {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4717}}
Aug 09 02:17:45 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Creating image(s)
Aug 09 02:17:45 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Acquiring lock "/opt/stack/data/nova/instances/d0c7002c-8a3a-4bc5-940c-63cf4264668b/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:17:45 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Lock "/opt/stack/data/nova/instances/d0c7002c-8a3a-4bc5-940c-63cf4264668b/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:17:45 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Lock "/opt/stack/data/nova/instances/d0c7002c-8a3a-4bc5-940c-63cf4264668b/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:17:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:17:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.161s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:17:45 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Acquiring lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:17:45 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:17:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:17:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.162s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:17:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/d0c7002c-8a3a-4bc5-940c-63cf4264668b/disk 1073741824 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:17:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/d0c7002c-8a3a-4bc5-940c-63cf4264668b/disk 1073741824" returned: 0 in 0.075s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:17:45 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.248s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:17:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:17:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.173s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:17:46 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Checking if we can resize image /opt/stack/data/nova/instances/d0c7002c-8a3a-4bc5-940c-63cf4264668b/disk. size=1073741824 {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Aug 09 02:17:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d0c7002c-8a3a-4bc5-940c-63cf4264668b/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:17:46 user nova-compute[72351]: DEBUG nova.network.neutron [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Successfully created port: 9734c099-ce77-4e2d-ab87-6ae4ab56c56b {{(pid=72351) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Aug 09 02:17:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d0c7002c-8a3a-4bc5-940c-63cf4264668b/disk --force-share --output=json" returned: 0 in 0.171s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:17:46 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Cannot resize image /opt/stack/data/nova/instances/d0c7002c-8a3a-4bc5-940c-63cf4264668b/disk to a smaller size. {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Aug 09 02:17:46 user nova-compute[72351]: DEBUG nova.objects.instance [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Lazy-loading 'migration_context' on Instance uuid d0c7002c-8a3a-4bc5-940c-63cf4264668b {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:17:46 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Created local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4851}}
Aug 09 02:17:46 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Ensure instance console log exists: /opt/stack/data/nova/instances/d0c7002c-8a3a-4bc5-940c-63cf4264668b/console.log {{(pid=72351) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4603}}
Aug 09 02:17:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:17:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:17:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:17:46 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._run_pending_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:17:46 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Cleaning up deleted instances {{(pid=72351) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11129}}
Aug 09 02:17:46 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] There are 0 instances to clean {{(pid=72351) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11138}}
Aug 09 02:17:46 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._cleanup_incomplete_migrations {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:17:46 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Cleaning up deleted instances with incomplete migration  {{(pid=72351) _cleanup_incomplete_migrations /opt/stack/nova/nova/compute/manager.py:11167}}
Aug 09 02:17:46 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._cleanup_expired_console_auth_tokens {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:17:47 user nova-compute[72351]: DEBUG nova.network.neutron [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Successfully updated port: 9734c099-ce77-4e2d-ab87-6ae4ab56c56b {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:17:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Acquiring lock "refresh_cache-d0c7002c-8a3a-4bc5-940c-63cf4264668b" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:17:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Acquired lock "refresh_cache-d0c7002c-8a3a-4bc5-940c-63cf4264668b" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:17:47 user nova-compute[72351]: DEBUG nova.network.neutron [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:17:47 user nova-compute[72351]: DEBUG nova.compute.manager [req-64d82c68-79bb-41d1-8f53-6beb891530a6 req-70b56661-0e1e-45d7-8138-48e1114554af service nova] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Received event network-changed-9734c099-ce77-4e2d-ab87-6ae4ab56c56b {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:17:47 user nova-compute[72351]: DEBUG nova.compute.manager [req-64d82c68-79bb-41d1-8f53-6beb891530a6 req-70b56661-0e1e-45d7-8138-48e1114554af service nova] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Refreshing instance network info cache due to event network-changed-9734c099-ce77-4e2d-ab87-6ae4ab56c56b. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:17:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-64d82c68-79bb-41d1-8f53-6beb891530a6 req-70b56661-0e1e-45d7-8138-48e1114554af service nova] Acquiring lock "refresh_cache-d0c7002c-8a3a-4bc5-940c-63cf4264668b" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:17:47 user nova-compute[72351]: DEBUG nova.network.neutron [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:17:47 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:47 user nova-compute[72351]: DEBUG nova.network.neutron [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Updating instance_info_cache with network_info: [{"id": "9734c099-ce77-4e2d-ab87-6ae4ab56c56b", "address": "fa:16:3e:6b:a4:c9", "network": {"id": "fc48221d-cf2e-4871-9741-c102904e10a6", "bridge": "br-int", "label": "tempest-VolumesActionsTest-857807030-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5209610aaaa34f7a8ffb7757dbfac85d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9734c099-ce", "ovs_interfaceid": "9734c099-ce77-4e2d-ab87-6ae4ab56c56b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:17:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Releasing lock "refresh_cache-d0c7002c-8a3a-4bc5-940c-63cf4264668b" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:17:47 user nova-compute[72351]: DEBUG nova.compute.manager [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Instance network_info: |[{"id": "9734c099-ce77-4e2d-ab87-6ae4ab56c56b", "address": "fa:16:3e:6b:a4:c9", "network": {"id": "fc48221d-cf2e-4871-9741-c102904e10a6", "bridge": "br-int", "label": "tempest-VolumesActionsTest-857807030-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5209610aaaa34f7a8ffb7757dbfac85d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9734c099-ce", "ovs_interfaceid": "9734c099-ce77-4e2d-ab87-6ae4ab56c56b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1971}}
Aug 09 02:17:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-64d82c68-79bb-41d1-8f53-6beb891530a6 req-70b56661-0e1e-45d7-8138-48e1114554af service nova] Acquired lock "refresh_cache-d0c7002c-8a3a-4bc5-940c-63cf4264668b" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:17:47 user nova-compute[72351]: DEBUG nova.network.neutron [req-64d82c68-79bb-41d1-8f53-6beb891530a6 req-70b56661-0e1e-45d7-8138-48e1114554af service nova] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Refreshing network info cache for port 9734c099-ce77-4e2d-ab87-6ae4ab56c56b {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:17:47 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Start _get_guest_xml network_info=[{"id": "9734c099-ce77-4e2d-ab87-6ae4ab56c56b", "address": "fa:16:3e:6b:a4:c9", "network": {"id": "fc48221d-cf2e-4871-9741-c102904e10a6", "bridge": "br-int", "label": "tempest-VolumesActionsTest-857807030-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5209610aaaa34f7a8ffb7757dbfac85d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9734c099-ce", "ovs_interfaceid": "9734c099-ce77-4e2d-ab87-6ae4ab56c56b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'size': 0, 'device_type': 'disk', 'encryption_secret_uuid': None, 'guest_format': None, 'encryption_options': None, 'disk_bus': 'virtio', 'encrypted': False, 'encryption_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'image_id': '18ccda7f-31d3-4978-ab9d-4ed2f74eff94'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:17:47 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:17:47 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:17:47 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:17:47 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:17:47 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:17:47 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:17:47 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:17:47 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:17:47 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:17:47 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:17:47 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:17:47 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:17:47 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:17:47 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:17:47 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:17:47 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:17:47 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:17:47 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:17:47 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:17:43Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesActionsTest-instance-2017545637',display_name='tempest-VolumesActionsTest-instance-2017545637',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-volumesactionstest-instance-2017545637',id=7,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='5209610aaaa34f7a8ffb7757dbfac85d',ramdisk_id='',reservation_id='r-huk5pbgj',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesActionsTest-1343359891',owner_user_name='tempest-VolumesActionsTest-1343359891-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:17:45Z,user_data=None,user_id='00a45ab3cb2d43028418af1ae209da51',uuid=d0c7002c-8a3a-4bc5-940c-63cf4264668b,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "9734c099-ce77-4e2d-ab87-6ae4ab56c56b", "address": "fa:16:3e:6b:a4:c9", "network": {"id": "fc48221d-cf2e-4871-9741-c102904e10a6", "bridge": "br-int", "label": "tempest-VolumesActionsTest-857807030-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5209610aaaa34f7a8ffb7757dbfac85d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9734c099-ce", "ovs_interfaceid": "9734c099-ce77-4e2d-ab87-6ae4ab56c56b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:17:47 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Converting VIF {"id": "9734c099-ce77-4e2d-ab87-6ae4ab56c56b", "address": "fa:16:3e:6b:a4:c9", "network": {"id": "fc48221d-cf2e-4871-9741-c102904e10a6", "bridge": "br-int", "label": "tempest-VolumesActionsTest-857807030-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5209610aaaa34f7a8ffb7757dbfac85d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9734c099-ce", "ovs_interfaceid": "9734c099-ce77-4e2d-ab87-6ae4ab56c56b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:17:47 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:6b:a4:c9,bridge_name='br-int',has_traffic_filtering=True,id=9734c099-ce77-4e2d-ab87-6ae4ab56c56b,network=Network(fc48221d-cf2e-4871-9741-c102904e10a6),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap9734c099-ce') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:17:47 user nova-compute[72351]: DEBUG nova.objects.instance [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Lazy-loading 'pci_devices' on Instance uuid d0c7002c-8a3a-4bc5-940c-63cf4264668b {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:17:47 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:17:47 user nova-compute[72351]:   <uuid>d0c7002c-8a3a-4bc5-940c-63cf4264668b</uuid>
Aug 09 02:17:47 user nova-compute[72351]:   <name>instance-00000007</name>
Aug 09 02:17:47 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:17:47 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:17:47 user nova-compute[72351]:   <metadata>
Aug 09 02:17:47 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:17:47 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:17:47 user nova-compute[72351]:       <nova:name>tempest-VolumesActionsTest-instance-2017545637</nova:name>
Aug 09 02:17:47 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:17:47</nova:creationTime>
Aug 09 02:17:47 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:17:47 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:17:47 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:17:47 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:17:47 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:17:47 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:17:47 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:17:47 user nova-compute[72351]:       <nova:owner>
Aug 09 02:17:47 user nova-compute[72351]:         <nova:user uuid="00a45ab3cb2d43028418af1ae209da51">tempest-VolumesActionsTest-1343359891-project-member</nova:user>
Aug 09 02:17:47 user nova-compute[72351]:         <nova:project uuid="5209610aaaa34f7a8ffb7757dbfac85d">tempest-VolumesActionsTest-1343359891</nova:project>
Aug 09 02:17:47 user nova-compute[72351]:       </nova:owner>
Aug 09 02:17:47 user nova-compute[72351]:       <nova:root type="image" uuid="18ccda7f-31d3-4978-ab9d-4ed2f74eff94"/>
Aug 09 02:17:47 user nova-compute[72351]:       <nova:ports>
Aug 09 02:17:47 user nova-compute[72351]:         <nova:port uuid="9734c099-ce77-4e2d-ab87-6ae4ab56c56b">
Aug 09 02:17:47 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.6" ipVersion="4"/>
Aug 09 02:17:47 user nova-compute[72351]:         </nova:port>
Aug 09 02:17:47 user nova-compute[72351]:       </nova:ports>
Aug 09 02:17:47 user nova-compute[72351]:     </nova:instance>
Aug 09 02:17:47 user nova-compute[72351]:   </metadata>
Aug 09 02:17:47 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:17:47 user nova-compute[72351]:     <system>
Aug 09 02:17:47 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:17:47 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:17:47 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:17:47 user nova-compute[72351]:       <entry name="serial">d0c7002c-8a3a-4bc5-940c-63cf4264668b</entry>
Aug 09 02:17:47 user nova-compute[72351]:       <entry name="uuid">d0c7002c-8a3a-4bc5-940c-63cf4264668b</entry>
Aug 09 02:17:47 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:17:47 user nova-compute[72351]:     </system>
Aug 09 02:17:47 user nova-compute[72351]:   </sysinfo>
Aug 09 02:17:47 user nova-compute[72351]:   <os>
Aug 09 02:17:47 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:17:47 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:17:47 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:17:47 user nova-compute[72351]:   </os>
Aug 09 02:17:47 user nova-compute[72351]:   <features>
Aug 09 02:17:47 user nova-compute[72351]:     <acpi/>
Aug 09 02:17:47 user nova-compute[72351]:     <apic/>
Aug 09 02:17:47 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:17:47 user nova-compute[72351]:   </features>
Aug 09 02:17:47 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:17:47 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:17:47 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:17:47 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:17:47 user nova-compute[72351]:   </clock>
Aug 09 02:17:47 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:17:47 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:17:47 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:17:47 user nova-compute[72351]:   </cpu>
Aug 09 02:17:47 user nova-compute[72351]:   <devices>
Aug 09 02:17:47 user nova-compute[72351]:     <disk type="file" device="disk">
Aug 09 02:17:47 user nova-compute[72351]:       <driver name="qemu" type="qcow2" cache="none"/>
Aug 09 02:17:47 user nova-compute[72351]:       <source file="/opt/stack/data/nova/instances/d0c7002c-8a3a-4bc5-940c-63cf4264668b/disk"/>
Aug 09 02:17:47 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:17:47 user nova-compute[72351]:     </disk>
Aug 09 02:17:47 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:17:47 user nova-compute[72351]:       <mac address="fa:16:3e:6b:a4:c9"/>
Aug 09 02:17:47 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:17:47 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:17:47 user nova-compute[72351]:       <target dev="tap9734c099-ce"/>
Aug 09 02:17:47 user nova-compute[72351]:     </interface>
Aug 09 02:17:47 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:17:47 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/d0c7002c-8a3a-4bc5-940c-63cf4264668b/console.log" append="off"/>
Aug 09 02:17:47 user nova-compute[72351]:     </serial>
Aug 09 02:17:47 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:17:47 user nova-compute[72351]:     <video>
Aug 09 02:17:47 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:17:47 user nova-compute[72351]:     </video>
Aug 09 02:17:47 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:17:47 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:17:47 user nova-compute[72351]:     </rng>
Aug 09 02:17:47 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:17:47 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:17:47 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:17:47 user nova-compute[72351]:     </memballoon>
Aug 09 02:17:47 user nova-compute[72351]:   </devices>
Aug 09 02:17:47 user nova-compute[72351]: </domain>
Aug 09 02:17:47 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:17:47 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:17:43Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesActionsTest-instance-2017545637',display_name='tempest-VolumesActionsTest-instance-2017545637',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-volumesactionstest-instance-2017545637',id=7,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='5209610aaaa34f7a8ffb7757dbfac85d',ramdisk_id='',reservation_id='r-huk5pbgj',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesActionsTest-1343359891',owner_user_name='tempest-VolumesActionsTest-1343359891-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:17:45Z,user_data=None,user_id='00a45ab3cb2d43028418af1ae209da51',uuid=d0c7002c-8a3a-4bc5-940c-63cf4264668b,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "9734c099-ce77-4e2d-ab87-6ae4ab56c56b", "address": "fa:16:3e:6b:a4:c9", "network": {"id": "fc48221d-cf2e-4871-9741-c102904e10a6", "bridge": "br-int", "label": "tempest-VolumesActionsTest-857807030-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5209610aaaa34f7a8ffb7757dbfac85d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9734c099-ce", "ovs_interfaceid": "9734c099-ce77-4e2d-ab87-6ae4ab56c56b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:17:47 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Converting VIF {"id": "9734c099-ce77-4e2d-ab87-6ae4ab56c56b", "address": "fa:16:3e:6b:a4:c9", "network": {"id": "fc48221d-cf2e-4871-9741-c102904e10a6", "bridge": "br-int", "label": "tempest-VolumesActionsTest-857807030-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5209610aaaa34f7a8ffb7757dbfac85d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9734c099-ce", "ovs_interfaceid": "9734c099-ce77-4e2d-ab87-6ae4ab56c56b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:17:47 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:6b:a4:c9,bridge_name='br-int',has_traffic_filtering=True,id=9734c099-ce77-4e2d-ab87-6ae4ab56c56b,network=Network(fc48221d-cf2e-4871-9741-c102904e10a6),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap9734c099-ce') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:17:47 user nova-compute[72351]: DEBUG os_vif [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:6b:a4:c9,bridge_name='br-int',has_traffic_filtering=True,id=9734c099-ce77-4e2d-ab87-6ae4ab56c56b,network=Network(fc48221d-cf2e-4871-9741-c102904e10a6),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap9734c099-ce') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:17:47 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:47 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:17:47 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:17:47 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:47 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap9734c099-ce, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:17:47 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap9734c099-ce, col_values=(('external_ids', {'iface-id': '9734c099-ce77-4e2d-ab87-6ae4ab56c56b', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:6b:a4:c9', 'vm-uuid': 'd0c7002c-8a3a-4bc5-940c-63cf4264668b'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:17:47 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:47 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:17:47 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:47 user nova-compute[72351]: INFO os_vif [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:6b:a4:c9,bridge_name='br-int',has_traffic_filtering=True,id=9734c099-ce77-4e2d-ab87-6ae4ab56c56b,network=Network(fc48221d-cf2e-4871-9741-c102904e10a6),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap9734c099-ce')
Aug 09 02:17:48 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:17:48 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] No VIF found with MAC fa:16:3e:6b:a4:c9, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:17:48 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:48 user nova-compute[72351]: DEBUG nova.network.neutron [req-64d82c68-79bb-41d1-8f53-6beb891530a6 req-70b56661-0e1e-45d7-8138-48e1114554af service nova] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Updated VIF entry in instance network info cache for port 9734c099-ce77-4e2d-ab87-6ae4ab56c56b. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:17:48 user nova-compute[72351]: DEBUG nova.network.neutron [req-64d82c68-79bb-41d1-8f53-6beb891530a6 req-70b56661-0e1e-45d7-8138-48e1114554af service nova] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Updating instance_info_cache with network_info: [{"id": "9734c099-ce77-4e2d-ab87-6ae4ab56c56b", "address": "fa:16:3e:6b:a4:c9", "network": {"id": "fc48221d-cf2e-4871-9741-c102904e10a6", "bridge": "br-int", "label": "tempest-VolumesActionsTest-857807030-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5209610aaaa34f7a8ffb7757dbfac85d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9734c099-ce", "ovs_interfaceid": "9734c099-ce77-4e2d-ab87-6ae4ab56c56b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:17:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-64d82c68-79bb-41d1-8f53-6beb891530a6 req-70b56661-0e1e-45d7-8138-48e1114554af service nova] Releasing lock "refresh_cache-d0c7002c-8a3a-4bc5-940c-63cf4264668b" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:17:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:49 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:17:49 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:17:50 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:17:50 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:17:50 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:17:50 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Auditing locally available compute resources for user (node: user) {{(pid=72351) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Aug 09 02:17:50 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:17:50 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2/disk --force-share --output=json" returned: 0 in 0.191s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:17:50 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:17:50 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:50 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:50 user nova-compute[72351]: DEBUG nova.compute.manager [req-c8a80aee-3d17-4214-a95f-872d612ce166 req-d16563d9-f29c-4287-899b-a91a53e61298 service nova] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Received event network-vif-plugged-9734c099-ce77-4e2d-ab87-6ae4ab56c56b {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:17:50 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c8a80aee-3d17-4214-a95f-872d612ce166 req-d16563d9-f29c-4287-899b-a91a53e61298 service nova] Acquiring lock "d0c7002c-8a3a-4bc5-940c-63cf4264668b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:17:50 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c8a80aee-3d17-4214-a95f-872d612ce166 req-d16563d9-f29c-4287-899b-a91a53e61298 service nova] Lock "d0c7002c-8a3a-4bc5-940c-63cf4264668b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:17:50 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c8a80aee-3d17-4214-a95f-872d612ce166 req-d16563d9-f29c-4287-899b-a91a53e61298 service nova] Lock "d0c7002c-8a3a-4bc5-940c-63cf4264668b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:17:50 user nova-compute[72351]: DEBUG nova.compute.manager [req-c8a80aee-3d17-4214-a95f-872d612ce166 req-d16563d9-f29c-4287-899b-a91a53e61298 service nova] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] No waiting events found dispatching network-vif-plugged-9734c099-ce77-4e2d-ab87-6ae4ab56c56b {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:17:50 user nova-compute[72351]: WARNING nova.compute.manager [req-c8a80aee-3d17-4214-a95f-872d612ce166 req-d16563d9-f29c-4287-899b-a91a53e61298 service nova] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Received unexpected event network-vif-plugged-9734c099-ce77-4e2d-ab87-6ae4ab56c56b for instance with vm_state building and task_state spawning.
Aug 09 02:17:50 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2/disk --force-share --output=json" returned: 0 in 0.209s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:17:50 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f45b57fa-9264-45f0-9573-e5fb58848d1f/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:17:51 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f45b57fa-9264-45f0-9573-e5fb58848d1f/disk --force-share --output=json" returned: 0 in 0.221s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:17:51 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f45b57fa-9264-45f0-9573-e5fb58848d1f/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:17:51 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f45b57fa-9264-45f0-9573-e5fb58848d1f/disk --force-share --output=json" returned: 0 in 0.262s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:17:51 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d0c7002c-8a3a-4bc5-940c-63cf4264668b/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:17:51 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d0c7002c-8a3a-4bc5-940c-63cf4264668b/disk --force-share --output=json" returned: 0 in 0.180s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:17:51 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d0c7002c-8a3a-4bc5-940c-63cf4264668b/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:17:51 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d0c7002c-8a3a-4bc5-940c-63cf4264668b/disk --force-share --output=json" returned: 0 in 0.188s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:17:51 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:17:51 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b/disk --force-share --output=json" returned: 0 in 0.145s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:17:51 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:17:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b/disk --force-share --output=json" returned: 0 in 0.172s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:17:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/74728ca8-34d4-45e6-a1f2-21c0952ff835/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:17:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/74728ca8-34d4-45e6-a1f2-21c0952ff835/disk --force-share --output=json" returned: 0 in 0.160s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:17:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/74728ca8-34d4-45e6-a1f2-21c0952ff835/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:17:52 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/74728ca8-34d4-45e6-a1f2-21c0952ff835/disk --force-share --output=json" returned: 0 in 0.170s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:17:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/50302d76-f6b9-4fe3-b55a-4a01e483e2e3/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:17:52 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547472.567448, d0c7002c-8a3a-4bc5-940c-63cf4264668b => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:17:52 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] VM Resumed (Lifecycle Event)
Aug 09 02:17:52 user nova-compute[72351]: DEBUG nova.compute.manager [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:17:52 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Guest created on hypervisor {{(pid=72351) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4411}}
Aug 09 02:17:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/50302d76-f6b9-4fe3-b55a-4a01e483e2e3/disk --force-share --output=json" returned: 0 in 0.168s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:17:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/50302d76-f6b9-4fe3-b55a-4a01e483e2e3/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:17:52 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:17:52 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Instance spawned successfully.
Aug 09 02:17:52 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:894}}
Aug 09 02:17:52 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:17:52 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:52 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Found default for hw_cdrom_bus of ide {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:17:52 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Found default for hw_disk_bus of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:17:52 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Found default for hw_input_bus of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:17:52 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Found default for hw_pointer_model of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:17:52 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Found default for hw_video_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:17:52 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Found default for hw_vif_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:17:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/50302d76-f6b9-4fe3-b55a-4a01e483e2e3/disk --force-share --output=json" returned: 0 in 0.148s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:17:52 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:17:52 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547472.56816, d0c7002c-8a3a-4bc5-940c-63cf4264668b => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:17:52 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] VM Started (Lifecycle Event)
Aug 09 02:17:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7c5e633d-a32f-4c5e-bb9d-1e40da964bd0/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:17:52 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:17:52 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:17:52 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:17:52 user nova-compute[72351]: INFO nova.compute.manager [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Took 7.37 seconds to spawn the instance on the hypervisor.
Aug 09 02:17:52 user nova-compute[72351]: DEBUG nova.compute.manager [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:17:52 user nova-compute[72351]: DEBUG nova.compute.manager [req-25072b7c-3681-438c-9bf0-b481823f2ed4 req-06e7cc15-5d1b-42c8-9236-80bfd0a075eb service nova] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Received event network-vif-plugged-9734c099-ce77-4e2d-ab87-6ae4ab56c56b {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:17:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-25072b7c-3681-438c-9bf0-b481823f2ed4 req-06e7cc15-5d1b-42c8-9236-80bfd0a075eb service nova] Acquiring lock "d0c7002c-8a3a-4bc5-940c-63cf4264668b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:17:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-25072b7c-3681-438c-9bf0-b481823f2ed4 req-06e7cc15-5d1b-42c8-9236-80bfd0a075eb service nova] Lock "d0c7002c-8a3a-4bc5-940c-63cf4264668b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:17:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-25072b7c-3681-438c-9bf0-b481823f2ed4 req-06e7cc15-5d1b-42c8-9236-80bfd0a075eb service nova] Lock "d0c7002c-8a3a-4bc5-940c-63cf4264668b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:17:52 user nova-compute[72351]: DEBUG nova.compute.manager [req-25072b7c-3681-438c-9bf0-b481823f2ed4 req-06e7cc15-5d1b-42c8-9236-80bfd0a075eb service nova] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] No waiting events found dispatching network-vif-plugged-9734c099-ce77-4e2d-ab87-6ae4ab56c56b {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:17:52 user nova-compute[72351]: WARNING nova.compute.manager [req-25072b7c-3681-438c-9bf0-b481823f2ed4 req-06e7cc15-5d1b-42c8-9236-80bfd0a075eb service nova] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Received unexpected event network-vif-plugged-9734c099-ce77-4e2d-ab87-6ae4ab56c56b for instance with vm_state building and task_state spawning.
Aug 09 02:17:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7c5e633d-a32f-4c5e-bb9d-1e40da964bd0/disk --force-share --output=json" returned: 0 in 0.154s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:17:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7c5e633d-a32f-4c5e-bb9d-1e40da964bd0/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:17:52 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:52 user nova-compute[72351]: INFO nova.compute.manager [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Took 8.65 seconds to build instance.
Aug 09 02:17:53 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-fe4b71d9-3f6f-414d-a078-b1f1e1248b33 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Lock "d0c7002c-8a3a-4bc5-940c-63cf4264668b" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 8.839s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:17:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7c5e633d-a32f-4c5e-bb9d-1e40da964bd0/disk --force-share --output=json" returned: 0 in 0.155s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:17:53 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:17:53 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:17:53 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Hypervisor/Node resource view: name=user free_ram=7655MB free_disk=25.86275863647461GB free_vcpus=5 pci_devices=[{"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72351) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Aug 09 02:17:53 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:17:53 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:17:53 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance f45b57fa-9264-45f0-9573-e5fb58848d1f actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:17:53 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 2d78475c-107d-4256-b12e-dbe9c8f3b9b2 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:17:53 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 50302d76-f6b9-4fe3-b55a-4a01e483e2e3 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:17:53 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 74728ca8-34d4-45e6-a1f2-21c0952ff835 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:17:53 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:17:53 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:17:53 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance d0c7002c-8a3a-4bc5-940c-63cf4264668b actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:17:53 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Total usable vcpus: 12, total allocated vcpus: 7 {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Aug 09 02:17:53 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Final resource view: name=user phys_ram=16011MB used_ram=4096MB phys_disk=40GB used_disk=7GB total_vcpus=12 used_vcpus=7 pci_stats=[] {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Aug 09 02:17:54 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:17:54 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:17:54 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Compute_service record updated for user:user {{(pid=72351) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Aug 09 02:17:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.719s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:17:55 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:17:55 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:17:55 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Starting heal instance info cache {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9842}}
Aug 09 02:17:55 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Rebuilding the list of instances to heal {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9846}}
Aug 09 02:17:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "refresh_cache-f45b57fa-9264-45f0-9573-e5fb58848d1f" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:17:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquired lock "refresh_cache-f45b57fa-9264-45f0-9573-e5fb58848d1f" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:17:55 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Forcefully refreshing network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1996}}
Aug 09 02:17:55 user nova-compute[72351]: DEBUG nova.objects.instance [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lazy-loading 'info_cache' on Instance uuid f45b57fa-9264-45f0-9573-e5fb58848d1f {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:17:56 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Updating instance_info_cache with network_info: [{"id": "e9a7bda3-90fa-4274-acc9-3597835503e9", "address": "fa:16:3e:a0:ba:9c", "network": {"id": "518cac69-7290-48e2-8026-3e1a92d7b7fd", "bridge": "br-int", "label": "tempest-DeleteServersTestJSON-799055594-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e31482d532bf425386728e29c8c90f19", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape9a7bda3-90", "ovs_interfaceid": "e9a7bda3-90fa-4274-acc9-3597835503e9", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:17:56 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Releasing lock "refresh_cache-f45b57fa-9264-45f0-9573-e5fb58848d1f" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:17:56 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Updated the network info_cache for instance {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9913}}
Aug 09 02:17:56 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:17:56 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:17:56 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:17:56 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:17:56 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:17:56 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72351) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10461}}
Aug 09 02:17:57 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:17:57 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:18:00 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:18:02 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:18:02 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:18:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Acquiring lock "97b13ccd-15b4-44bf-b865-71b4da861168" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:18:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "97b13ccd-15b4-44bf-b865-71b4da861168" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:18:05 user nova-compute[72351]: DEBUG nova.compute.manager [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Starting instance... {{(pid=72351) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2406}}
Aug 09 02:18:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:18:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:18:05 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72351) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Aug 09 02:18:05 user nova-compute[72351]: INFO nova.compute.claims [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Claim successful on node user
Aug 09 02:18:06 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Refreshing inventories for resource provider 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:804}}
Aug 09 02:18:06 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Updating ProviderTree inventory for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 from _refresh_and_get_inventory using data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) _refresh_and_get_inventory /opt/stack/nova/nova/scheduler/client/report.py:768}}
Aug 09 02:18:06 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Updating inventory in ProviderTree for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 with inventory: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:176}}
Aug 09 02:18:06 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Refreshing aggregate associations for resource provider 8eae9b37-1236-4e6d-91a5-b650de27b009, aggregates: None {{(pid=72351) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:813}}
Aug 09 02:18:06 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Refreshing trait associations for resource provider 8eae9b37-1236-4e6d-91a5-b650de27b009, traits: COMPUTE_VIOMMU_MODEL_INTEL,COMPUTE_STORAGE_BUS_IDE,COMPUTE_GRAPHICS_MODEL_NONE,COMPUTE_NET_VIF_MODEL_E1000E,COMPUTE_IMAGE_TYPE_AKI,COMPUTE_VOLUME_ATTACH_WITH_TAG,COMPUTE_NET_VIF_MODEL_VMXNET3,COMPUTE_SOCKET_PCI_NUMA_AFFINITY,COMPUTE_ACCELERATORS,HW_CPU_X86_SSE41,COMPUTE_NET_ATTACH_INTERFACE_WITH_TAG,COMPUTE_GRAPHICS_MODEL_VGA,COMPUTE_GRAPHICS_MODEL_QXL,COMPUTE_VIOMMU_MODEL_AUTO,COMPUTE_SECURITY_UEFI_SECURE_BOOT,COMPUTE_NET_VIF_MODEL_E1000,COMPUTE_TRUSTED_CERTS,COMPUTE_VOLUME_MULTI_ATTACH,COMPUTE_STORAGE_BUS_FDC,COMPUTE_VOLUME_EXTEND,COMPUTE_GRAPHICS_MODEL_CIRRUS,COMPUTE_GRAPHICS_MODEL_VMVGA,COMPUTE_GRAPHICS_MODEL_BOCHS,COMPUTE_DEVICE_TAGGING,COMPUTE_STORAGE_BUS_SCSI,COMPUTE_STORAGE_BUS_VIRTIO,HW_CPU_X86_SSSE3,HW_CPU_X86_SSE2,COMPUTE_IMAGE_TYPE_AMI,COMPUTE_STORAGE_BUS_USB,COMPUTE_NET_VIF_MODEL_PCNET,COMPUTE_NET_VIF_MODEL_VIRTIO,COMPUTE_IMAGE_TYPE_ISO,HW_CPU_X86_SSE42,HW_CPU_X86_SSE,COMPUTE_IMAGE_TYPE_ARI,COMPUTE_IMAGE_TYPE_RAW,COMPUTE_IMAGE_TYPE_QCOW2,COMPUTE_NET_VIF_MODEL_NE2K_PCI,COMPUTE_NODE,HW_CPU_X86_MMX,COMPUTE_NET_VIF_MODEL_RTL8139,COMPUTE_GRAPHICS_MODEL_VIRTIO,COMPUTE_NET_VIF_MODEL_SPAPR_VLAN,COMPUTE_STORAGE_BUS_SATA,COMPUTE_NET_ATTACH_INTERFACE,COMPUTE_RESCUE_BFV {{(pid=72351) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:825}}
Aug 09 02:18:06 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:18:06 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:18:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.617s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:18:06 user nova-compute[72351]: DEBUG nova.compute.manager [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Start building networks asynchronously for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2803}}
Aug 09 02:18:06 user nova-compute[72351]: DEBUG nova.compute.manager [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Allocating IP information in the background. {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1956}}
Aug 09 02:18:06 user nova-compute[72351]: DEBUG nova.network.neutron [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] allocate_for_instance() {{(pid=72351) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Aug 09 02:18:06 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Aug 09 02:18:06 user nova-compute[72351]: DEBUG nova.compute.manager [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Start building block device mappings for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2838}}
Aug 09 02:18:06 user nova-compute[72351]: DEBUG nova.policy [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'bec5b5704e9b4e3480f428d5c037c20d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c2f67da658d94fabbab82ee6a6171c4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72351) authorize /opt/stack/nova/nova/policy.py:203}}
Aug 09 02:18:06 user nova-compute[72351]: DEBUG nova.compute.manager [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Start spawning the instance on the hypervisor. {{(pid=72351) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2612}}
Aug 09 02:18:06 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Creating instance directory {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4717}}
Aug 09 02:18:06 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Creating image(s)
Aug 09 02:18:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Acquiring lock "/opt/stack/data/nova/instances/97b13ccd-15b4-44bf-b865-71b4da861168/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:18:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "/opt/stack/data/nova/instances/97b13ccd-15b4-44bf-b865-71b4da861168/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:18:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "/opt/stack/data/nova/instances/97b13ccd-15b4-44bf-b865-71b4da861168/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:18:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Acquiring lock "78680203fb06db2bb9e4fc0abad2df7c1c424054" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:18:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "78680203fb06db2bb9e4fc0abad2df7c1c424054" acquired by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:18:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/78680203fb06db2bb9e4fc0abad2df7c1c424054.part --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/78680203fb06db2bb9e4fc0abad2df7c1c424054.part --force-share --output=json" returned: 0 in 0.182s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:07 user nova-compute[72351]: DEBUG nova.virt.images [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] 9eadc1b0-9cb8-4bc1-97d6-af86e590c548 was qcow2, converting to raw {{(pid=72351) fetch_to_raw /opt/stack/nova/nova/virt/images.py:165}}
Aug 09 02:18:07 user nova-compute[72351]: DEBUG nova.privsep.utils [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=72351) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Aug 09 02:18:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Running cmd (subprocess): qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/78680203fb06db2bb9e4fc0abad2df7c1c424054.part /opt/stack/data/nova/instances/_base/78680203fb06db2bb9e4fc0abad2df7c1c424054.converted {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:07 user nova-compute[72351]: DEBUG nova.network.neutron [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Successfully created port: 74c98543-ce04-43d8-a58c-267e372db698 {{(pid=72351) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Aug 09 02:18:07 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:18:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] CMD "qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/78680203fb06db2bb9e4fc0abad2df7c1c424054.part /opt/stack/data/nova/instances/_base/78680203fb06db2bb9e4fc0abad2df7c1c424054.converted" returned: 0 in 0.155s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/78680203fb06db2bb9e4fc0abad2df7c1c424054.converted --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/78680203fb06db2bb9e4fc0abad2df7c1c424054.converted --force-share --output=json" returned: 0 in 0.138s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:07 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "78680203fb06db2bb9e4fc0abad2df7c1c424054" "released" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: held 0.876s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:18:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/78680203fb06db2bb9e4fc0abad2df7c1c424054 --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/78680203fb06db2bb9e4fc0abad2df7c1c424054 --force-share --output=json" returned: 0 in 0.144s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:07 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Acquiring lock "78680203fb06db2bb9e4fc0abad2df7c1c424054" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:18:07 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "78680203fb06db2bb9e4fc0abad2df7c1c424054" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:18:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/78680203fb06db2bb9e4fc0abad2df7c1c424054 --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/78680203fb06db2bb9e4fc0abad2df7c1c424054 --force-share --output=json" returned: 0 in 0.136s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/78680203fb06db2bb9e4fc0abad2df7c1c424054,backing_fmt=raw /opt/stack/data/nova/instances/97b13ccd-15b4-44bf-b865-71b4da861168/disk 1073741824 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:07 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:18:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/78680203fb06db2bb9e4fc0abad2df7c1c424054,backing_fmt=raw /opt/stack/data/nova/instances/97b13ccd-15b4-44bf-b865-71b4da861168/disk 1073741824" returned: 0 in 0.212s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "78680203fb06db2bb9e4fc0abad2df7c1c424054" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.352s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:18:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/78680203fb06db2bb9e4fc0abad2df7c1c424054 --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/78680203fb06db2bb9e4fc0abad2df7c1c424054 --force-share --output=json" returned: 0 in 0.142s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:08 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Checking if we can resize image /opt/stack/data/nova/instances/97b13ccd-15b4-44bf-b865-71b4da861168/disk. size=1073741824 {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Aug 09 02:18:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/97b13ccd-15b4-44bf-b865-71b4da861168/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:08 user nova-compute[72351]: DEBUG nova.network.neutron [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Successfully updated port: 74c98543-ce04-43d8-a58c-267e372db698 {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:18:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Acquiring lock "refresh_cache-97b13ccd-15b4-44bf-b865-71b4da861168" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:18:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Acquired lock "refresh_cache-97b13ccd-15b4-44bf-b865-71b4da861168" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:18:08 user nova-compute[72351]: DEBUG nova.network.neutron [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:18:08 user nova-compute[72351]: DEBUG nova.compute.manager [req-340b8766-4412-4e53-af27-f06c63ed06f1 req-8b721e8c-a3c8-4d50-91bd-3d1a3453b3f7 service nova] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Received event network-changed-74c98543-ce04-43d8-a58c-267e372db698 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:18:08 user nova-compute[72351]: DEBUG nova.compute.manager [req-340b8766-4412-4e53-af27-f06c63ed06f1 req-8b721e8c-a3c8-4d50-91bd-3d1a3453b3f7 service nova] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Refreshing instance network info cache due to event network-changed-74c98543-ce04-43d8-a58c-267e372db698. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:18:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-340b8766-4412-4e53-af27-f06c63ed06f1 req-8b721e8c-a3c8-4d50-91bd-3d1a3453b3f7 service nova] Acquiring lock "refresh_cache-97b13ccd-15b4-44bf-b865-71b4da861168" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:18:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/97b13ccd-15b4-44bf-b865-71b4da861168/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:08 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Cannot resize image /opt/stack/data/nova/instances/97b13ccd-15b4-44bf-b865-71b4da861168/disk to a smaller size. {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Aug 09 02:18:08 user nova-compute[72351]: DEBUG nova.objects.instance [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lazy-loading 'migration_context' on Instance uuid 97b13ccd-15b4-44bf-b865-71b4da861168 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:18:08 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Created local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4851}}
Aug 09 02:18:08 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Ensure instance console log exists: /opt/stack/data/nova/instances/97b13ccd-15b4-44bf-b865-71b4da861168/console.log {{(pid=72351) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4603}}
Aug 09 02:18:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:18:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:18:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:18:08 user nova-compute[72351]: DEBUG nova.network.neutron [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:18:08 user nova-compute[72351]: DEBUG nova.network.neutron [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Updating instance_info_cache with network_info: [{"id": "74c98543-ce04-43d8-a58c-267e372db698", "address": "fa:16:3e:d4:4d:0a", "network": {"id": "7024d1f6-a400-4482-9364-78ae3b6adc80", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1603258958-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c2f67da658d94fabbab82ee6a6171c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap74c98543-ce", "ovs_interfaceid": "74c98543-ce04-43d8-a58c-267e372db698", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:18:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Releasing lock "refresh_cache-97b13ccd-15b4-44bf-b865-71b4da861168" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:18:08 user nova-compute[72351]: DEBUG nova.compute.manager [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Instance network_info: |[{"id": "74c98543-ce04-43d8-a58c-267e372db698", "address": "fa:16:3e:d4:4d:0a", "network": {"id": "7024d1f6-a400-4482-9364-78ae3b6adc80", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1603258958-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c2f67da658d94fabbab82ee6a6171c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap74c98543-ce", "ovs_interfaceid": "74c98543-ce04-43d8-a58c-267e372db698", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1971}}
Aug 09 02:18:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-340b8766-4412-4e53-af27-f06c63ed06f1 req-8b721e8c-a3c8-4d50-91bd-3d1a3453b3f7 service nova] Acquired lock "refresh_cache-97b13ccd-15b4-44bf-b865-71b4da861168" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:18:08 user nova-compute[72351]: DEBUG nova.network.neutron [req-340b8766-4412-4e53-af27-f06c63ed06f1 req-8b721e8c-a3c8-4d50-91bd-3d1a3453b3f7 service nova] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Refreshing network info cache for port 74c98543-ce04-43d8-a58c-267e372db698 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:18:08 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Start _get_guest_xml network_info=[{"id": "74c98543-ce04-43d8-a58c-267e372db698", "address": "fa:16:3e:d4:4d:0a", "network": {"id": "7024d1f6-a400-4482-9364-78ae3b6adc80", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1603258958-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c2f67da658d94fabbab82ee6a6171c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap74c98543-ce", "ovs_interfaceid": "74c98543-ce04-43d8-a58c-267e372db698", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:18:02Z,direct_url=<?>,disk_format='qcow2',id=9eadc1b0-9cb8-4bc1-97d6-af86e590c548,min_disk=0,min_ram=0,name='tempest-scenario-img--58943559',owner='c2f67da658d94fabbab82ee6a6171c4b',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:18:04Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'size': 0, 'device_type': 'disk', 'encryption_secret_uuid': None, 'guest_format': None, 'encryption_options': None, 'disk_bus': 'virtio', 'encrypted': False, 'encryption_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'image_id': '9eadc1b0-9cb8-4bc1-97d6-af86e590c548'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:18:08 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:18:08 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:18:08 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:18:08 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:18:08 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:18:08 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:18:08 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:18:08 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:18:02Z,direct_url=<?>,disk_format='qcow2',id=9eadc1b0-9cb8-4bc1-97d6-af86e590c548,min_disk=0,min_ram=0,name='tempest-scenario-img--58943559',owner='c2f67da658d94fabbab82ee6a6171c4b',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:18:04Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:18:08 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:18:08 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:18:08 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:18:08 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:18:08 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:18:08 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:18:08 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:18:08 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:18:08 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:18:08 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:18:08 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:18:04Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestMinimumBasicScenario-server-1233822315',display_name='tempest-TestMinimumBasicScenario-server-1233822315',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testminimumbasicscenario-server-1233822315',id=8,image_ref='9eadc1b0-9cb8-4bc1-97d6-af86e590c548',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOk8lLSpSCaCDIKlT4lhWRaHWy+oi2s0IJNKXHyKayH1X4sCjRLnVKeip/ShH5Av4ZZCopMp+S7MD8OAGw8MyvDsE398yCbkjnI/7XCIasyAoPOQXBUUZ52swhczfVlPLg==',key_name='tempest-TestMinimumBasicScenario-1974156570',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='c2f67da658d94fabbab82ee6a6171c4b',ramdisk_id='',reservation_id='r-rhqfwdu0',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='9eadc1b0-9cb8-4bc1-97d6-af86e590c548',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_min_disk='1',image_min_ram='0',network_allocated='True',owner_project_name='tempest-TestMinimumBasicScenario-1380132717',owner_user_name='tempest-TestMinimumBasicScenario-1380132717-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:18:07Z,user_data=None,user_id='bec5b5704e9b4e3480f428d5c037c20d',uuid=97b13ccd-15b4-44bf-b865-71b4da861168,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "74c98543-ce04-43d8-a58c-267e372db698", "address": "fa:16:3e:d4:4d:0a", "network": {"id": "7024d1f6-a400-4482-9364-78ae3b6adc80", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1603258958-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c2f67da658d94fabbab82ee6a6171c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap74c98543-ce", "ovs_interfaceid": "74c98543-ce04-43d8-a58c-267e372db698", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:18:08 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Converting VIF {"id": "74c98543-ce04-43d8-a58c-267e372db698", "address": "fa:16:3e:d4:4d:0a", "network": {"id": "7024d1f6-a400-4482-9364-78ae3b6adc80", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1603258958-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c2f67da658d94fabbab82ee6a6171c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap74c98543-ce", "ovs_interfaceid": "74c98543-ce04-43d8-a58c-267e372db698", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:18:08 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:d4:4d:0a,bridge_name='br-int',has_traffic_filtering=True,id=74c98543-ce04-43d8-a58c-267e372db698,network=Network(7024d1f6-a400-4482-9364-78ae3b6adc80),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap74c98543-ce') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:18:08 user nova-compute[72351]: DEBUG nova.objects.instance [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lazy-loading 'pci_devices' on Instance uuid 97b13ccd-15b4-44bf-b865-71b4da861168 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:18:08 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:18:08 user nova-compute[72351]:   <uuid>97b13ccd-15b4-44bf-b865-71b4da861168</uuid>
Aug 09 02:18:08 user nova-compute[72351]:   <name>instance-00000008</name>
Aug 09 02:18:08 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:18:08 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:18:08 user nova-compute[72351]:   <metadata>
Aug 09 02:18:08 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:18:08 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:18:08 user nova-compute[72351]:       <nova:name>tempest-TestMinimumBasicScenario-server-1233822315</nova:name>
Aug 09 02:18:08 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:18:08</nova:creationTime>
Aug 09 02:18:08 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:18:08 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:18:08 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:18:08 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:18:08 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:18:08 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:18:08 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:18:08 user nova-compute[72351]:       <nova:owner>
Aug 09 02:18:08 user nova-compute[72351]:         <nova:user uuid="bec5b5704e9b4e3480f428d5c037c20d">tempest-TestMinimumBasicScenario-1380132717-project-member</nova:user>
Aug 09 02:18:08 user nova-compute[72351]:         <nova:project uuid="c2f67da658d94fabbab82ee6a6171c4b">tempest-TestMinimumBasicScenario-1380132717</nova:project>
Aug 09 02:18:08 user nova-compute[72351]:       </nova:owner>
Aug 09 02:18:08 user nova-compute[72351]:       <nova:root type="image" uuid="9eadc1b0-9cb8-4bc1-97d6-af86e590c548"/>
Aug 09 02:18:08 user nova-compute[72351]:       <nova:ports>
Aug 09 02:18:08 user nova-compute[72351]:         <nova:port uuid="74c98543-ce04-43d8-a58c-267e372db698">
Aug 09 02:18:08 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.12" ipVersion="4"/>
Aug 09 02:18:08 user nova-compute[72351]:         </nova:port>
Aug 09 02:18:08 user nova-compute[72351]:       </nova:ports>
Aug 09 02:18:08 user nova-compute[72351]:     </nova:instance>
Aug 09 02:18:08 user nova-compute[72351]:   </metadata>
Aug 09 02:18:08 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:18:08 user nova-compute[72351]:     <system>
Aug 09 02:18:08 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:18:08 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:18:08 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:18:08 user nova-compute[72351]:       <entry name="serial">97b13ccd-15b4-44bf-b865-71b4da861168</entry>
Aug 09 02:18:08 user nova-compute[72351]:       <entry name="uuid">97b13ccd-15b4-44bf-b865-71b4da861168</entry>
Aug 09 02:18:08 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:18:08 user nova-compute[72351]:     </system>
Aug 09 02:18:08 user nova-compute[72351]:   </sysinfo>
Aug 09 02:18:08 user nova-compute[72351]:   <os>
Aug 09 02:18:08 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:18:08 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:18:08 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:18:08 user nova-compute[72351]:   </os>
Aug 09 02:18:08 user nova-compute[72351]:   <features>
Aug 09 02:18:08 user nova-compute[72351]:     <acpi/>
Aug 09 02:18:08 user nova-compute[72351]:     <apic/>
Aug 09 02:18:08 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:18:08 user nova-compute[72351]:   </features>
Aug 09 02:18:08 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:18:08 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:18:08 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:18:08 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:18:08 user nova-compute[72351]:   </clock>
Aug 09 02:18:08 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:18:08 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:18:08 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:18:08 user nova-compute[72351]:   </cpu>
Aug 09 02:18:08 user nova-compute[72351]:   <devices>
Aug 09 02:18:08 user nova-compute[72351]:     <disk type="file" device="disk">
Aug 09 02:18:08 user nova-compute[72351]:       <driver name="qemu" type="qcow2" cache="none"/>
Aug 09 02:18:08 user nova-compute[72351]:       <source file="/opt/stack/data/nova/instances/97b13ccd-15b4-44bf-b865-71b4da861168/disk"/>
Aug 09 02:18:08 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:18:08 user nova-compute[72351]:     </disk>
Aug 09 02:18:08 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:18:08 user nova-compute[72351]:       <mac address="fa:16:3e:d4:4d:0a"/>
Aug 09 02:18:08 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:18:08 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:18:08 user nova-compute[72351]:       <target dev="tap74c98543-ce"/>
Aug 09 02:18:08 user nova-compute[72351]:     </interface>
Aug 09 02:18:08 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:18:08 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/97b13ccd-15b4-44bf-b865-71b4da861168/console.log" append="off"/>
Aug 09 02:18:08 user nova-compute[72351]:     </serial>
Aug 09 02:18:08 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:18:08 user nova-compute[72351]:     <video>
Aug 09 02:18:08 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:18:08 user nova-compute[72351]:     </video>
Aug 09 02:18:08 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:18:08 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:18:08 user nova-compute[72351]:     </rng>
Aug 09 02:18:08 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:18:08 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:18:08 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:18:08 user nova-compute[72351]:     </memballoon>
Aug 09 02:18:08 user nova-compute[72351]:   </devices>
Aug 09 02:18:08 user nova-compute[72351]: </domain>
Aug 09 02:18:08 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:18:08 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:18:04Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestMinimumBasicScenario-server-1233822315',display_name='tempest-TestMinimumBasicScenario-server-1233822315',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testminimumbasicscenario-server-1233822315',id=8,image_ref='9eadc1b0-9cb8-4bc1-97d6-af86e590c548',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOk8lLSpSCaCDIKlT4lhWRaHWy+oi2s0IJNKXHyKayH1X4sCjRLnVKeip/ShH5Av4ZZCopMp+S7MD8OAGw8MyvDsE398yCbkjnI/7XCIasyAoPOQXBUUZ52swhczfVlPLg==',key_name='tempest-TestMinimumBasicScenario-1974156570',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='c2f67da658d94fabbab82ee6a6171c4b',ramdisk_id='',reservation_id='r-rhqfwdu0',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='9eadc1b0-9cb8-4bc1-97d6-af86e590c548',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_min_disk='1',image_min_ram='0',network_allocated='True',owner_project_name='tempest-TestMinimumBasicScenario-1380132717',owner_user_name='tempest-TestMinimumBasicScenario-1380132717-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:18:07Z,user_data=None,user_id='bec5b5704e9b4e3480f428d5c037c20d',uuid=97b13ccd-15b4-44bf-b865-71b4da861168,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "74c98543-ce04-43d8-a58c-267e372db698", "address": "fa:16:3e:d4:4d:0a", "network": {"id": "7024d1f6-a400-4482-9364-78ae3b6adc80", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1603258958-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c2f67da658d94fabbab82ee6a6171c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap74c98543-ce", "ovs_interfaceid": "74c98543-ce04-43d8-a58c-267e372db698", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:18:08 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Converting VIF {"id": "74c98543-ce04-43d8-a58c-267e372db698", "address": "fa:16:3e:d4:4d:0a", "network": {"id": "7024d1f6-a400-4482-9364-78ae3b6adc80", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1603258958-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c2f67da658d94fabbab82ee6a6171c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap74c98543-ce", "ovs_interfaceid": "74c98543-ce04-43d8-a58c-267e372db698", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:18:08 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:d4:4d:0a,bridge_name='br-int',has_traffic_filtering=True,id=74c98543-ce04-43d8-a58c-267e372db698,network=Network(7024d1f6-a400-4482-9364-78ae3b6adc80),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap74c98543-ce') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:18:08 user nova-compute[72351]: DEBUG os_vif [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:d4:4d:0a,bridge_name='br-int',has_traffic_filtering=True,id=74c98543-ce04-43d8-a58c-267e372db698,network=Network(7024d1f6-a400-4482-9364-78ae3b6adc80),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap74c98543-ce') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:18:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:18:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:18:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:18:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:18:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap74c98543-ce, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:18:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap74c98543-ce, col_values=(('external_ids', {'iface-id': '74c98543-ce04-43d8-a58c-267e372db698', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:d4:4d:0a', 'vm-uuid': '97b13ccd-15b4-44bf-b865-71b4da861168'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:18:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:18:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:18:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:18:08 user nova-compute[72351]: INFO os_vif [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:d4:4d:0a,bridge_name='br-int',has_traffic_filtering=True,id=74c98543-ce04-43d8-a58c-267e372db698,network=Network(7024d1f6-a400-4482-9364-78ae3b6adc80),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap74c98543-ce')
Aug 09 02:18:09 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:18:09 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] No VIF found with MAC fa:16:3e:d4:4d:0a, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:18:09 user nova-compute[72351]: DEBUG nova.network.neutron [req-340b8766-4412-4e53-af27-f06c63ed06f1 req-8b721e8c-a3c8-4d50-91bd-3d1a3453b3f7 service nova] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Updated VIF entry in instance network info cache for port 74c98543-ce04-43d8-a58c-267e372db698. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:18:09 user nova-compute[72351]: DEBUG nova.network.neutron [req-340b8766-4412-4e53-af27-f06c63ed06f1 req-8b721e8c-a3c8-4d50-91bd-3d1a3453b3f7 service nova] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Updating instance_info_cache with network_info: [{"id": "74c98543-ce04-43d8-a58c-267e372db698", "address": "fa:16:3e:d4:4d:0a", "network": {"id": "7024d1f6-a400-4482-9364-78ae3b6adc80", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1603258958-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c2f67da658d94fabbab82ee6a6171c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap74c98543-ce", "ovs_interfaceid": "74c98543-ce04-43d8-a58c-267e372db698", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:18:09 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-340b8766-4412-4e53-af27-f06c63ed06f1 req-8b721e8c-a3c8-4d50-91bd-3d1a3453b3f7 service nova] Releasing lock "refresh_cache-97b13ccd-15b4-44bf-b865-71b4da861168" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:18:10 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:18:10 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:18:10 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:18:10 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:18:10 user nova-compute[72351]: DEBUG nova.compute.manager [req-a70b0dcc-4764-49a3-a605-a28433e65c88 req-77a3f106-165e-4be1-9b3b-c3820de293ef service nova] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Received event network-vif-plugged-74c98543-ce04-43d8-a58c-267e372db698 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:18:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a70b0dcc-4764-49a3-a605-a28433e65c88 req-77a3f106-165e-4be1-9b3b-c3820de293ef service nova] Acquiring lock "97b13ccd-15b4-44bf-b865-71b4da861168-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:18:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a70b0dcc-4764-49a3-a605-a28433e65c88 req-77a3f106-165e-4be1-9b3b-c3820de293ef service nova] Lock "97b13ccd-15b4-44bf-b865-71b4da861168-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:18:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a70b0dcc-4764-49a3-a605-a28433e65c88 req-77a3f106-165e-4be1-9b3b-c3820de293ef service nova] Lock "97b13ccd-15b4-44bf-b865-71b4da861168-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:18:10 user nova-compute[72351]: DEBUG nova.compute.manager [req-a70b0dcc-4764-49a3-a605-a28433e65c88 req-77a3f106-165e-4be1-9b3b-c3820de293ef service nova] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] No waiting events found dispatching network-vif-plugged-74c98543-ce04-43d8-a58c-267e372db698 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:18:10 user nova-compute[72351]: WARNING nova.compute.manager [req-a70b0dcc-4764-49a3-a605-a28433e65c88 req-77a3f106-165e-4be1-9b3b-c3820de293ef service nova] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Received unexpected event network-vif-plugged-74c98543-ce04-43d8-a58c-267e372db698 for instance with vm_state building and task_state spawning.
Aug 09 02:18:11 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:18:11 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:18:11 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:18:11 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:18:12 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:18:12 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547492.6646059, 97b13ccd-15b4-44bf-b865-71b4da861168 => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:18:12 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] VM Resumed (Lifecycle Event)
Aug 09 02:18:12 user nova-compute[72351]: DEBUG nova.compute.manager [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:18:12 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Guest created on hypervisor {{(pid=72351) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4411}}
Aug 09 02:18:12 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Instance spawned successfully.
Aug 09 02:18:12 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:894}}
Aug 09 02:18:12 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:18:12 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:18:12 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Found default for hw_cdrom_bus of ide {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:18:12 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Found default for hw_disk_bus of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:18:12 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Found default for hw_input_bus of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:18:12 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Found default for hw_pointer_model of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:18:12 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Found default for hw_video_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:18:12 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Found default for hw_vif_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:18:12 user nova-compute[72351]: DEBUG nova.compute.manager [req-c8e3c75c-4ad6-4080-8676-77890662b863 req-9f3d9a64-7927-44c5-8183-72fa47fd54bc service nova] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Received event network-vif-plugged-74c98543-ce04-43d8-a58c-267e372db698 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:18:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c8e3c75c-4ad6-4080-8676-77890662b863 req-9f3d9a64-7927-44c5-8183-72fa47fd54bc service nova] Acquiring lock "97b13ccd-15b4-44bf-b865-71b4da861168-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:18:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c8e3c75c-4ad6-4080-8676-77890662b863 req-9f3d9a64-7927-44c5-8183-72fa47fd54bc service nova] Lock "97b13ccd-15b4-44bf-b865-71b4da861168-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:18:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c8e3c75c-4ad6-4080-8676-77890662b863 req-9f3d9a64-7927-44c5-8183-72fa47fd54bc service nova] Lock "97b13ccd-15b4-44bf-b865-71b4da861168-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:18:12 user nova-compute[72351]: DEBUG nova.compute.manager [req-c8e3c75c-4ad6-4080-8676-77890662b863 req-9f3d9a64-7927-44c5-8183-72fa47fd54bc service nova] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] No waiting events found dispatching network-vif-plugged-74c98543-ce04-43d8-a58c-267e372db698 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:18:12 user nova-compute[72351]: WARNING nova.compute.manager [req-c8e3c75c-4ad6-4080-8676-77890662b863 req-9f3d9a64-7927-44c5-8183-72fa47fd54bc service nova] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Received unexpected event network-vif-plugged-74c98543-ce04-43d8-a58c-267e372db698 for instance with vm_state building and task_state spawning.
Aug 09 02:18:12 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:18:12 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547492.6692233, 97b13ccd-15b4-44bf-b865-71b4da861168 => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:18:12 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] VM Started (Lifecycle Event)
Aug 09 02:18:12 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:18:12 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:18:12 user nova-compute[72351]: INFO nova.compute.manager [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Took 6.20 seconds to spawn the instance on the hypervisor.
Aug 09 02:18:12 user nova-compute[72351]: DEBUG nova.compute.manager [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:18:13 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:18:13 user nova-compute[72351]: INFO nova.compute.manager [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Took 7.40 seconds to build instance.
Aug 09 02:18:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-bb1ce7cd-d8e8-4317-acd5-20ba334dc76c tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "97b13ccd-15b4-44bf-b865-71b4da861168" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 7.538s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:18:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:18:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:18:17 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:18:17 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:18:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:18:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:18:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:18:23 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:18:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:18:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Acquiring lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:18:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:18:26 user nova-compute[72351]: DEBUG nova.compute.manager [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Starting instance... {{(pid=72351) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2406}}
Aug 09 02:18:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:18:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:18:26 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72351) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Aug 09 02:18:26 user nova-compute[72351]: INFO nova.compute.claims [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Claim successful on node user
Aug 09 02:18:26 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:18:26 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:18:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.490s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:18:26 user nova-compute[72351]: DEBUG nova.compute.manager [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Start building networks asynchronously for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2803}}
Aug 09 02:18:26 user nova-compute[72351]: DEBUG nova.compute.manager [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Allocating IP information in the background. {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1956}}
Aug 09 02:18:26 user nova-compute[72351]: DEBUG nova.network.neutron [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] allocate_for_instance() {{(pid=72351) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Aug 09 02:18:26 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Aug 09 02:18:26 user nova-compute[72351]: DEBUG nova.compute.manager [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Start building block device mappings for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2838}}
Aug 09 02:18:26 user nova-compute[72351]: INFO nova.virt.block_device [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Booting with volume a8ce86ce-79f7-4138-a49b-c45c9db38c97 at /dev/vda
Aug 09 02:18:26 user nova-compute[72351]: DEBUG os_brick.utils [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:18:26 user nova-compute[72351]: INFO oslo.privsep.daemon [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Running privsep helper: ['sudo', 'nova-rootwrap', '/etc/nova/rootwrap.conf', 'privsep-helper', '--config-file', '/etc/nova/nova-cpu.conf', '--privsep_context', 'os_brick.privileged.default', '--privsep_sock_path', '/tmp/tmp0rh9b3br/privsep.sock']
Aug 09 02:18:27 user nova-compute[72351]: DEBUG nova.policy [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'c74c3939709643c8915c7d1678935e9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6908d784cc1a41dbbb4bee47e70c259b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72351) authorize /opt/stack/nova/nova/policy.py:203}}
Aug 09 02:18:27 user sudo[83361]:    stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/nova-rootwrap /etc/nova/rootwrap.conf privsep-helper --config-file /etc/nova/nova-cpu.conf --privsep_context os_brick.privileged.default --privsep_sock_path /tmp/tmp0rh9b3br/privsep.sock
Aug 09 02:18:27 user sudo[83361]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=1001)
Aug 09 02:18:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:18:27 user nova-compute[72351]: DEBUG nova.network.neutron [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Successfully created port: e021d67e-09ab-4779-b639-dff1ba39798a {{(pid=72351) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Aug 09 02:18:28 user nova-compute[72351]: DEBUG nova.network.neutron [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Successfully created port: 5d0bb297-60b1-4e66-8f43-3bf1c32ae4ef {{(pid=72351) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Aug 09 02:18:28 user nova-compute[72351]: DEBUG nova.network.neutron [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Successfully created port: c2df74bf-22ce-48b1-9ab8-3a684cef7266 {{(pid=72351) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Aug 09 02:18:28 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:18:29 user sudo[83361]: pam_unix(sudo:session): session closed for user root
Aug 09 02:18:29 user nova-compute[72351]: INFO oslo.privsep.daemon [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Spawned new privsep daemon via rootwrap
Aug 09 02:18:29 user nova-compute[72351]: INFO oslo.privsep.daemon [-] privsep daemon starting
Aug 09 02:18:29 user nova-compute[72351]: INFO oslo.privsep.daemon [-] privsep process running with uid/gid: 0/0
Aug 09 02:18:29 user nova-compute[72351]: INFO oslo.privsep.daemon [-] privsep process running with capabilities (eff/prm/inh): CAP_SYS_ADMIN/CAP_SYS_ADMIN/none
Aug 09 02:18:29 user nova-compute[72351]: INFO oslo.privsep.daemon [-] privsep daemon running as pid 83366
Aug 09 02:18:29 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[c6ea7dfb-3040-48c1-a244-c312e90544cb]: (2,) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:18:29 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:29 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.015s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:29 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[fdde5dc3-df91-40fa-af31-4b4fbdc47770]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:a52269602ff4\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:18:29 user nova-compute[72351]: WARNING os_brick.initiator.connectors.nvmeof [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Could not find nvme_core/parameters/multipath: FileNotFoundError: [Errno 2] No such file or directory: '/sys/module/nvme_core/parameters/multipath'
Aug 09 02:18:29 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:29 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:29 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[53064f51-61ae-454e-9744-1fb7c582e5d6]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:18:29 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:29 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.016s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:29 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e4540ce7-abf4-43b7-8c95-4596351e8623]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:18:29 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[1b414408-1cea-4905-a719-115c62b04833]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:18:29 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Running cmd (subprocess): nvme version {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:29 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] 'nvme version' failed. Not Retrying. {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:18:29 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.nvmeof [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] nvme not present on system {{(pid=72351) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Aug 09 02:18:29 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:29 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:18:29 user nova-compute[72351]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Aug 09 02:18:29 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[5221dde7-d8da-4c6b-bf50-43cf00f39170]: (4, '') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:18:29 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72351) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Aug 09 02:18:29 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Aug 09 02:18:29 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] LIGHTOS: no hostnqn found. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Aug 09 02:18:29 user nova-compute[72351]: DEBUG os_brick.utils [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] <== get_connector_properties: return (2479ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:a52269602ff4', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:18:29 user nova-compute[72351]: DEBUG nova.virt.block_device [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Updating existing volume attachment record: 1176474a-644d-471f-8922-f986241b8f62 {{(pid=72351) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Aug 09 02:18:30 user nova-compute[72351]: DEBUG nova.network.neutron [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Successfully created port: d1b9314b-1800-41fb-a34b-0186f8e746da {{(pid=72351) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Aug 09 02:18:30 user nova-compute[72351]: DEBUG nova.network.neutron [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Successfully created port: c7c195c0-af76-445e-8a18-8169f0cdf5fc {{(pid=72351) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Aug 09 02:18:31 user nova-compute[72351]: DEBUG nova.network.neutron [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Successfully updated port: e021d67e-09ab-4779-b639-dff1ba39798a {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:18:31 user nova-compute[72351]: DEBUG nova.compute.manager [req-ed58fa7e-0d39-4d15-b453-f9d9d4b5ea19 req-64b9fce7-96ac-4121-b318-77401754c3a9 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received event network-changed-e021d67e-09ab-4779-b639-dff1ba39798a {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:18:31 user nova-compute[72351]: DEBUG nova.compute.manager [req-ed58fa7e-0d39-4d15-b453-f9d9d4b5ea19 req-64b9fce7-96ac-4121-b318-77401754c3a9 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Refreshing instance network info cache due to event network-changed-e021d67e-09ab-4779-b639-dff1ba39798a. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:18:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ed58fa7e-0d39-4d15-b453-f9d9d4b5ea19 req-64b9fce7-96ac-4121-b318-77401754c3a9 service nova] Acquiring lock "refresh_cache-629e177e-afbb-4765-a0b5-7a3f66f72ebf" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:18:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ed58fa7e-0d39-4d15-b453-f9d9d4b5ea19 req-64b9fce7-96ac-4121-b318-77401754c3a9 service nova] Acquired lock "refresh_cache-629e177e-afbb-4765-a0b5-7a3f66f72ebf" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:18:31 user nova-compute[72351]: DEBUG nova.network.neutron [req-ed58fa7e-0d39-4d15-b453-f9d9d4b5ea19 req-64b9fce7-96ac-4121-b318-77401754c3a9 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Refreshing network info cache for port e021d67e-09ab-4779-b639-dff1ba39798a {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:18:31 user nova-compute[72351]: DEBUG nova.network.neutron [req-ed58fa7e-0d39-4d15-b453-f9d9d4b5ea19 req-64b9fce7-96ac-4121-b318-77401754c3a9 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:18:32 user nova-compute[72351]: DEBUG nova.network.neutron [req-ed58fa7e-0d39-4d15-b453-f9d9d4b5ea19 req-64b9fce7-96ac-4121-b318-77401754c3a9 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:18:32 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ed58fa7e-0d39-4d15-b453-f9d9d4b5ea19 req-64b9fce7-96ac-4121-b318-77401754c3a9 service nova] Releasing lock "refresh_cache-629e177e-afbb-4765-a0b5-7a3f66f72ebf" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:18:32 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:18:32 user nova-compute[72351]: DEBUG nova.network.neutron [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Successfully updated port: f5ecb594-7ac3-496b-9bb0-9f908b91c8fb {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:18:33 user nova-compute[72351]: DEBUG nova.network.neutron [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Successfully updated port: 2e9c8a9c-2d81-4b84-b481-9a6b513e730c {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:18:33 user nova-compute[72351]: DEBUG nova.compute.manager [req-a638901f-7ca3-4cd4-ad45-f9f666d489d2 req-565cd695-3879-468b-a935-b2b5e1f4c350 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received event network-changed-f5ecb594-7ac3-496b-9bb0-9f908b91c8fb {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:18:33 user nova-compute[72351]: DEBUG nova.compute.manager [req-a638901f-7ca3-4cd4-ad45-f9f666d489d2 req-565cd695-3879-468b-a935-b2b5e1f4c350 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Refreshing instance network info cache due to event network-changed-f5ecb594-7ac3-496b-9bb0-9f908b91c8fb. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:18:33 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a638901f-7ca3-4cd4-ad45-f9f666d489d2 req-565cd695-3879-468b-a935-b2b5e1f4c350 service nova] Acquiring lock "refresh_cache-629e177e-afbb-4765-a0b5-7a3f66f72ebf" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:18:33 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a638901f-7ca3-4cd4-ad45-f9f666d489d2 req-565cd695-3879-468b-a935-b2b5e1f4c350 service nova] Acquired lock "refresh_cache-629e177e-afbb-4765-a0b5-7a3f66f72ebf" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:18:33 user nova-compute[72351]: DEBUG nova.network.neutron [req-a638901f-7ca3-4cd4-ad45-f9f666d489d2 req-565cd695-3879-468b-a935-b2b5e1f4c350 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Refreshing network info cache for port f5ecb594-7ac3-496b-9bb0-9f908b91c8fb {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:18:33 user nova-compute[72351]: DEBUG nova.network.neutron [req-a638901f-7ca3-4cd4-ad45-f9f666d489d2 req-565cd695-3879-468b-a935-b2b5e1f4c350 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:18:33 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:18:34 user nova-compute[72351]: DEBUG nova.network.neutron [req-a638901f-7ca3-4cd4-ad45-f9f666d489d2 req-565cd695-3879-468b-a935-b2b5e1f4c350 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:18:34 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:18:34 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a638901f-7ca3-4cd4-ad45-f9f666d489d2 req-565cd695-3879-468b-a935-b2b5e1f4c350 service nova] Releasing lock "refresh_cache-629e177e-afbb-4765-a0b5-7a3f66f72ebf" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:18:34 user nova-compute[72351]: DEBUG nova.compute.manager [req-a638901f-7ca3-4cd4-ad45-f9f666d489d2 req-565cd695-3879-468b-a935-b2b5e1f4c350 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received event network-changed-2e9c8a9c-2d81-4b84-b481-9a6b513e730c {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:18:34 user nova-compute[72351]: DEBUG nova.compute.manager [req-a638901f-7ca3-4cd4-ad45-f9f666d489d2 req-565cd695-3879-468b-a935-b2b5e1f4c350 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Refreshing instance network info cache due to event network-changed-2e9c8a9c-2d81-4b84-b481-9a6b513e730c. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:18:34 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a638901f-7ca3-4cd4-ad45-f9f666d489d2 req-565cd695-3879-468b-a935-b2b5e1f4c350 service nova] Acquiring lock "refresh_cache-629e177e-afbb-4765-a0b5-7a3f66f72ebf" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:18:34 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a638901f-7ca3-4cd4-ad45-f9f666d489d2 req-565cd695-3879-468b-a935-b2b5e1f4c350 service nova] Acquired lock "refresh_cache-629e177e-afbb-4765-a0b5-7a3f66f72ebf" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:18:34 user nova-compute[72351]: DEBUG nova.network.neutron [req-a638901f-7ca3-4cd4-ad45-f9f666d489d2 req-565cd695-3879-468b-a935-b2b5e1f4c350 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Refreshing network info cache for port 2e9c8a9c-2d81-4b84-b481-9a6b513e730c {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:18:34 user nova-compute[72351]: DEBUG nova.network.neutron [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Successfully updated port: 5d0bb297-60b1-4e66-8f43-3bf1c32ae4ef {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:18:34 user nova-compute[72351]: DEBUG nova.network.neutron [req-a638901f-7ca3-4cd4-ad45-f9f666d489d2 req-565cd695-3879-468b-a935-b2b5e1f4c350 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:18:34 user nova-compute[72351]: DEBUG nova.network.neutron [req-a638901f-7ca3-4cd4-ad45-f9f666d489d2 req-565cd695-3879-468b-a935-b2b5e1f4c350 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:18:34 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a638901f-7ca3-4cd4-ad45-f9f666d489d2 req-565cd695-3879-468b-a935-b2b5e1f4c350 service nova] Releasing lock "refresh_cache-629e177e-afbb-4765-a0b5-7a3f66f72ebf" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:18:34 user nova-compute[72351]: DEBUG nova.network.neutron [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Successfully updated port: c2df74bf-22ce-48b1-9ab8-3a684cef7266 {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:18:35 user nova-compute[72351]: DEBUG nova.compute.manager [req-5779cfef-7ebc-4b66-a64c-138d296b7b0e req-7ac85f73-ebdf-45c8-a968-4051aa19b5f2 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received event network-changed-c2df74bf-22ce-48b1-9ab8-3a684cef7266 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:18:35 user nova-compute[72351]: DEBUG nova.compute.manager [req-5779cfef-7ebc-4b66-a64c-138d296b7b0e req-7ac85f73-ebdf-45c8-a968-4051aa19b5f2 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Refreshing instance network info cache due to event network-changed-c2df74bf-22ce-48b1-9ab8-3a684cef7266. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:18:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-5779cfef-7ebc-4b66-a64c-138d296b7b0e req-7ac85f73-ebdf-45c8-a968-4051aa19b5f2 service nova] Acquiring lock "refresh_cache-629e177e-afbb-4765-a0b5-7a3f66f72ebf" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:18:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-5779cfef-7ebc-4b66-a64c-138d296b7b0e req-7ac85f73-ebdf-45c8-a968-4051aa19b5f2 service nova] Acquired lock "refresh_cache-629e177e-afbb-4765-a0b5-7a3f66f72ebf" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:18:35 user nova-compute[72351]: DEBUG nova.network.neutron [req-5779cfef-7ebc-4b66-a64c-138d296b7b0e req-7ac85f73-ebdf-45c8-a968-4051aa19b5f2 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Refreshing network info cache for port c2df74bf-22ce-48b1-9ab8-3a684cef7266 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:18:35 user nova-compute[72351]: DEBUG nova.network.neutron [req-5779cfef-7ebc-4b66-a64c-138d296b7b0e req-7ac85f73-ebdf-45c8-a968-4051aa19b5f2 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:18:35 user nova-compute[72351]: DEBUG nova.network.neutron [req-5779cfef-7ebc-4b66-a64c-138d296b7b0e req-7ac85f73-ebdf-45c8-a968-4051aa19b5f2 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:18:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-5779cfef-7ebc-4b66-a64c-138d296b7b0e req-7ac85f73-ebdf-45c8-a968-4051aa19b5f2 service nova] Releasing lock "refresh_cache-629e177e-afbb-4765-a0b5-7a3f66f72ebf" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:18:35 user nova-compute[72351]: DEBUG nova.compute.manager [req-7f1a5d8f-3346-4bb3-adb9-fb70e989883d req-187b5823-6b19-433a-afc9-fbc28f8f7b50 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received event network-changed-5d0bb297-60b1-4e66-8f43-3bf1c32ae4ef {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:18:35 user nova-compute[72351]: DEBUG nova.compute.manager [req-7f1a5d8f-3346-4bb3-adb9-fb70e989883d req-187b5823-6b19-433a-afc9-fbc28f8f7b50 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Refreshing instance network info cache due to event network-changed-5d0bb297-60b1-4e66-8f43-3bf1c32ae4ef. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:18:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-7f1a5d8f-3346-4bb3-adb9-fb70e989883d req-187b5823-6b19-433a-afc9-fbc28f8f7b50 service nova] Acquiring lock "refresh_cache-629e177e-afbb-4765-a0b5-7a3f66f72ebf" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:18:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-7f1a5d8f-3346-4bb3-adb9-fb70e989883d req-187b5823-6b19-433a-afc9-fbc28f8f7b50 service nova] Acquired lock "refresh_cache-629e177e-afbb-4765-a0b5-7a3f66f72ebf" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:18:35 user nova-compute[72351]: DEBUG nova.network.neutron [req-7f1a5d8f-3346-4bb3-adb9-fb70e989883d req-187b5823-6b19-433a-afc9-fbc28f8f7b50 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Refreshing network info cache for port 5d0bb297-60b1-4e66-8f43-3bf1c32ae4ef {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:18:35 user nova-compute[72351]: DEBUG nova.network.neutron [req-7f1a5d8f-3346-4bb3-adb9-fb70e989883d req-187b5823-6b19-433a-afc9-fbc28f8f7b50 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:18:36 user nova-compute[72351]: DEBUG nova.network.neutron [req-7f1a5d8f-3346-4bb3-adb9-fb70e989883d req-187b5823-6b19-433a-afc9-fbc28f8f7b50 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:18:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-7f1a5d8f-3346-4bb3-adb9-fb70e989883d req-187b5823-6b19-433a-afc9-fbc28f8f7b50 service nova] Releasing lock "refresh_cache-629e177e-afbb-4765-a0b5-7a3f66f72ebf" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:18:36 user nova-compute[72351]: DEBUG nova.network.neutron [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Successfully updated port: d1b9314b-1800-41fb-a34b-0186f8e746da {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:18:37 user nova-compute[72351]: DEBUG nova.compute.manager [req-a6f2d6c5-b8ba-4247-a758-858468d56a59 req-1459a57d-2e79-4a45-8ead-7cc0e87552a0 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received event network-changed-d1b9314b-1800-41fb-a34b-0186f8e746da {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:18:37 user nova-compute[72351]: DEBUG nova.compute.manager [req-a6f2d6c5-b8ba-4247-a758-858468d56a59 req-1459a57d-2e79-4a45-8ead-7cc0e87552a0 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Refreshing instance network info cache due to event network-changed-d1b9314b-1800-41fb-a34b-0186f8e746da. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:18:37 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a6f2d6c5-b8ba-4247-a758-858468d56a59 req-1459a57d-2e79-4a45-8ead-7cc0e87552a0 service nova] Acquiring lock "refresh_cache-629e177e-afbb-4765-a0b5-7a3f66f72ebf" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:18:37 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a6f2d6c5-b8ba-4247-a758-858468d56a59 req-1459a57d-2e79-4a45-8ead-7cc0e87552a0 service nova] Acquired lock "refresh_cache-629e177e-afbb-4765-a0b5-7a3f66f72ebf" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:18:37 user nova-compute[72351]: DEBUG nova.network.neutron [req-a6f2d6c5-b8ba-4247-a758-858468d56a59 req-1459a57d-2e79-4a45-8ead-7cc0e87552a0 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Refreshing network info cache for port d1b9314b-1800-41fb-a34b-0186f8e746da {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:18:37 user nova-compute[72351]: DEBUG nova.network.neutron [req-a6f2d6c5-b8ba-4247-a758-858468d56a59 req-1459a57d-2e79-4a45-8ead-7cc0e87552a0 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:18:37 user nova-compute[72351]: DEBUG nova.network.neutron [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Successfully updated port: c7c195c0-af76-445e-8a18-8169f0cdf5fc {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:18:37 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:18:37 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Acquiring lock "refresh_cache-629e177e-afbb-4765-a0b5-7a3f66f72ebf" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:18:37 user nova-compute[72351]: DEBUG nova.network.neutron [req-a6f2d6c5-b8ba-4247-a758-858468d56a59 req-1459a57d-2e79-4a45-8ead-7cc0e87552a0 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:18:37 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a6f2d6c5-b8ba-4247-a758-858468d56a59 req-1459a57d-2e79-4a45-8ead-7cc0e87552a0 service nova] Releasing lock "refresh_cache-629e177e-afbb-4765-a0b5-7a3f66f72ebf" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:18:37 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Acquired lock "refresh_cache-629e177e-afbb-4765-a0b5-7a3f66f72ebf" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:18:37 user nova-compute[72351]: DEBUG nova.network.neutron [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:18:37 user nova-compute[72351]: DEBUG nova.network.neutron [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:18:38 user nova-compute[72351]: INFO nova.virt.block_device [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Booting with volume d1fa2bd8-1c39-4730-b60c-35dcbf24b18b at /dev/vdb
Aug 09 02:18:38 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:18:38 user nova-compute[72351]: DEBUG os_brick.utils [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:18:38 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:38 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.016s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:38 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[0deaaf3b-93a0-4733-9f8d-f66f6a1f2018]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:a52269602ff4\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:18:38 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:39 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[58968c1b-8f17-43f4-8089-408b97476947]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:18:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:39 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[2abba347-83b9-4eab-b59d-aa8747f64372]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:18:39 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[c9df02e2-1908-4572-98b7-757a9dcf5fc8]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:18:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Running cmd (subprocess): nvme version {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] 'nvme version' failed. Not Retrying. {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:18:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.nvmeof [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] nvme not present on system {{(pid=72351) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Aug 09 02:18:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:18:39 user nova-compute[72351]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Aug 09 02:18:39 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[33584a2d-7dbb-43c1-9034-cbcf0898bdb4]: (4, '') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:18:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72351) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Aug 09 02:18:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Aug 09 02:18:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] LIGHTOS: no hostnqn found. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Aug 09 02:18:39 user nova-compute[72351]: DEBUG os_brick.utils [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] <== get_connector_properties: return (136ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:a52269602ff4', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:18:39 user nova-compute[72351]: DEBUG nova.virt.block_device [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Updating existing volume attachment record: 3d9dd892-b059-4184-8e9a-93491f57bdeb {{(pid=72351) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Aug 09 02:18:39 user nova-compute[72351]: DEBUG nova.compute.manager [req-ab34e2a2-b3cf-45d3-a749-7a16dc9a9614 req-f34cc860-b942-49c5-a5fc-a621aa91c7b0 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received event network-changed-c7c195c0-af76-445e-8a18-8169f0cdf5fc {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:18:39 user nova-compute[72351]: DEBUG nova.compute.manager [req-ab34e2a2-b3cf-45d3-a749-7a16dc9a9614 req-f34cc860-b942-49c5-a5fc-a621aa91c7b0 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Refreshing instance network info cache due to event network-changed-c7c195c0-af76-445e-8a18-8169f0cdf5fc. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:18:39 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ab34e2a2-b3cf-45d3-a749-7a16dc9a9614 req-f34cc860-b942-49c5-a5fc-a621aa91c7b0 service nova] Acquiring lock "refresh_cache-629e177e-afbb-4765-a0b5-7a3f66f72ebf" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:18:40 user nova-compute[72351]: DEBUG nova.network.neutron [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Updating instance_info_cache with network_info: [{"id": "e021d67e-09ab-4779-b639-dff1ba39798a", "address": "fa:16:3e:60:29:2e", "network": {"id": "9d2c7895-3144-4087-8a69-40739a321574", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest-566420926-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape021d67e-09", "ovs_interfaceid": "e021d67e-09ab-4779-b639-dff1ba39798a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "f5ecb594-7ac3-496b-9bb0-9f908b91c8fb", "address": "fa:16:3e:30:31:a4", "network": {"id": "4e0c5236-b30d-4213-b1b4-26c0f1a636eb", "bridge": "br-int", "label": "tempest-device-tagging-net1-1640959147", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.122", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf5ecb594-7a", "ovs_interfaceid": "f5ecb594-7ac3-496b-9bb0-9f908b91c8fb", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "2e9c8a9c-2d81-4b84-b481-9a6b513e730c", "address": "fa:16:3e:39:9b:9d", "network": {"id": "4e0c5236-b30d-4213-b1b4-26c0f1a636eb", "bridge": "br-int", "label": "tempest-device-tagging-net1-1640959147", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.75", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2e9c8a9c-2d", "ovs_interfaceid": "2e9c8a9c-2d81-4b84-b481-9a6b513e730c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "5d0bb297-60b1-4e66-8f43-3bf1c32ae4ef", "address": "fa:16:3e:9b:17:d2", "network": {"id": "4e0c5236-b30d-4213-b1b4-26c0f1a636eb", "bridge": "br-int", "label": "tempest-device-tagging-net1-1640959147", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.192", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5d0bb297-60", "ovs_interfaceid": "5d0bb297-60b1-4e66-8f43-3bf1c32ae4ef", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "c2df74bf-22ce-48b1-9ab8-3a684cef7266", "address": "fa:16:3e:3f:be:14", "network": {"id": "4e0c5236-b30d-4213-b1b4-26c0f1a636eb", "bridge": "br-int", "label": "tempest-device-tagging-net1-1640959147", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.128", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc2df74bf-22", "ovs_interfaceid": "c2df74bf-22ce-48b1-9ab8-3a684cef7266", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "d1b9314b-1800-41fb-a34b-0186f8e746da", "address": "fa:16:3e:37:27:3b", "network": {"id": "b4ebdbb0-e697-4a42-93f5-946522a3dbce", "bridge": "br-int", "label": "tempest-device-tagging-net2-356335140", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd1b9314b-18", "ovs_interfaceid": "d1b9314b-1800-41fb-a34b-0186f8e746da", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "c7c195c0-af76-445e-8a18-8169f0cdf5fc", "address": "fa:16:3e:f7:2d:31", "network": {"id": "b4ebdbb0-e697-4a42-93f5-946522a3dbce", "bridge": "br-int", "label": "tempest-device-tagging-net2-356335140", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc7c195c0-af", "ovs_interfaceid": "c7c195c0-af76-445e-8a18-8169f0cdf5fc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:18:40 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Releasing lock "refresh_cache-629e177e-afbb-4765-a0b5-7a3f66f72ebf" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:18:40 user nova-compute[72351]: DEBUG nova.compute.manager [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Instance network_info: |[{"id": "e021d67e-09ab-4779-b639-dff1ba39798a", "address": "fa:16:3e:60:29:2e", "network": {"id": "9d2c7895-3144-4087-8a69-40739a321574", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest-566420926-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape021d67e-09", "ovs_interfaceid": "e021d67e-09ab-4779-b639-dff1ba39798a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "f5ecb594-7ac3-496b-9bb0-9f908b91c8fb", "address": "fa:16:3e:30:31:a4", "network": {"id": "4e0c5236-b30d-4213-b1b4-26c0f1a636eb", "bridge": "br-int", "label": "tempest-device-tagging-net1-1640959147", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.122", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf5ecb594-7a", "ovs_interfaceid": "f5ecb594-7ac3-496b-9bb0-9f908b91c8fb", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "2e9c8a9c-2d81-4b84-b481-9a6b513e730c", "address": "fa:16:3e:39:9b:9d", "network": {"id": "4e0c5236-b30d-4213-b1b4-26c0f1a636eb", "bridge": "br-int", "label": "tempest-device-tagging-net1-1640959147", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.75", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2e9c8a9c-2d", "ovs_interfaceid": "2e9c8a9c-2d81-4b84-b481-9a6b513e730c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "5d0bb297-60b1-4e66-8f43-3bf1c32ae4ef", "address": "fa:16:3e:9b:17:d2", "network": {"id": "4e0c5236-b30d-4213-b1b4-26c0f1a636eb", "bridge": "br-int", "label": "tempest-device-tagging-net1-1640959147", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.192", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5d0bb297-60", "ovs_interfaceid": "5d0bb297-60b1-4e66-8f43-3bf1c32ae4ef", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "c2df74bf-22ce-48b1-9ab8-3a684cef7266", "address": "fa:16:3e:3f:be:14", "network": {"id": "4e0c5236-b30d-4213-b1b4-26c0f1a636eb", "bridge": "br-int", "label": "tempest-device-tagging-net1-1640959147", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.128", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc2df74bf-22", "ovs_interfaceid": "c2df74bf-22ce-48b1-9ab8-3a684cef7266", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "d1b9314b-1800-41fb-a34b-0186f8e746da", "address": "fa:16:3e:37:27:3b", "network": {"id": "b4ebdbb0-e697-4a42-93f5-946522a3dbce", "bridge": "br-int", "label": "tempest-device-tagging-net2-356335140", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd1b9314b-18", "ovs_interfaceid": "d1b9314b-1800-41fb-a34b-0186f8e746da", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "c7c195c0-af76-445e-8a18-8169f0cdf5fc", "address": "fa:16:3e:f7:2d:31", "network": {"id": "b4ebdbb0-e697-4a42-93f5-946522a3dbce", "bridge": "br-int", "label": "tempest-device-tagging-net2-356335140", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc7c195c0-af", "ovs_interfaceid": "c7c195c0-af76-445e-8a18-8169f0cdf5fc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1971}}
Aug 09 02:18:40 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ab34e2a2-b3cf-45d3-a749-7a16dc9a9614 req-f34cc860-b942-49c5-a5fc-a621aa91c7b0 service nova] Acquired lock "refresh_cache-629e177e-afbb-4765-a0b5-7a3f66f72ebf" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:18:40 user nova-compute[72351]: DEBUG nova.network.neutron [req-ab34e2a2-b3cf-45d3-a749-7a16dc9a9614 req-f34cc860-b942-49c5-a5fc-a621aa91c7b0 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Refreshing network info cache for port c7c195c0-af76-445e-8a18-8169f0cdf5fc {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:18:41 user nova-compute[72351]: DEBUG nova.network.neutron [req-ab34e2a2-b3cf-45d3-a749-7a16dc9a9614 req-f34cc860-b942-49c5-a5fc-a621aa91c7b0 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Updated VIF entry in instance network info cache for port c7c195c0-af76-445e-8a18-8169f0cdf5fc. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:18:41 user nova-compute[72351]: DEBUG nova.network.neutron [req-ab34e2a2-b3cf-45d3-a749-7a16dc9a9614 req-f34cc860-b942-49c5-a5fc-a621aa91c7b0 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Updating instance_info_cache with network_info: [{"id": "e021d67e-09ab-4779-b639-dff1ba39798a", "address": "fa:16:3e:60:29:2e", "network": {"id": "9d2c7895-3144-4087-8a69-40739a321574", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest-566420926-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape021d67e-09", "ovs_interfaceid": "e021d67e-09ab-4779-b639-dff1ba39798a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "f5ecb594-7ac3-496b-9bb0-9f908b91c8fb", "address": "fa:16:3e:30:31:a4", "network": {"id": "4e0c5236-b30d-4213-b1b4-26c0f1a636eb", "bridge": "br-int", "label": "tempest-device-tagging-net1-1640959147", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.122", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf5ecb594-7a", "ovs_interfaceid": "f5ecb594-7ac3-496b-9bb0-9f908b91c8fb", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "2e9c8a9c-2d81-4b84-b481-9a6b513e730c", "address": "fa:16:3e:39:9b:9d", "network": {"id": "4e0c5236-b30d-4213-b1b4-26c0f1a636eb", "bridge": "br-int", "label": "tempest-device-tagging-net1-1640959147", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.75", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2e9c8a9c-2d", "ovs_interfaceid": "2e9c8a9c-2d81-4b84-b481-9a6b513e730c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "5d0bb297-60b1-4e66-8f43-3bf1c32ae4ef", "address": "fa:16:3e:9b:17:d2", "network": {"id": "4e0c5236-b30d-4213-b1b4-26c0f1a636eb", "bridge": "br-int", "label": "tempest-device-tagging-net1-1640959147", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.192", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5d0bb297-60", "ovs_interfaceid": "5d0bb297-60b1-4e66-8f43-3bf1c32ae4ef", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "c2df74bf-22ce-48b1-9ab8-3a684cef7266", "address": "fa:16:3e:3f:be:14", "network": {"id": "4e0c5236-b30d-4213-b1b4-26c0f1a636eb", "bridge": "br-int", "label": "tempest-device-tagging-net1-1640959147", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.128", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc2df74bf-22", "ovs_interfaceid": "c2df74bf-22ce-48b1-9ab8-3a684cef7266", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "d1b9314b-1800-41fb-a34b-0186f8e746da", "address": "fa:16:3e:37:27:3b", "network": {"id": "b4ebdbb0-e697-4a42-93f5-946522a3dbce", "bridge": "br-int", "label": "tempest-device-tagging-net2-356335140", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd1b9314b-18", "ovs_interfaceid": "d1b9314b-1800-41fb-a34b-0186f8e746da", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "c7c195c0-af76-445e-8a18-8169f0cdf5fc", "address": "fa:16:3e:f7:2d:31", "network": {"id": "b4ebdbb0-e697-4a42-93f5-946522a3dbce", "bridge": "br-int", "label": "tempest-device-tagging-net2-356335140", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc7c195c0-af", "ovs_interfaceid": "c7c195c0-af76-445e-8a18-8169f0cdf5fc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:18:41 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ab34e2a2-b3cf-45d3-a749-7a16dc9a9614 req-f34cc860-b942-49c5-a5fc-a621aa91c7b0 service nova] Releasing lock "refresh_cache-629e177e-afbb-4765-a0b5-7a3f66f72ebf" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:18:42 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:18:43 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquiring lock "a5ca6400-1125-431e-8a91-057e746d40b5" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:18:43 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "a5ca6400-1125-431e-8a91-057e746d40b5" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:18:43 user nova-compute[72351]: DEBUG nova.compute.manager [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Starting instance... {{(pid=72351) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2406}}
Aug 09 02:18:43 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:18:43 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:18:43 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:18:43 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72351) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Aug 09 02:18:43 user nova-compute[72351]: INFO nova.compute.claims [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Claim successful on node user
Aug 09 02:18:44 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:18:44 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:18:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.507s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:18:44 user nova-compute[72351]: DEBUG nova.compute.manager [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Start building networks asynchronously for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2803}}
Aug 09 02:18:44 user nova-compute[72351]: DEBUG nova.compute.manager [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Allocating IP information in the background. {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1956}}
Aug 09 02:18:44 user nova-compute[72351]: DEBUG nova.network.neutron [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] allocate_for_instance() {{(pid=72351) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Aug 09 02:18:44 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Aug 09 02:18:44 user nova-compute[72351]: DEBUG nova.compute.manager [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Start building block device mappings for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2838}}
Aug 09 02:18:44 user nova-compute[72351]: INFO nova.virt.block_device [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Booting with volume snapshot 80da46a3-d57e-45ee-9585-0cce55b44a09 at /dev/vda
Aug 09 02:18:44 user nova-compute[72351]: DEBUG nova.policy [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '07d8d5a634274df19c2db769d2d27b29', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4c6bbfd8d904ba1ac69ba5da4564513', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72351) authorize /opt/stack/nova/nova/policy.py:203}}
Aug 09 02:18:45 user nova-compute[72351]: DEBUG nova.network.neutron [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Successfully created port: 49802a57-3939-4cc2-956b-3d396b7c738e {{(pid=72351) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Aug 09 02:18:47 user nova-compute[72351]: DEBUG nova.network.neutron [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Successfully updated port: 49802a57-3939-4cc2-956b-3d396b7c738e {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:18:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquiring lock "refresh_cache-a5ca6400-1125-431e-8a91-057e746d40b5" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:18:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquired lock "refresh_cache-a5ca6400-1125-431e-8a91-057e746d40b5" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:18:47 user nova-compute[72351]: DEBUG nova.network.neutron [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:18:47 user nova-compute[72351]: DEBUG nova.compute.manager [req-0a4b4a41-8546-43fd-a62e-c56c0dd86106 req-4dcfba7e-f074-4c0f-a8dc-318801504efd service nova] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Received event network-changed-49802a57-3939-4cc2-956b-3d396b7c738e {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:18:47 user nova-compute[72351]: DEBUG nova.compute.manager [req-0a4b4a41-8546-43fd-a62e-c56c0dd86106 req-4dcfba7e-f074-4c0f-a8dc-318801504efd service nova] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Refreshing instance network info cache due to event network-changed-49802a57-3939-4cc2-956b-3d396b7c738e. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:18:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-0a4b4a41-8546-43fd-a62e-c56c0dd86106 req-4dcfba7e-f074-4c0f-a8dc-318801504efd service nova] Acquiring lock "refresh_cache-a5ca6400-1125-431e-8a91-057e746d40b5" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:18:47 user nova-compute[72351]: DEBUG nova.network.neutron [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:18:47 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:18:47 user nova-compute[72351]: DEBUG nova.network.neutron [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Updating instance_info_cache with network_info: [{"id": "49802a57-3939-4cc2-956b-3d396b7c738e", "address": "fa:16:3e:9e:eb:c0", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap49802a57-39", "ovs_interfaceid": "49802a57-3939-4cc2-956b-3d396b7c738e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:18:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Releasing lock "refresh_cache-a5ca6400-1125-431e-8a91-057e746d40b5" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:18:47 user nova-compute[72351]: DEBUG nova.compute.manager [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Instance network_info: |[{"id": "49802a57-3939-4cc2-956b-3d396b7c738e", "address": "fa:16:3e:9e:eb:c0", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap49802a57-39", "ovs_interfaceid": "49802a57-3939-4cc2-956b-3d396b7c738e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1971}}
Aug 09 02:18:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-0a4b4a41-8546-43fd-a62e-c56c0dd86106 req-4dcfba7e-f074-4c0f-a8dc-318801504efd service nova] Acquired lock "refresh_cache-a5ca6400-1125-431e-8a91-057e746d40b5" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:18:47 user nova-compute[72351]: DEBUG nova.network.neutron [req-0a4b4a41-8546-43fd-a62e-c56c0dd86106 req-4dcfba7e-f074-4c0f-a8dc-318801504efd service nova] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Refreshing network info cache for port 49802a57-3939-4cc2-956b-3d396b7c738e {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:18:48 user nova-compute[72351]: INFO nova.virt.block_device [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Booting with volume a3218523-2c3f-4787-b0ea-f589160d940c at /dev/vdc
Aug 09 02:18:48 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:18:48 user nova-compute[72351]: DEBUG os_brick.utils [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:18:48 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:48 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:48 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[46fae104-9dbc-414e-a097-cbaf998dccc7]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:a52269602ff4\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:18:48 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:48 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:48 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[7169249e-83e7-4d52-a3e0-7c108ad18177]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:18:48 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:48 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:48 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[8c7c8dd1-571f-4fc8-974e-dd008868f2a8]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:18:48 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e54ac553-a718-433e-be25-fe711f8c60cd]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:18:48 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Running cmd (subprocess): nvme version {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:48 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] 'nvme version' failed. Not Retrying. {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:18:48 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.nvmeof [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] nvme not present on system {{(pid=72351) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Aug 09 02:18:48 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:48 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:18:48 user nova-compute[72351]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Aug 09 02:18:48 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[c6efc0a9-0aba-490c-81ba-4fa9abea6684]: (4, '') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:18:48 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72351) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Aug 09 02:18:48 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Aug 09 02:18:48 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] LIGHTOS: no hostnqn found. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Aug 09 02:18:48 user nova-compute[72351]: DEBUG os_brick.utils [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] <== get_connector_properties: return (120ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:a52269602ff4', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:18:48 user nova-compute[72351]: DEBUG nova.virt.block_device [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Updating existing volume attachment record: 94cedf2c-53d3-49bd-9cac-f79dbb38df48 {{(pid=72351) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Aug 09 02:18:48 user nova-compute[72351]: DEBUG nova.network.neutron [req-0a4b4a41-8546-43fd-a62e-c56c0dd86106 req-4dcfba7e-f074-4c0f-a8dc-318801504efd service nova] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Updated VIF entry in instance network info cache for port 49802a57-3939-4cc2-956b-3d396b7c738e. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:18:48 user nova-compute[72351]: DEBUG nova.network.neutron [req-0a4b4a41-8546-43fd-a62e-c56c0dd86106 req-4dcfba7e-f074-4c0f-a8dc-318801504efd service nova] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Updating instance_info_cache with network_info: [{"id": "49802a57-3939-4cc2-956b-3d396b7c738e", "address": "fa:16:3e:9e:eb:c0", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap49802a57-39", "ovs_interfaceid": "49802a57-3939-4cc2-956b-3d396b7c738e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:18:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-0a4b4a41-8546-43fd-a62e-c56c0dd86106 req-4dcfba7e-f074-4c0f-a8dc-318801504efd service nova] Releasing lock "refresh_cache-a5ca6400-1125-431e-8a91-057e746d40b5" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:18:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b9022cce-9987-48ad-80df-a841c13b264b tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Acquiring lock "f45b57fa-9264-45f0-9573-e5fb58848d1f" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:18:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b9022cce-9987-48ad-80df-a841c13b264b tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Lock "f45b57fa-9264-45f0-9573-e5fb58848d1f" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:18:48 user nova-compute[72351]: DEBUG nova.objects.instance [None req-b9022cce-9987-48ad-80df-a841c13b264b tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Lazy-loading 'flavor' on Instance uuid f45b57fa-9264-45f0-9573-e5fb58848d1f {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:18:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b9022cce-9987-48ad-80df-a841c13b264b tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Lock "f45b57fa-9264-45f0-9573-e5fb58848d1f" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.083s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:18:48 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:18:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b9022cce-9987-48ad-80df-a841c13b264b tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Acquiring lock "f45b57fa-9264-45f0-9573-e5fb58848d1f" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:18:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b9022cce-9987-48ad-80df-a841c13b264b tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Lock "f45b57fa-9264-45f0-9573-e5fb58848d1f" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:18:48 user nova-compute[72351]: INFO nova.compute.manager [None req-b9022cce-9987-48ad-80df-a841c13b264b tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Attaching volume a266475f-f65f-4f28-bc8b-724378d46b7d to /dev/vdb
Aug 09 02:18:49 user nova-compute[72351]: DEBUG os_brick.utils [None req-b9022cce-9987-48ad-80df-a841c13b264b tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:18:49 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:49 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:49 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[826374f0-05da-4c3b-960d-e8fa82699eee]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:a52269602ff4\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:18:49 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:49 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.015s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:49 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ae8c8a83-777f-4a37-9a8a-8452089916c6]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:18:49 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:49 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.012s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:49 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[f1493233-2db3-412d-b907-1aec00325f27]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:18:49 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[c8da4ac3-81fe-48ce-9b34-e2727c3799bb]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:18:49 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-b9022cce-9987-48ad-80df-a841c13b264b tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Running cmd (subprocess): nvme version {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:49 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-b9022cce-9987-48ad-80df-a841c13b264b tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] 'nvme version' failed. Not Retrying. {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:18:49 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.nvmeof [None req-b9022cce-9987-48ad-80df-a841c13b264b tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] nvme not present on system {{(pid=72351) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Aug 09 02:18:49 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:49 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:18:49 user nova-compute[72351]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Aug 09 02:18:49 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a5d9a159-4de0-4074-a0f4-75da88b94c09]: (4, '') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:18:49 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-b9022cce-9987-48ad-80df-a841c13b264b tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72351) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Aug 09 02:18:49 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-b9022cce-9987-48ad-80df-a841c13b264b tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Aug 09 02:18:49 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-b9022cce-9987-48ad-80df-a841c13b264b tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] LIGHTOS: no hostnqn found. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Aug 09 02:18:49 user nova-compute[72351]: DEBUG os_brick.utils [None req-b9022cce-9987-48ad-80df-a841c13b264b tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] <== get_connector_properties: return (136ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:a52269602ff4', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:18:49 user nova-compute[72351]: DEBUG nova.virt.block_device [None req-b9022cce-9987-48ad-80df-a841c13b264b tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Updating existing volume attachment record: 639d89ed-963d-46c7-b569-aecbd409716f {{(pid=72351) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Aug 09 02:18:49 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:18:50 user nova-compute[72351]: DEBUG os_brick.utils [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:18:50 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:50 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:50 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[b94504a2-43f7-4409-ab7a-f8682b3d68bd]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:a52269602ff4\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:18:50 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:50 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.014s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:50 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[296ba4d3-78f4-4e80-a8cb-11186048d982]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:18:50 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:50 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:50 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[f6a90121-fb9b-4d88-b944-efe065282a86]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:18:50 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[62a670d5-e4a8-4f1d-ab88-b3df0baf2e0e]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:18:50 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Running cmd (subprocess): nvme version {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:50 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] 'nvme version' failed. Not Retrying. {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:18:50 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.nvmeof [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] nvme not present on system {{(pid=72351) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Aug 09 02:18:50 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:50 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:18:50 user nova-compute[72351]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Aug 09 02:18:50 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[248c5620-9ce9-4b0c-a250-6e4120bf7496]: (4, '') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:18:50 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72351) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Aug 09 02:18:50 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Aug 09 02:18:50 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] LIGHTOS: no hostnqn found. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Aug 09 02:18:50 user nova-compute[72351]: DEBUG os_brick.utils [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] <== get_connector_properties: return (117ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:a52269602ff4', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:18:50 user nova-compute[72351]: DEBUG nova.virt.block_device [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Updating existing volume attachment record: 8a76321a-2bee-4927-99c8-8a0d76a08cdc {{(pid=72351) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Aug 09 02:18:50 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:18:50 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:18:50 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72351) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10461}}
Aug 09 02:18:50 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:18:50 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:18:50 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:18:50 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:18:50 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Auditing locally available compute resources for user (node: user) {{(pid=72351) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Aug 09 02:18:51 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/50302d76-f6b9-4fe3-b55a-4a01e483e2e3/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:51 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/50302d76-f6b9-4fe3-b55a-4a01e483e2e3/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:51 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/50302d76-f6b9-4fe3-b55a-4a01e483e2e3/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:51 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/50302d76-f6b9-4fe3-b55a-4a01e483e2e3/disk --force-share --output=json" returned: 0 in 0.128s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:51 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d0c7002c-8a3a-4bc5-940c-63cf4264668b/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:51 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d0c7002c-8a3a-4bc5-940c-63cf4264668b/disk --force-share --output=json" returned: 0 in 0.132s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:51 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d0c7002c-8a3a-4bc5-940c-63cf4264668b/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:51 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d0c7002c-8a3a-4bc5-940c-63cf4264668b/disk --force-share --output=json" returned: 0 in 0.148s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:51 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7c5e633d-a32f-4c5e-bb9d-1e40da964bd0/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:51 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7c5e633d-a32f-4c5e-bb9d-1e40da964bd0/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:51 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7c5e633d-a32f-4c5e-bb9d-1e40da964bd0/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:51 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7c5e633d-a32f-4c5e-bb9d-1e40da964bd0/disk --force-share --output=json" returned: 0 in 0.128s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:51 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b/disk --force-share --output=json" returned: 0 in 0.269s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b/disk --force-share --output=json" returned: 0 in 0.139s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:52 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:18:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2/disk --force-share --output=json" returned: 0 in 0.151s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2/disk --force-share --output=json" returned: 0 in 0.130s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f45b57fa-9264-45f0-9573-e5fb58848d1f/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f45b57fa-9264-45f0-9573-e5fb58848d1f/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f45b57fa-9264-45f0-9573-e5fb58848d1f/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f45b57fa-9264-45f0-9573-e5fb58848d1f/disk --force-share --output=json" returned: 0 in 0.180s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/74728ca8-34d4-45e6-a1f2-21c0952ff835/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/74728ca8-34d4-45e6-a1f2-21c0952ff835/disk --force-share --output=json" returned: 0 in 0.174s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/74728ca8-34d4-45e6-a1f2-21c0952ff835/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/74728ca8-34d4-45e6-a1f2-21c0952ff835/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/97b13ccd-15b4-44bf-b865-71b4da861168/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/97b13ccd-15b4-44bf-b865-71b4da861168/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/97b13ccd-15b4-44bf-b865-71b4da861168/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/97b13ccd-15b4-44bf-b865-71b4da861168/disk --force-share --output=json" returned: 0 in 0.128s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:18:54 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:18:54 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:18:54 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Hypervisor/Node resource view: name=user free_ram=7203MB free_disk=25.637577056884766GB free_vcpus=4 pci_devices=[{"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72351) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Aug 09 02:18:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:18:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:18:54 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance f45b57fa-9264-45f0-9573-e5fb58848d1f actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:18:54 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 2d78475c-107d-4256-b12e-dbe9c8f3b9b2 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:18:54 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 50302d76-f6b9-4fe3-b55a-4a01e483e2e3 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:18:54 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 74728ca8-34d4-45e6-a1f2-21c0952ff835 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:18:54 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:18:54 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:18:54 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance d0c7002c-8a3a-4bc5-940c-63cf4264668b actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:18:54 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 97b13ccd-15b4-44bf-b865-71b4da861168 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:18:54 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 629e177e-afbb-4765-a0b5-7a3f66f72ebf actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:18:54 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance a5ca6400-1125-431e-8a91-057e746d40b5 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:18:54 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Total usable vcpus: 12, total allocated vcpus: 10 {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Aug 09 02:18:54 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Final resource view: name=user phys_ram=16011MB used_ram=5632MB phys_disk=40GB used_disk=8GB total_vcpus=12 used_vcpus=10 pci_stats=[] {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Aug 09 02:18:54 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:18:54 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:18:54 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Compute_service record updated for user:user {{(pid=72351) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Aug 09 02:18:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.494s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:18:55 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:18:55 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:18:55 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:18:55 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:18:55 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Starting heal instance info cache {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9842}}
Aug 09 02:18:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "refresh_cache-2d78475c-107d-4256-b12e-dbe9c8f3b9b2" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:18:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquired lock "refresh_cache-2d78475c-107d-4256-b12e-dbe9c8f3b9b2" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:18:55 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Forcefully refreshing network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1996}}
Aug 09 02:18:56 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Updating instance_info_cache with network_info: [{"id": "ff25eb15-1af0-4110-9d9a-ecd0a1d4627d", "address": "fa:16:3e:0a:89:6a", "network": {"id": "fd1fe4e0-5d09-43c1-8dbc-1f60cb7f38d7", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1014783449-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "747d5750a2174aaea71849c70167a816", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapff25eb15-1a", "ovs_interfaceid": "ff25eb15-1af0-4110-9d9a-ecd0a1d4627d", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:18:56 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Releasing lock "refresh_cache-2d78475c-107d-4256-b12e-dbe9c8f3b9b2" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:18:56 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Updated the network info_cache for instance {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9913}}
Aug 09 02:18:56 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:18:56 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:18:56 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:18:56 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:18:56 user nova-compute[72351]: DEBUG nova.compute.manager [req-a0886e7e-0aab-4a73-8127-1a35d5dbbb63 req-0dd62909-330a-4a3b-acd1-4eaa115e33d0 service nova] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Received event network-changed-ff25eb15-1af0-4110-9d9a-ecd0a1d4627d {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:18:56 user nova-compute[72351]: DEBUG nova.compute.manager [req-a0886e7e-0aab-4a73-8127-1a35d5dbbb63 req-0dd62909-330a-4a3b-acd1-4eaa115e33d0 service nova] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Refreshing instance network info cache due to event network-changed-ff25eb15-1af0-4110-9d9a-ecd0a1d4627d. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:18:56 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a0886e7e-0aab-4a73-8127-1a35d5dbbb63 req-0dd62909-330a-4a3b-acd1-4eaa115e33d0 service nova] Acquiring lock "refresh_cache-2d78475c-107d-4256-b12e-dbe9c8f3b9b2" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:18:56 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a0886e7e-0aab-4a73-8127-1a35d5dbbb63 req-0dd62909-330a-4a3b-acd1-4eaa115e33d0 service nova] Acquired lock "refresh_cache-2d78475c-107d-4256-b12e-dbe9c8f3b9b2" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:18:56 user nova-compute[72351]: DEBUG nova.network.neutron [req-a0886e7e-0aab-4a73-8127-1a35d5dbbb63 req-0dd62909-330a-4a3b-acd1-4eaa115e33d0 service nova] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Refreshing network info cache for port ff25eb15-1af0-4110-9d9a-ecd0a1d4627d {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:18:57 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:18:57 user nova-compute[72351]: DEBUG nova.compute.manager [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Start spawning the instance on the hypervisor. {{(pid=72351) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2612}}
Aug 09 02:18:57 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Creating instance directory {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4717}}
Aug 09 02:18:57 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Creating image(s)
Aug 09 02:18:57 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Did not create local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4853}}
Aug 09 02:18:57 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Ensure instance console log exists: /opt/stack/data/nova/instances/629e177e-afbb-4765-a0b5-7a3f66f72ebf/console.log {{(pid=72351) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4603}}
Aug 09 02:18:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:18:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:18:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:18:57 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Start _get_guest_xml network_info=[{"id": "e021d67e-09ab-4779-b639-dff1ba39798a", "address": "fa:16:3e:60:29:2e", "network": {"id": "9d2c7895-3144-4087-8a69-40739a321574", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest-566420926-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape021d67e-09", "ovs_interfaceid": "e021d67e-09ab-4779-b639-dff1ba39798a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "f5ecb594-7ac3-496b-9bb0-9f908b91c8fb", "address": "fa:16:3e:30:31:a4", "network": {"id": "4e0c5236-b30d-4213-b1b4-26c0f1a636eb", "bridge": "br-int", "label": "tempest-device-tagging-net1-1640959147", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.122", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf5ecb594-7a", "ovs_interfaceid": "f5ecb594-7ac3-496b-9bb0-9f908b91c8fb", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "2e9c8a9c-2d81-4b84-b481-9a6b513e730c", "address": "fa:16:3e:39:9b:9d", "network": {"id": "4e0c5236-b30d-4213-b1b4-26c0f1a636eb", "bridge": "br-int", "label": "tempest-device-tagging-net1-1640959147", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.75", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2e9c8a9c-2d", "ovs_interfaceid": "2e9c8a9c-2d81-4b84-b481-9a6b513e730c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "5d0bb297-60b1-4e66-8f43-3bf1c32ae4ef", "address": "fa:16:3e:9b:17:d2", "network": {"id": "4e0c5236-b30d-4213-b1b4-26c0f1a636eb", "bridge": "br-int", "label": "tempest-device-tagging-net1-1640959147", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.192", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5d0bb297-60", "ovs_interfaceid": "5d0bb297-60b1-4e66-8f43-3bf1c32ae4ef", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "c2df74bf-22ce-48b1-9ab8-3a684cef7266", "address": "fa:16:3e:3f:be:14", "network": {"id": "4e0c5236-b30d-4213-b1b4-26c0f1a636eb", "bridge": "br-int", "label": "tempest-device-tagging-net1-1640959147", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.128", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc2df74bf-22", "ovs_interfaceid": "c2df74bf-22ce-48b1-9ab8-3a684cef7266", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "d1b9314b-1800-41fb-a34b-0186f8e746da", "address": "fa:16:3e:37:27:3b", "network": {"id": "b4ebdbb0-e697-4a42-93f5-946522a3dbce", "bridge": "br-int", "label": "tempest-device-tagging-net2-356335140", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd1b9314b-18", "ovs_interfaceid": "d1b9314b-1800-41fb-a34b-0186f8e746da", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "c7c195c0-af76-445e-8a18-8169f0cdf5fc", "address": "fa:16:3e:f7:2d:31", "network": {"id": "b4ebdbb0-e697-4a42-93f5-946522a3dbce", "bridge": "br-int", "label": "tempest-device-tagging-net2-356335140", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc7c195c0-af", "ovs_interfaceid": "c7c195c0-af76-445e-8a18-8169f0cdf5fc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vdb': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk', 'boot_index': '2'}, '/dev/vdc': {'bus': 'virtio', 'dev': 'vdc', 'type': 'disk', 'boot_index': '3'}, 'disk.config': {'bus': 'ide', 'dev': 'hda', 'type': 'cdrom'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'device_type': 'disk', 'guest_format': None, 'disk_bus': 'virtio', 'mount_device': '/dev/vda', 'boot_index': 0, 'attachment_id': '1176474a-644d-471f-8922-f986241b8f62', 'delete_on_termination': False, 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqns': ['iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'FftGaAqL', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '629e177e-afbb-4765-a0b5-7a3f66f72ebf', 'attached_at': '', 'detached_at': '', 'volume_id': 'a8ce86ce-79f7-4138-a49b-c45c9db38c97', 'serial': 'a8ce86ce-79f7-4138-a49b-c45c9db38c97'}, 'volume_type': None}, {'device_type': 'disk', 'guest_format': None, 'disk_bus': 'virtio', 'mount_device': '/dev/vdb', 'boot_index': 1, 'attachment_id': '3d9dd892-b059-4184-8e9a-93491f57bdeb', 'delete_on_termination': False, 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqns': ['iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'pynsiAWj', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '629e177e-afbb-4765-a0b5-7a3f66f72ebf', 'attached_at': '', 'detached_at': '', 'volume_id': 'd1fa2bd8-1c39-4730-b60c-35dcbf24b18b', 'serial': 'd1fa2bd8-1c39-4730-b60c-35dcbf24b18b'}, 'volume_type': None}, {'device_type': 'disk', 'guest_format': None, 'disk_bus': 'virtio', 'mount_device': '/dev/vdc', 'boot_index': 2, 'attachment_id': '94cedf2c-53d3-49bd-9cac-f79dbb38df48', 'delete_on_termination': False, 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqns': ['iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'nLZUTgbV', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '629e177e-afbb-4765-a0b5-7a3f66f72ebf', 'attached_at': '', 'detached_at': '', 'volume_id': 'a3218523-2c3f-4787-b0ea-f589160d940c', 'serial': 'a3218523-2c3f-4787-b0ea-f589160d940c'}, 'volume_type': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:18:57 user nova-compute[72351]: DEBUG nova.network.neutron [req-a0886e7e-0aab-4a73-8127-1a35d5dbbb63 req-0dd62909-330a-4a3b-acd1-4eaa115e33d0 service nova] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Updated VIF entry in instance network info cache for port ff25eb15-1af0-4110-9d9a-ecd0a1d4627d. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:18:57 user nova-compute[72351]: DEBUG nova.network.neutron [req-a0886e7e-0aab-4a73-8127-1a35d5dbbb63 req-0dd62909-330a-4a3b-acd1-4eaa115e33d0 service nova] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Updating instance_info_cache with network_info: [{"id": "ff25eb15-1af0-4110-9d9a-ecd0a1d4627d", "address": "fa:16:3e:0a:89:6a", "network": {"id": "fd1fe4e0-5d09-43c1-8dbc-1f60cb7f38d7", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1014783449-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.229", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "747d5750a2174aaea71849c70167a816", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapff25eb15-1a", "ovs_interfaceid": "ff25eb15-1af0-4110-9d9a-ecd0a1d4627d", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:18:57 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:18:57 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:18:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a0886e7e-0aab-4a73-8127-1a35d5dbbb63 req-0dd62909-330a-4a3b-acd1-4eaa115e33d0 service nova] Releasing lock "refresh_cache-2d78475c-107d-4256-b12e-dbe9c8f3b9b2" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:18:57 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:18:57 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:18:57 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:18:57 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:18:57 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:18:57 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:18:57 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:18:57 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:18:57 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:18:57 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:18:57 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:18:57 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:18:57 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:18:57 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:18:57 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:18:57 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:18:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Acquiring lock "cache_volume_driver" by "nova.virt.libvirt.driver.LibvirtDriver._get_volume_driver.<locals>._cache_volume_driver" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:18:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Lock "cache_volume_driver" acquired by "nova.virt.libvirt.driver.LibvirtDriver._get_volume_driver.<locals>._cache_volume_driver" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:18:57 user nova-compute[72351]: DEBUG os_brick.initiator.connector [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Factory for ISCSI on None {{(pid=72351) factory /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connector.py:281}}
Aug 09 02:18:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Lock "cache_volume_driver" "released" by "nova.virt.libvirt.driver.LibvirtDriver._get_volume_driver.<locals>._cache_volume_driver" :: held 0.005s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:18:57 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 02:18:57 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'FftGaAqL', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:18:57 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:18:57 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:18:57 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:18:57 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:18:57 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:18:57 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:18:57 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 02:18:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 -p 172.16.0.220:3260" returned: 21 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:57 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[c62b5214-f9e5-4b80-80dd-f48b62243f5b]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:18:57 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Aug 09 02:18:57 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:18:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 -p 172.16.0.220:3260 --interface default --op new {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:57 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ee991dd0-1486-4433-8b45-aa2042cd759b]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97] added\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:18:57 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97] added
Aug 09 02:18:57 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:18:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.016s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:57 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[bbe25cef-6ed6-451f-9fe3-41cca4af6d6c]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:18:57 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:18:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:57 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[8f2e8c90-d836-43fb-b79c-636de1807b77]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:18:57 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:18:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v FftGaAqL {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v FftGaAqL" returned: 0 in 0.025s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:57 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[7d063a4b-0fc1-44c3-a540-acc629691255]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:18:57 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'FftGaAqL'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:18:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.027s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:57 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[9db3c2e7-1670-4f16-bf7f-757ec5cfb1b8]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:18:57 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:18:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 21 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:58 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[52340e76-c910-44c2-b470-5a35d2db26db]: (4, ('', 'iscsiadm: No active sessions.\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:18:58 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] iscsiadm ('-m', 'session'): stdout= stderr=iscsiadm: No active sessions.
Aug 09 02:18:58 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:18:58 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] iscsi session list stdout= stderr=iscsiadm: No active sessions.
Aug 09 02:18:58 user nova-compute[72351]:  {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:18:58 user nova-compute[72351]: WARNING os_brick.initiator.connectors.iscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] iscsiadm stderr output when getting sessions: iscsiadm: No active sessions.
Aug 09 02:18:58 user nova-compute[72351]: 
Aug 09 02:18:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 -p 172.16.0.220:3260 --login {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 -p 172.16.0.220:3260 --login" returned: 0 in 0.049s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:58 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[549f8224-556d-48e1-9e5c-89b3901c8431]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:18:58 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97, portal: 172.16.0.220,3260]
Aug 09 02:18:58 user nova-compute[72351]: Login to [iface: default, target: iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97, portal: 172.16.0.220,3260] successful.
Aug 09 02:18:58 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:18:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:58 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[fa524838-7332-4fe8-a472-60739fdc4caf]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:18:58 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:18:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:58 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[10522af8-1c0c-405b-b1c9-10c4d422eca5]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:18:58 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:18:58 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:18:58 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:18:58 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:18:58 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.280s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:18:58 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 02:18:58 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] HCTL ('33', '-', '-', 0) found on session 2 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 02:18:58 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 02:18:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.025s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:58 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e1159aaf-214d-4f04-901d-31ce3f5df3ce]: (4, ('- - 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:18:58 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Searching for a device in session 2 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:18:58 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Connected to sdb using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'FftGaAqL', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 02:18:58 user nova-compute[72351]: DEBUG nova.compute.manager [None req-e752ebac-1332-42c7-87ba-2597e9decaf2 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:18:58 user nova-compute[72351]: INFO nova.compute.manager [None req-e752ebac-1332-42c7-87ba-2597e9decaf2 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] instance snapshotting
Aug 09 02:18:58 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:18:58 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-e752ebac-1332-42c7-87ba-2597e9decaf2 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Beginning live snapshot process
Aug 09 02:18:58 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b9022cce-9987-48ad-80df-a841c13b264b tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 02:18:58 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-b9022cce-9987-48ad-80df-a841c13b264b tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'tWHOBIpS', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:18:58 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-b9022cce-9987-48ad-80df-a841c13b264b tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:18:58 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-b9022cce-9987-48ad-80df-a841c13b264b tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:18:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-e752ebac-1332-42c7-87ba-2597e9decaf2 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2/disk --force-share --output=json -f qcow2 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-e752ebac-1332-42c7-87ba-2597e9decaf2 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2/disk --force-share --output=json -f qcow2" returned: 0 in 0.185s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-e752ebac-1332-42c7-87ba-2597e9decaf2 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2/disk --force-share --output=json -f qcow2 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-e752ebac-1332-42c7-87ba-2597e9decaf2 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2/disk --force-share --output=json -f qcow2" returned: 0 in 0.142s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-e752ebac-1332-42c7-87ba-2597e9decaf2 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-e752ebac-1332-42c7-87ba-2597e9decaf2 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.145s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-e752ebac-1332-42c7-87ba-2597e9decaf2 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/snapshots/tmp6vwrf042/fa32ae5053d94b79b570d2b999672e90.delta 1073741824 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.354s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] <== connect_volume: return (1357ms) {'type': 'block', 'scsi_wwn': '26439656138373730', 'path': '/dev/sdb'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26439656138373730', 'path': '/dev/sdb'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-b9022cce-9987-48ad-80df-a841c13b264b tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.711s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:18:59 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-b9022cce-9987-48ad-80df-a841c13b264b tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:18:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-b9022cce-9987-48ad-80df-a841c13b264b tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-b9022cce-9987-48ad-80df-a841c13b264b tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:18:59 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-b9022cce-9987-48ad-80df-a841c13b264b tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 02:18:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-e752ebac-1332-42c7-87ba-2597e9decaf2 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/snapshots/tmp6vwrf042/fa32ae5053d94b79b570d2b999672e90.delta 1073741824" returned: 0 in 0.062s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:59 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-e752ebac-1332-42c7-87ba-2597e9decaf2 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Quiescing instance not available: QEMU guest agent is not enabled.
Aug 09 02:18:59 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-e752ebac-1332-42c7-87ba-2597e9decaf2 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Checking UEFI support for host arch (x86_64) {{(pid=72351) supports_uefi /opt/stack/nova/nova/virt/libvirt/host.py:1751}}
Aug 09 02:18:59 user nova-compute[72351]: INFO nova.virt.libvirt.host [None req-e752ebac-1332-42c7-87ba-2597e9decaf2 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] UEFI support detected
Aug 09 02:18:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d -p 172.16.0.220:3260" returned: 21 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[755c558d-6386-4306-aca7-c06ade697c53]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-b9022cce-9987-48ad-80df-a841c13b264b tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Aug 09 02:18:59 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d -p 172.16.0.220:3260 --interface default --op new {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'pynsiAWj', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:18:59 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:18:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[5c19b8c7-b5f0-49a8-b303-5e5914d6e067]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d] added\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-b9022cce-9987-48ad-80df-a841c13b264b tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d] added
Aug 09 02:18:59 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[0f374760-9b40-4b4b-8458-1d7d3cd1f18f]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-b9022cce-9987-48ad-80df-a841c13b264b tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[309d5a11-9e49-42d1-b84b-a40d37fd5d8c]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-b9022cce-9987-48ad-80df-a841c13b264b tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d -p 172.16.0.220:3260 --op update -n node.session.auth.username -v tWHOBIpS {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d -p 172.16.0.220:3260 --op update -n node.session.auth.username -v tWHOBIpS" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[b91a6ff9-a54b-4693-ac3f-1e3761ac46a7]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-b9022cce-9987-48ad-80df-a841c13b264b tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'tWHOBIpS'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.024s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[94e960de-e2ed-4419-9dc3-549edaff78a4]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-b9022cce-9987-48ad-80df-a841c13b264b tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e24643dc-dbf0-4a01-958b-bde1d5b6d78c]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-b9022cce-9987-48ad-80df-a841c13b264b tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:18:59 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-b9022cce-9987-48ad-80df-a841c13b264b tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:18:59 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d -p 172.16.0.220:3260 --login {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d -p 172.16.0.220:3260 --login" returned: 0 in 0.054s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[2c0f1833-d0e6-42f5-8a2c-47f2e218f981]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-b9022cce-9987-48ad-80df-a841c13b264b tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d, portal: 172.16.0.220,3260]
Aug 09 02:18:59 user nova-compute[72351]: Login to [iface: default, target: iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d, portal: 172.16.0.220,3260] successful.
Aug 09 02:18:59 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[5bfd48d8-0a41-4c34-a90e-e4cb58244870]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-b9022cce-9987-48ad-80df-a841c13b264b tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a358d06a-d58c-434b-bbe7-6f82423b248a]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-b9022cce-9987-48ad-80df-a841c13b264b tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:18:59 user nova-compute[72351]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d (non-flash)
Aug 09 02:18:59 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-b9022cce-9987-48ad-80df-a841c13b264b tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:18:59 user nova-compute[72351]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d (non-flash)
Aug 09 02:18:59 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-b9022cce-9987-48ad-80df-a841c13b264b tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.280s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-b9022cce-9987-48ad-80df-a841c13b264b tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-b9022cce-9987-48ad-80df-a841c13b264b tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] HCTL ('34', '-', '-', 0) found on session 3 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-b9022cce-9987-48ad-80df-a841c13b264b tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Scanning host 34 c: -, t: -, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[045eb46f-10df-44f6-8350-a5cbe8928cf1]: (4, ('- - 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-b9022cce-9987-48ad-80df-a841c13b264b tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Searching for a device in session 3 and hctl ['34', '*', '*', 0] yield: sdc {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-b9022cce-9987-48ad-80df-a841c13b264b tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Connected to sdc using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'tWHOBIpS', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG nova.compute.manager [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Start spawning the instance on the hypervisor. {{(pid=72351) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2612}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Creating instance directory {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4717}}
Aug 09 02:18:59 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Creating image(s)
Aug 09 02:18:59 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Did not create local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4853}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Ensure instance console log exists: /opt/stack/data/nova/instances/a5ca6400-1125-431e-8a91-057e746d40b5/console.log {{(pid=72351) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4603}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Start _get_guest_xml network_info=[{"id": "49802a57-3939-4cc2-956b-3d396b7c738e", "address": "fa:16:3e:9e:eb:c0", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap49802a57-39", "ovs_interfaceid": "49802a57-3939-4cc2-956b-3d396b7c738e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'device_type': 'disk', 'guest_format': None, 'disk_bus': 'virtio', 'mount_device': '/dev/vda', 'boot_index': 0, 'attachment_id': '8a76321a-2bee-4927-99c8-8a0d76a08cdc', 'delete_on_termination': True, 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqns': ['iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'lYLkVxRr', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': 'a5ca6400-1125-431e-8a91-057e746d40b5', 'attached_at': '', 'detached_at': '', 'volume_id': '3f5c13ad-c735-40fa-89eb-c68adda906d7', 'serial': '3f5c13ad-c735-40fa-89eb-c68adda906d7'}, 'volume_type': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:18:59 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:18:59 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:18:59 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 02:18:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'lYLkVxRr', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:18:59 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:18:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:19:00 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-e752ebac-1332-42c7-87ba-2597e9decaf2 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] COPY block job progress, current cursor: 1114112 final cursor: 75497472 {{(pid=72351) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Aug 09 02:19:00 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-b9022cce-9987-48ad-80df-a841c13b264b tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.312s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:19:00 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-b9022cce-9987-48ad-80df-a841c13b264b tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] <== connect_volume: return (2023ms) {'type': 'block', 'scsi_wwn': '23235383435373262', 'path': '/dev/sdc'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:19:00 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b9022cce-9987-48ad-80df-a841c13b264b tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23235383435373262', 'path': '/dev/sdc'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 02:19:00 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 1.276s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:19:00 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:19:00 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:19:00 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:19:00 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 02:19:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:00 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-e752ebac-1332-42c7-87ba-2597e9decaf2 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] COPY block job progress, current cursor: 75497472 final cursor: 75497472 {{(pid=72351) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Aug 09 02:19:00 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-e752ebac-1332-42c7-87ba-2597e9decaf2 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Skipping quiescing instance: QEMU guest agent is not enabled.
Aug 09 02:19:00 user nova-compute[72351]: DEBUG nova.objects.instance [None req-b9022cce-9987-48ad-80df-a841c13b264b tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Lazy-loading 'flavor' on Instance uuid f45b57fa-9264-45f0-9573-e5fb58848d1f {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:19:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b -p 172.16.0.220:3260" returned: 21 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:00 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[08d744ba-23e2-4d2c-8677-7419f6bf3d1d]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:00 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Aug 09 02:19:00 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b -p 172.16.0.220:3260 --interface default --op new {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:00 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[4b87fb32-4ae2-4380-93ae-03d35e7113d3]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b] added\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:00 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b] added
Aug 09 02:19:00 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:00 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-b9022cce-9987-48ad-80df-a841c13b264b tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] attach device xml: <disk type="block" device="disk">
Aug 09 02:19:00 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:19:00 user nova-compute[72351]:   <source dev="/dev/sdc"/>
Aug 09 02:19:00 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:19:00 user nova-compute[72351]:   <serial>a266475f-f65f-4f28-bc8b-724378d46b7d</serial>
Aug 09 02:19:00 user nova-compute[72351]: </disk>
Aug 09 02:19:00 user nova-compute[72351]:  {{(pid=72351) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Aug 09 02:19:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:00 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[cd721214-ec23-45e8-8c41-65be5dc0764a]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:00 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:00 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[2a4e0dec-c3f9-4f5e-992b-ec733932504f]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:00 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b -p 172.16.0.220:3260 --op update -n node.session.auth.username -v pynsiAWj {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:00 user nova-compute[72351]: DEBUG nova.privsep.utils [None req-e752ebac-1332-42c7-87ba-2597e9decaf2 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=72351) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Aug 09 02:19:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-e752ebac-1332-42c7-87ba-2597e9decaf2 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Running cmd (subprocess): qemu-img convert -t none -O qcow2 -f qcow2 /opt/stack/data/nova/instances/snapshots/tmp6vwrf042/fa32ae5053d94b79b570d2b999672e90.delta /opt/stack/data/nova/instances/snapshots/tmp6vwrf042/fa32ae5053d94b79b570d2b999672e90 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b -p 172.16.0.220:3260 --op update -n node.session.auth.username -v pynsiAWj" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:00 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[69ba3599-5f5e-432c-b041-06f7ae5d61e9]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:00 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'pynsiAWj'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:00 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[41deaba6-982f-417f-af2f-ba49d7bda11a]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:00 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.026s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:00 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e413714e-fb19-4132-b21e-fb7963e75fb9]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:00 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:19:00 user nova-compute[72351]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d (non-flash)
Aug 09 02:19:00 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:19:00 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:19:00 user nova-compute[72351]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d (non-flash)
Aug 09 02:19:00 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:19:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b -p 172.16.0.220:3260 --login {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b -p 172.16.0.220:3260 --login" returned: 0 in 0.067s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:00 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[f030ed89-37b2-4375-8553-0d3183f73cdf]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:00 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b, portal: 172.16.0.220,3260]
Aug 09 02:19:00 user nova-compute[72351]: Login to [iface: default, target: iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b, portal: 172.16.0.220,3260] successful.
Aug 09 02:19:00 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.024s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:00 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[44291d29-c2d4-4c9b-a397-f9fcb3d53a9a]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:00 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.031s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:00 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[46456afa-f8eb-4fce-a0f8-8ac126961733]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:00 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:19:00 user nova-compute[72351]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d (non-flash)
Aug 09 02:19:00 user nova-compute[72351]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)
Aug 09 02:19:00 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:19:00 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:19:00 user nova-compute[72351]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d (non-flash)
Aug 09 02:19:00 user nova-compute[72351]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)
Aug 09 02:19:00 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:19:00 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.357s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:19:00 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 02:19:00 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] HCTL ('35', '-', '-', 0) found on session 4 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 02:19:00 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Scanning host 35 c: -, t: -, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 02:19:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host35/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host35/scan" returned: 0 in 0.051s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:00 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[64cfad71-5a39-4e21-899e-09961ce49cef]: (4, ('- - 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:00 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Searching for a device in session 4 and hctl ['35', '*', '*', 0] yield: None {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:19:01 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b9022cce-9987-48ad-80df-a841c13b264b tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:19:01 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b9022cce-9987-48ad-80df-a841c13b264b tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:19:01 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b9022cce-9987-48ad-80df-a841c13b264b tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] No VIF found with MAC fa:16:3e:a0:ba:9c, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:19:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-e752ebac-1332-42c7-87ba-2597e9decaf2 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] CMD "qemu-img convert -t none -O qcow2 -f qcow2 /opt/stack/data/nova/instances/snapshots/tmp6vwrf042/fa32ae5053d94b79b570d2b999672e90.delta /opt/stack/data/nova/instances/snapshots/tmp6vwrf042/fa32ae5053d94b79b570d2b999672e90" returned: 0 in 0.644s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:01 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-e752ebac-1332-42c7-87ba-2597e9decaf2 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Snapshot extracted, beginning image upload
Aug 09 02:19:01 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b9022cce-9987-48ad-80df-a841c13b264b tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Lock "f45b57fa-9264-45f0-9573-e5fb58848d1f" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 12.324s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:01 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Searching for a device in session 4 and hctl ['35', '*', '*', 0] yield: sdd {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:19:01 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Connected to sdd using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'pynsiAWj', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 02:19:01 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.427s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:19:01 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] <== connect_volume: return (2704ms) {'type': 'block', 'scsi_wwn': '23662633531383234', 'path': '/dev/sdd'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:19:01 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23662633531383234', 'path': '/dev/sdd'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 02:19:01 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 1.945s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:19:01 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:19:01 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:19:01 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:19:01 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 02:19:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 -p 172.16.0.220:3260" returned: 21 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:01 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[f5448733-743d-4515-98b6-3f455b00f773]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:01 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Aug 09 02:19:01 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 -p 172.16.0.220:3260 --interface default --op new {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:01 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e1550c5e-5083-4976-8dac-910bf6e1cb6f]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7] added\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:01 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7] added
Aug 09 02:19:01 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:01 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 02:19:01 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'nLZUTgbV', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:19:01 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:19:01 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.076s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[8a2b7008-9eb6-4340-a2d1-10c25f70f6e0]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.031s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[f6c24417-2018-4e06-93d4-7dd2b40699f6]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v lYLkVxRr {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v lYLkVxRr" returned: 0 in 0.026s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[42ebca10-a012-430c-8003-f3c77b9db3d5]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'lYLkVxRr'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.024s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[315f9ffb-e27e-4122-868b-f2134c0e84c8]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.028s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[1c94083e-b4f6-47a3-808c-c788af18ea08]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:19:02 user nova-compute[72351]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d (non-flash)
Aug 09 02:19:02 user nova-compute[72351]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)
Aug 09 02:19:02 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:19:02 user nova-compute[72351]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d (non-flash)
Aug 09 02:19:02 user nova-compute[72351]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)
Aug 09 02:19:02 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 -p 172.16.0.220:3260 --login {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 -p 172.16.0.220:3260 --login" returned: 0 in 0.071s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[76e52ab5-b2d5-40a0-9bf2-6d135efdf25d]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7, portal: 172.16.0.220,3260]
Aug 09 02:19:02 user nova-compute[72351]: Login to [iface: default, target: iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7, portal: 172.16.0.220,3260] successful.
Aug 09 02:19:02 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a5417b92-4294-4e3f-82df-60873b9b1017]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[1db8c221-dbc6-4f44-8202-e6e5270738b1]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:19:02 user nova-compute[72351]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d (non-flash)
Aug 09 02:19:02 user nova-compute[72351]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)
Aug 09 02:19:02 user nova-compute[72351]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:19:02 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:19:02 user nova-compute[72351]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d (non-flash)
Aug 09 02:19:02 user nova-compute[72351]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)
Aug 09 02:19:02 user nova-compute[72351]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:19:02 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.390s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] HCTL ('36', '-', '-', 0) found on session 5 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Scanning host 36 c: -, t: -, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host36/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host36/scan" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[af2a9996-f4ac-4dad-b1e8-d9568e7b48e6]: (4, ('- - 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Searching for a device in session 5 and hctl ['36', '*', '*', 0] yield: sde {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Connected to sde using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'lYLkVxRr', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Acquiring lock "822e25c4-8dd1-4e21-b0ed-f340d59c6c98" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Lock "822e25c4-8dd1-4e21-b0ed-f340d59c6c98" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG nova.compute.manager [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Starting instance... {{(pid=72351) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2406}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-45ab9351-d624-4fdc-ade5-5a49e57667ac tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Acquiring lock "f45b57fa-9264-45f0-9573-e5fb58848d1f" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-45ab9351-d624-4fdc-ade5-5a49e57667ac tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Lock "f45b57fa-9264-45f0-9573-e5fb58848d1f" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-45ab9351-d624-4fdc-ade5-5a49e57667ac tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Acquiring lock "f45b57fa-9264-45f0-9573-e5fb58848d1f-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-45ab9351-d624-4fdc-ade5-5a49e57667ac tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Lock "f45b57fa-9264-45f0-9573-e5fb58848d1f-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-45ab9351-d624-4fdc-ade5-5a49e57667ac tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Lock "f45b57fa-9264-45f0-9573-e5fb58848d1f-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:02 user nova-compute[72351]: INFO nova.compute.manager [None req-45ab9351-d624-4fdc-ade5-5a49e57667ac tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Terminating instance
Aug 09 02:19:02 user nova-compute[72351]: DEBUG nova.compute.manager [None req-45ab9351-d624-4fdc-ade5-5a49e57667ac tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Start destroying the instance on the hypervisor. {{(pid=72351) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3124}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72351) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Aug 09 02:19:02 user nova-compute[72351]: INFO nova.compute.claims [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Claim successful on node user
Aug 09 02:19:02 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG nova.compute.manager [req-5314fd9d-46fe-45ce-806b-a226f32da1cb req-7afc73f6-d187-4f21-b578-c9b8f89d50c4 service nova] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Received event network-vif-unplugged-e9a7bda3-90fa-4274-acc9-3597835503e9 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-5314fd9d-46fe-45ce-806b-a226f32da1cb req-7afc73f6-d187-4f21-b578-c9b8f89d50c4 service nova] Acquiring lock "f45b57fa-9264-45f0-9573-e5fb58848d1f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-5314fd9d-46fe-45ce-806b-a226f32da1cb req-7afc73f6-d187-4f21-b578-c9b8f89d50c4 service nova] Lock "f45b57fa-9264-45f0-9573-e5fb58848d1f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-5314fd9d-46fe-45ce-806b-a226f32da1cb req-7afc73f6-d187-4f21-b578-c9b8f89d50c4 service nova] Lock "f45b57fa-9264-45f0-9573-e5fb58848d1f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG nova.compute.manager [req-5314fd9d-46fe-45ce-806b-a226f32da1cb req-7afc73f6-d187-4f21-b578-c9b8f89d50c4 service nova] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] No waiting events found dispatching network-vif-unplugged-e9a7bda3-90fa-4274-acc9-3597835503e9 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG nova.compute.manager [req-5314fd9d-46fe-45ce-806b-a226f32da1cb req-7afc73f6-d187-4f21-b578-c9b8f89d50c4 service nova] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Received event network-vif-unplugged-e9a7bda3-90fa-4274-acc9-3597835503e9 for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.482s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:02 user nova-compute[72351]: DEBUG nova.compute.manager [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Start building networks asynchronously for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2803}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG nova.compute.manager [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Allocating IP information in the background. {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1956}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG nova.network.neutron [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] allocate_for_instance() {{(pid=72351) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Aug 09 02:19:03 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Aug 09 02:19:03 user nova-compute[72351]: DEBUG nova.compute.manager [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Start building block device mappings for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2838}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG nova.policy [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '761c64dbddf443dbb05b2c1211b9a91d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fd127f53fc54fdbbf8ae524f3578f91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72351) authorize /opt/stack/nova/nova/policy.py:203}}
Aug 09 02:19:03 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Instance destroyed successfully.
Aug 09 02:19:03 user nova-compute[72351]: DEBUG nova.objects.instance [None req-45ab9351-d624-4fdc-ade5-5a49e57667ac tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Lazy-loading 'resources' on Instance uuid f45b57fa-9264-45f0-9573-e5fb58848d1f {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-45ab9351-d624-4fdc-ade5-5a49e57667ac tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:16:53Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-DeleteServersTestJSON-server-1715605419',display_name='tempest-DeleteServersTestJSON-server-1715605419',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-deleteserverstestjson-server-1715605419',id=1,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:17:10Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='e31482d532bf425386728e29c8c90f19',ramdisk_id='',reservation_id='r-1qpnq7xe',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-DeleteServersTestJSON-845954460',owner_user_name='tempest-DeleteServersTestJSON-845954460-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:17:11Z,user_data=None,user_id='77f49e6422b24e83887b084bc2c8e727',uuid=f45b57fa-9264-45f0-9573-e5fb58848d1f,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "e9a7bda3-90fa-4274-acc9-3597835503e9", "address": "fa:16:3e:a0:ba:9c", "network": {"id": "518cac69-7290-48e2-8026-3e1a92d7b7fd", "bridge": "br-int", "label": "tempest-DeleteServersTestJSON-799055594-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e31482d532bf425386728e29c8c90f19", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape9a7bda3-90", "ovs_interfaceid": "e9a7bda3-90fa-4274-acc9-3597835503e9", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-45ab9351-d624-4fdc-ade5-5a49e57667ac tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Converting VIF {"id": "e9a7bda3-90fa-4274-acc9-3597835503e9", "address": "fa:16:3e:a0:ba:9c", "network": {"id": "518cac69-7290-48e2-8026-3e1a92d7b7fd", "bridge": "br-int", "label": "tempest-DeleteServersTestJSON-799055594-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e31482d532bf425386728e29c8c90f19", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape9a7bda3-90", "ovs_interfaceid": "e9a7bda3-90fa-4274-acc9-3597835503e9", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-45ab9351-d624-4fdc-ade5-5a49e57667ac tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:a0:ba:9c,bridge_name='br-int',has_traffic_filtering=True,id=e9a7bda3-90fa-4274-acc9-3597835503e9,network=Network(518cac69-7290-48e2-8026-3e1a92d7b7fd),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape9a7bda3-90') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG os_vif [None req-45ab9351-d624-4fdc-ade5-5a49e57667ac tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:a0:ba:9c,bridge_name='br-int',has_traffic_filtering=True,id=e9a7bda3-90fa-4274-acc9-3597835503e9,network=Network(518cac69-7290-48e2-8026-3e1a92d7b7fd),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape9a7bda3-90') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tape9a7bda3-90, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:19:03 user nova-compute[72351]: INFO os_vif [None req-45ab9351-d624-4fdc-ade5-5a49e57667ac tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:a0:ba:9c,bridge_name='br-int',has_traffic_filtering=True,id=e9a7bda3-90fa-4274-acc9-3597835503e9,network=Network(518cac69-7290-48e2-8026-3e1a92d7b7fd),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape9a7bda3-90')
Aug 09 02:19:03 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-45ab9351-d624-4fdc-ade5-5a49e57667ac tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-45ab9351-d624-4fdc-ade5-5a49e57667ac tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'tWHOBIpS', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc'}, None), 'kwargs': True}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:19:03 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-45ab9351-d624-4fdc-ade5-5a49e57667ac tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:19:03 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-45ab9351-d624-4fdc-ade5-5a49e57667ac tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.438s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] <== connect_volume: return (3383ms) {'type': 'block', 'scsi_wwn': '23338353236663431', 'path': '/dev/sde'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23338353236663431', 'path': '/dev/sde'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 1.380s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:19:03 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:19:03 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:19:03 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 02:19:03 user nova-compute[72351]: DEBUG nova.compute.manager [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Start spawning the instance on the hypervisor. {{(pid=72351) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2612}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Creating instance directory {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4717}}
Aug 09 02:19:03 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Creating image(s)
Aug 09 02:19:03 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Acquiring lock "/opt/stack/data/nova/instances/822e25c4-8dd1-4e21-b0ed-f340d59c6c98/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Lock "/opt/stack/data/nova/instances/822e25c4-8dd1-4e21-b0ed-f340d59c6c98/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Lock "/opt/stack/data/nova/instances/822e25c4-8dd1-4e21-b0ed-f340d59c6c98/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.009s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c -p 172.16.0.220:3260" returned: 21 in 0.029s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[9c5f0d4f-bab7-41fa-b4e1-7bd987e8322e]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Aug 09 02:19:03 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c -p 172.16.0.220:3260 --interface default --op new {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e5aa8fab-609d-4701-a693-7ce8ad68d1e7]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c] added\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c] added
Aug 09 02:19:03 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.060s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a572e069-db51-4fac-9cb4-ddf60dc93581]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:18:43Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-server-1448258389',display_name='tempest-TestVolumeBootPattern-server-1448258389',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testvolumebootpattern-server-1448258389',id=10,image_ref='',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='a4c6bbfd8d904ba1ac69ba5da4564513',ramdisk_id='',reservation_id='r-o29ig997',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestVolumeBootPattern-350743460',owner_user_name='tempest-TestVolumeBootPattern-350743460-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:18:45Z,user_data=None,user_id='07d8d5a634274df19c2db769d2d27b29',uuid=a5ca6400-1125-431e-8a91-057e746d40b5,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "49802a57-3939-4cc2-956b-3d396b7c738e", "address": "fa:16:3e:9e:eb:c0", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap49802a57-39", "ovs_interfaceid": "49802a57-3939-4cc2-956b-3d396b7c738e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Converting VIF {"id": "49802a57-3939-4cc2-956b-3d396b7c738e", "address": "fa:16:3e:9e:eb:c0", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap49802a57-39", "ovs_interfaceid": "49802a57-3939-4cc2-956b-3d396b7c738e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:9e:eb:c0,bridge_name='br-int',has_traffic_filtering=True,id=49802a57-3939-4cc2-956b-3d396b7c738e,network=Network(5ef428cf-e408-490c-8f60-70934f760f4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap49802a57-39') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG nova.objects.instance [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lazy-loading 'pci_devices' on Instance uuid a5ca6400-1125-431e-8a91-057e746d40b5 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.204s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Acquiring lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.039s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[bddebfef-1e67-4e9d-922b-fb37c5379dd8]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:19:03 user nova-compute[72351]:   <uuid>a5ca6400-1125-431e-8a91-057e746d40b5</uuid>
Aug 09 02:19:03 user nova-compute[72351]:   <name>instance-0000000a</name>
Aug 09 02:19:03 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:19:03 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:19:03 user nova-compute[72351]:   <metadata>
Aug 09 02:19:03 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:19:03 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:19:03 user nova-compute[72351]:       <nova:name>tempest-TestVolumeBootPattern-server-1448258389</nova:name>
Aug 09 02:19:03 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:18:59</nova:creationTime>
Aug 09 02:19:03 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:19:03 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:19:03 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:19:03 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:19:03 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:19:03 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:19:03 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:19:03 user nova-compute[72351]:       <nova:owner>
Aug 09 02:19:03 user nova-compute[72351]:         <nova:user uuid="07d8d5a634274df19c2db769d2d27b29">tempest-TestVolumeBootPattern-350743460-project-member</nova:user>
Aug 09 02:19:03 user nova-compute[72351]:         <nova:project uuid="a4c6bbfd8d904ba1ac69ba5da4564513">tempest-TestVolumeBootPattern-350743460</nova:project>
Aug 09 02:19:03 user nova-compute[72351]:       </nova:owner>
Aug 09 02:19:03 user nova-compute[72351]:       <nova:ports>
Aug 09 02:19:03 user nova-compute[72351]:         <nova:port uuid="49802a57-3939-4cc2-956b-3d396b7c738e">
Aug 09 02:19:03 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.6" ipVersion="4"/>
Aug 09 02:19:03 user nova-compute[72351]:         </nova:port>
Aug 09 02:19:03 user nova-compute[72351]:       </nova:ports>
Aug 09 02:19:03 user nova-compute[72351]:     </nova:instance>
Aug 09 02:19:03 user nova-compute[72351]:   </metadata>
Aug 09 02:19:03 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:19:03 user nova-compute[72351]:     <system>
Aug 09 02:19:03 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:19:03 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:19:03 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:19:03 user nova-compute[72351]:       <entry name="serial">a5ca6400-1125-431e-8a91-057e746d40b5</entry>
Aug 09 02:19:03 user nova-compute[72351]:       <entry name="uuid">a5ca6400-1125-431e-8a91-057e746d40b5</entry>
Aug 09 02:19:03 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:19:03 user nova-compute[72351]:     </system>
Aug 09 02:19:03 user nova-compute[72351]:   </sysinfo>
Aug 09 02:19:03 user nova-compute[72351]:   <os>
Aug 09 02:19:03 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:19:03 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:19:03 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:19:03 user nova-compute[72351]:   </os>
Aug 09 02:19:03 user nova-compute[72351]:   <features>
Aug 09 02:19:03 user nova-compute[72351]:     <acpi/>
Aug 09 02:19:03 user nova-compute[72351]:     <apic/>
Aug 09 02:19:03 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:19:03 user nova-compute[72351]:   </features>
Aug 09 02:19:03 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:19:03 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:19:03 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:19:03 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:19:03 user nova-compute[72351]:   </clock>
Aug 09 02:19:03 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:19:03 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:19:03 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:19:03 user nova-compute[72351]:   </cpu>
Aug 09 02:19:03 user nova-compute[72351]:   <devices>
Aug 09 02:19:03 user nova-compute[72351]:     <disk type="block" device="disk">
Aug 09 02:19:03 user nova-compute[72351]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:19:03 user nova-compute[72351]:       <source dev="/dev/sde"/>
Aug 09 02:19:03 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:19:03 user nova-compute[72351]:       <serial>3f5c13ad-c735-40fa-89eb-c68adda906d7</serial>
Aug 09 02:19:03 user nova-compute[72351]:     </disk>
Aug 09 02:19:03 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:19:03 user nova-compute[72351]:       <mac address="fa:16:3e:9e:eb:c0"/>
Aug 09 02:19:03 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:19:03 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:19:03 user nova-compute[72351]:       <target dev="tap49802a57-39"/>
Aug 09 02:19:03 user nova-compute[72351]:     </interface>
Aug 09 02:19:03 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:19:03 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/a5ca6400-1125-431e-8a91-057e746d40b5/console.log" append="off"/>
Aug 09 02:19:03 user nova-compute[72351]:     </serial>
Aug 09 02:19:03 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:19:03 user nova-compute[72351]:     <video>
Aug 09 02:19:03 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:19:03 user nova-compute[72351]:     </video>
Aug 09 02:19:03 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:19:03 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:19:03 user nova-compute[72351]:     </rng>
Aug 09 02:19:03 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:19:03 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:19:03 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:19:03 user nova-compute[72351]:     </memballoon>
Aug 09 02:19:03 user nova-compute[72351]:   </devices>
Aug 09 02:19:03 user nova-compute[72351]: </domain>
Aug 09 02:19:03 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:18:43Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-server-1448258389',display_name='tempest-TestVolumeBootPattern-server-1448258389',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testvolumebootpattern-server-1448258389',id=10,image_ref='',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='a4c6bbfd8d904ba1ac69ba5da4564513',ramdisk_id='',reservation_id='r-o29ig997',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestVolumeBootPattern-350743460',owner_user_name='tempest-TestVolumeBootPattern-350743460-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:18:45Z,user_data=None,user_id='07d8d5a634274df19c2db769d2d27b29',uuid=a5ca6400-1125-431e-8a91-057e746d40b5,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "49802a57-3939-4cc2-956b-3d396b7c738e", "address": "fa:16:3e:9e:eb:c0", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap49802a57-39", "ovs_interfaceid": "49802a57-3939-4cc2-956b-3d396b7c738e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Converting VIF {"id": "49802a57-3939-4cc2-956b-3d396b7c738e", "address": "fa:16:3e:9e:eb:c0", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap49802a57-39", "ovs_interfaceid": "49802a57-3939-4cc2-956b-3d396b7c738e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:9e:eb:c0,bridge_name='br-int',has_traffic_filtering=True,id=49802a57-3939-4cc2-956b-3d396b7c738e,network=Network(5ef428cf-e408-490c-8f60-70934f760f4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap49802a57-39') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG os_vif [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:9e:eb:c0,bridge_name='br-int',has_traffic_filtering=True,id=49802a57-3939-4cc2-956b-3d396b7c738e,network=Network(5ef428cf-e408-490c-8f60-70934f760f4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap49802a57-39') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c -p 172.16.0.220:3260 --op update -n node.session.auth.username -v nLZUTgbV {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap49802a57-39, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap49802a57-39, col_values=(('external_ids', {'iface-id': '49802a57-3939-4cc2-956b-3d396b7c738e', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:9e:eb:c0', 'vm-uuid': 'a5ca6400-1125-431e-8a91-057e746d40b5'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c -p 172.16.0.220:3260 --op update -n node.session.auth.username -v nLZUTgbV" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d3528416-0b37-4cb1-88e3-e30487c96229]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'nLZUTgbV'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:19:03 user nova-compute[72351]: INFO os_vif [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:9e:eb:c0,bridge_name='br-int',has_traffic_filtering=True,id=49802a57-3939-4cc2-956b-3d396b7c738e,network=Network(5ef428cf-e408-490c-8f60-70934f760f4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap49802a57-39')
Aug 09 02:19:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.025s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[58940b11-acaa-4f71-bd14-a6999fa2e9e5]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.136s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/822e25c4-8dd1-4e21-b0ed-f340d59c6c98/disk 1073741824 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[4ef5db10-10d2-453b-8910-72cd073284cf]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:19:03 user nova-compute[72351]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d (non-flash)
Aug 09 02:19:03 user nova-compute[72351]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)
Aug 09 02:19:03 user nova-compute[72351]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:19:03 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:19:03 user nova-compute[72351]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d (non-flash)
Aug 09 02:19:03 user nova-compute[72351]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)
Aug 09 02:19:03 user nova-compute[72351]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:19:03 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c -p 172.16.0.220:3260 --login {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/822e25c4-8dd1-4e21-b0ed-f340d59c6c98/disk 1073741824" returned: 0 in 0.053s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.192s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] No VIF found with MAC fa:16:3e:9e:eb:c0, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c -p 172.16.0.220:3260 --login" returned: 0 in 0.082s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[14d0916f-fc1b-4316-9056-afd04846d3b7]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c, portal: 172.16.0.220,3260]
Aug 09 02:19:03 user nova-compute[72351]: Login to [iface: default, target: iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c, portal: 172.16.0.220,3260] successful.
Aug 09 02:19:03 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[266fbae5-2c88-45f2-8be8-9e34b2ddbeae]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.036s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[9435043a-1a8a-41ab-9004-df5c35ea18f7]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:19:03 user nova-compute[72351]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d (non-flash)
Aug 09 02:19:03 user nova-compute[72351]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)
Aug 09 02:19:03 user nova-compute[72351]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:19:03 user nova-compute[72351]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)
Aug 09 02:19:03 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:19:03 user nova-compute[72351]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d (non-flash)
Aug 09 02:19:03 user nova-compute[72351]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)
Aug 09 02:19:03 user nova-compute[72351]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:19:03 user nova-compute[72351]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)
Aug 09 02:19:03 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.559s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] HCTL ('37', '-', '-', 0) found on session 6 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Scanning host 37 c: -, t: -, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host37/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.164s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Checking if we can resize image /opt/stack/data/nova/instances/822e25c4-8dd1-4e21-b0ed-f340d59c6c98/disk. size=1073741824 {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Aug 09 02:19:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/822e25c4-8dd1-4e21-b0ed-f340d59c6c98/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host37/scan" returned: 0 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:04 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[9abae784-a0f4-4ba0-94e4-8260bf18fbec]: (4, ('- - 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:04 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Searching for a device in session 6 and hctl ['37', '*', '*', 0] yield: sdf {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:19:04 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Connected to sdf using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'nLZUTgbV', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 02:19:04 user nova-compute[72351]: DEBUG nova.network.neutron [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Successfully created port: 6bfee618-fc3b-43ba-b96f-c75db4b7a4e9 {{(pid=72351) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Aug 09 02:19:04 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-e752ebac-1332-42c7-87ba-2597e9decaf2 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Snapshot image upload complete
Aug 09 02:19:04 user nova-compute[72351]: INFO nova.compute.manager [None req-e752ebac-1332-42c7-87ba-2597e9decaf2 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Took 5.74 seconds to snapshot the instance on the hypervisor.
Aug 09 02:19:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/822e25c4-8dd1-4e21-b0ed-f340d59c6c98/disk --force-share --output=json" returned: 0 in 0.172s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:04 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Cannot resize image /opt/stack/data/nova/instances/822e25c4-8dd1-4e21-b0ed-f340d59c6c98/disk to a smaller size. {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Aug 09 02:19:04 user nova-compute[72351]: DEBUG nova.objects.instance [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Lazy-loading 'migration_context' on Instance uuid 822e25c4-8dd1-4e21-b0ed-f340d59c6c98 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:19:04 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Created local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4851}}
Aug 09 02:19:04 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Ensure instance console log exists: /opt/stack/data/nova/instances/822e25c4-8dd1-4e21-b0ed-f340d59c6c98/console.log {{(pid=72351) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4603}}
Aug 09 02:19:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:04 user nova-compute[72351]: DEBUG nova.network.neutron [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Successfully updated port: 6bfee618-fc3b-43ba-b96f-c75db4b7a4e9 {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:19:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Acquiring lock "refresh_cache-822e25c4-8dd1-4e21-b0ed-f340d59c6c98" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:19:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Acquired lock "refresh_cache-822e25c4-8dd1-4e21-b0ed-f340d59c6c98" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:19:04 user nova-compute[72351]: DEBUG nova.network.neutron [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:19:04 user nova-compute[72351]: DEBUG nova.compute.manager [req-33f60766-b235-4848-9982-70674147dc46 req-87c2e634-cd5b-4d0d-8ad4-e47f1d0a8358 service nova] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Received event network-changed-6bfee618-fc3b-43ba-b96f-c75db4b7a4e9 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:19:04 user nova-compute[72351]: DEBUG nova.compute.manager [req-33f60766-b235-4848-9982-70674147dc46 req-87c2e634-cd5b-4d0d-8ad4-e47f1d0a8358 service nova] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Refreshing instance network info cache due to event network-changed-6bfee618-fc3b-43ba-b96f-c75db4b7a4e9. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:19:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-33f60766-b235-4848-9982-70674147dc46 req-87c2e634-cd5b-4d0d-8ad4-e47f1d0a8358 service nova] Acquiring lock "refresh_cache-822e25c4-8dd1-4e21-b0ed-f340d59c6c98" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:19:04 user nova-compute[72351]: DEBUG nova.compute.manager [req-76b89c16-d5d8-4af9-8316-cc78209918df req-555767d1-24f0-46e4-b021-0f6cb4500030 service nova] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Received event network-vif-plugged-e9a7bda3-90fa-4274-acc9-3597835503e9 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:19:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-76b89c16-d5d8-4af9-8316-cc78209918df req-555767d1-24f0-46e4-b021-0f6cb4500030 service nova] Acquiring lock "f45b57fa-9264-45f0-9573-e5fb58848d1f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-76b89c16-d5d8-4af9-8316-cc78209918df req-555767d1-24f0-46e4-b021-0f6cb4500030 service nova] Lock "f45b57fa-9264-45f0-9573-e5fb58848d1f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-76b89c16-d5d8-4af9-8316-cc78209918df req-555767d1-24f0-46e4-b021-0f6cb4500030 service nova] Lock "f45b57fa-9264-45f0-9573-e5fb58848d1f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:04 user nova-compute[72351]: DEBUG nova.compute.manager [req-76b89c16-d5d8-4af9-8316-cc78209918df req-555767d1-24f0-46e4-b021-0f6cb4500030 service nova] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] No waiting events found dispatching network-vif-plugged-e9a7bda3-90fa-4274-acc9-3597835503e9 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:19:04 user nova-compute[72351]: WARNING nova.compute.manager [req-76b89c16-d5d8-4af9-8316-cc78209918df req-555767d1-24f0-46e4-b021-0f6cb4500030 service nova] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Received unexpected event network-vif-plugged-e9a7bda3-90fa-4274-acc9-3597835503e9 for instance with vm_state active and task_state deleting.
Aug 09 02:19:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.645s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] <== connect_volume: return (3025ms) {'type': 'block', 'scsi_wwn': '26132333162316334', 'path': '/dev/sdf'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26132333162316334', 'path': '/dev/sdf'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-45ab9351-d624-4fdc-ade5-5a49e57667ac tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 1.653s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-45ab9351-d624-4fdc-ade5-5a49e57667ac tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[1a172c3a-8114-498c-ab15-54c9e5224cf2]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.045s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a91a5fbf-ab9e-4aaf-aeea-833bac13e13b]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-45ab9351-d624-4fdc-ade5-5a49e57667ac tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:19:05 user nova-compute[72351]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d (non-flash)
Aug 09 02:19:05 user nova-compute[72351]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)
Aug 09 02:19:05 user nova-compute[72351]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:19:05 user nova-compute[72351]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)
Aug 09 02:19:05 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-45ab9351-d624-4fdc-ade5-5a49e57667ac tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:19:05 user nova-compute[72351]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d (non-flash)
Aug 09 02:19:05 user nova-compute[72351]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)
Aug 09 02:19:05 user nova-compute[72351]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:19:05 user nova-compute[72351]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)
Aug 09 02:19:05 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-45ab9351-d624-4fdc-ade5-5a49e57667ac tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fcfdc7d2b00>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d'): ({'sdc'}, set())}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-45ab9351-d624-4fdc-ade5-5a49e57667ac tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Removing single pathed devices sdc {{(pid=72351) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-08-09T02:18:24Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-180184049',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-180184049',id=9,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNlyDeWP+c+bzipwXZAMO2Z2+8MHps3hDZ0m7Lqwq/td4x0E+p/3ch4cYsc0Ot8OOMGBR+QCJGss0jSgkeH83ijl4d9uevYtC5oVznEoGx5YCooCmb7aiHDwnwmo8gkNlA==',key_name='tempest-keypair-1051869033',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='6908d784cc1a41dbbb4bee47e70c259b',ramdisk_id='',reservation_id='r-rl5gtt0e',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-1303879303',owner_user_name='tempest-TaggedBootDevicesTest-1303879303-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:18:27Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='c74c3939709643c8915c7d1678935e9a',uuid=629e177e-afbb-4765-a0b5-7a3f66f72ebf,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "e021d67e-09ab-4779-b639-dff1ba39798a", "address": "fa:16:3e:60:29:2e", "network": {"id": "9d2c7895-3144-4087-8a69-40739a321574", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest-566420926-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape021d67e-09", "ovs_interfaceid": "e021d67e-09ab-4779-b639-dff1ba39798a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Converting VIF {"id": "e021d67e-09ab-4779-b639-dff1ba39798a", "address": "fa:16:3e:60:29:2e", "network": {"id": "9d2c7895-3144-4087-8a69-40739a321574", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest-566420926-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape021d67e-09", "ovs_interfaceid": "e021d67e-09ab-4779-b639-dff1ba39798a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:60:29:2e,bridge_name='br-int',has_traffic_filtering=True,id=e021d67e-09ab-4779-b639-dff1ba39798a,network=Network(9d2c7895-3144-4087-8a69-40739a321574),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape021d67e-09') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-08-09T02:18:24Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-180184049',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-180184049',id=9,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNlyDeWP+c+bzipwXZAMO2Z2+8MHps3hDZ0m7Lqwq/td4x0E+p/3ch4cYsc0Ot8OOMGBR+QCJGss0jSgkeH83ijl4d9uevYtC5oVznEoGx5YCooCmb7aiHDwnwmo8gkNlA==',key_name='tempest-keypair-1051869033',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='6908d784cc1a41dbbb4bee47e70c259b',ramdisk_id='',reservation_id='r-rl5gtt0e',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-1303879303',owner_user_name='tempest-TaggedBootDevicesTest-1303879303-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:18:27Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='c74c3939709643c8915c7d1678935e9a',uuid=629e177e-afbb-4765-a0b5-7a3f66f72ebf,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "f5ecb594-7ac3-496b-9bb0-9f908b91c8fb", "address": "fa:16:3e:30:31:a4", "network": {"id": "4e0c5236-b30d-4213-b1b4-26c0f1a636eb", "bridge": "br-int", "label": "tempest-device-tagging-net1-1640959147", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.122", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf5ecb594-7a", "ovs_interfaceid": "f5ecb594-7ac3-496b-9bb0-9f908b91c8fb", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Converting VIF {"id": "f5ecb594-7ac3-496b-9bb0-9f908b91c8fb", "address": "fa:16:3e:30:31:a4", "network": {"id": "4e0c5236-b30d-4213-b1b4-26c0f1a636eb", "bridge": "br-int", "label": "tempest-device-tagging-net1-1640959147", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.122", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf5ecb594-7a", "ovs_interfaceid": "f5ecb594-7ac3-496b-9bb0-9f908b91c8fb", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:30:31:a4,bridge_name='br-int',has_traffic_filtering=True,id=f5ecb594-7ac3-496b-9bb0-9f908b91c8fb,network=Network(4e0c5236-b30d-4213-b1b4-26c0f1a636eb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tapf5ecb594-7a') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-08-09T02:18:24Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-180184049',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-180184049',id=9,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNlyDeWP+c+bzipwXZAMO2Z2+8MHps3hDZ0m7Lqwq/td4x0E+p/3ch4cYsc0Ot8OOMGBR+QCJGss0jSgkeH83ijl4d9uevYtC5oVznEoGx5YCooCmb7aiHDwnwmo8gkNlA==',key_name='tempest-keypair-1051869033',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='6908d784cc1a41dbbb4bee47e70c259b',ramdisk_id='',reservation_id='r-rl5gtt0e',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-1303879303',owner_user_name='tempest-TaggedBootDevicesTest-1303879303-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:18:27Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='c74c3939709643c8915c7d1678935e9a',uuid=629e177e-afbb-4765-a0b5-7a3f66f72ebf,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "2e9c8a9c-2d81-4b84-b481-9a6b513e730c", "address": "fa:16:3e:39:9b:9d", "network": {"id": "4e0c5236-b30d-4213-b1b4-26c0f1a636eb", "bridge": "br-int", "label": "tempest-device-tagging-net1-1640959147", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.75", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2e9c8a9c-2d", "ovs_interfaceid": "2e9c8a9c-2d81-4b84-b481-9a6b513e730c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Converting VIF {"id": "2e9c8a9c-2d81-4b84-b481-9a6b513e730c", "address": "fa:16:3e:39:9b:9d", "network": {"id": "4e0c5236-b30d-4213-b1b4-26c0f1a636eb", "bridge": "br-int", "label": "tempest-device-tagging-net1-1640959147", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.75", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2e9c8a9c-2d", "ovs_interfaceid": "2e9c8a9c-2d81-4b84-b481-9a6b513e730c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:39:9b:9d,bridge_name='br-int',has_traffic_filtering=True,id=2e9c8a9c-2d81-4b84-b481-9a6b513e730c,network=Network(4e0c5236-b30d-4213-b1b4-26c0f1a636eb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap2e9c8a9c-2d') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-08-09T02:18:24Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-180184049',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-180184049',id=9,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNlyDeWP+c+bzipwXZAMO2Z2+8MHps3hDZ0m7Lqwq/td4x0E+p/3ch4cYsc0Ot8OOMGBR+QCJGss0jSgkeH83ijl4d9uevYtC5oVznEoGx5YCooCmb7aiHDwnwmo8gkNlA==',key_name='tempest-keypair-1051869033',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='6908d784cc1a41dbbb4bee47e70c259b',ramdisk_id='',reservation_id='r-rl5gtt0e',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-1303879303',owner_user_name='tempest-TaggedBootDevicesTest-1303879303-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:18:27Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='c74c3939709643c8915c7d1678935e9a',uuid=629e177e-afbb-4765-a0b5-7a3f66f72ebf,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "5d0bb297-60b1-4e66-8f43-3bf1c32ae4ef", "address": "fa:16:3e:9b:17:d2", "network": {"id": "4e0c5236-b30d-4213-b1b4-26c0f1a636eb", "bridge": "br-int", "label": "tempest-device-tagging-net1-1640959147", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.192", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5d0bb297-60", "ovs_interfaceid": "5d0bb297-60b1-4e66-8f43-3bf1c32ae4ef", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Converting VIF {"id": "5d0bb297-60b1-4e66-8f43-3bf1c32ae4ef", "address": "fa:16:3e:9b:17:d2", "network": {"id": "4e0c5236-b30d-4213-b1b4-26c0f1a636eb", "bridge": "br-int", "label": "tempest-device-tagging-net1-1640959147", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.192", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5d0bb297-60", "ovs_interfaceid": "5d0bb297-60b1-4e66-8f43-3bf1c32ae4ef", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:9b:17:d2,bridge_name='br-int',has_traffic_filtering=True,id=5d0bb297-60b1-4e66-8f43-3bf1c32ae4ef,network=Network(4e0c5236-b30d-4213-b1b4-26c0f1a636eb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap5d0bb297-60') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-08-09T02:18:24Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-180184049',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-180184049',id=9,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNlyDeWP+c+bzipwXZAMO2Z2+8MHps3hDZ0m7Lqwq/td4x0E+p/3ch4cYsc0Ot8OOMGBR+QCJGss0jSgkeH83ijl4d9uevYtC5oVznEoGx5YCooCmb7aiHDwnwmo8gkNlA==',key_name='tempest-keypair-1051869033',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='6908d784cc1a41dbbb4bee47e70c259b',ramdisk_id='',reservation_id='r-rl5gtt0e',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-1303879303',owner_user_name='tempest-TaggedBootDevicesTest-1303879303-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:18:27Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='c74c3939709643c8915c7d1678935e9a',uuid=629e177e-afbb-4765-a0b5-7a3f66f72ebf,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "c2df74bf-22ce-48b1-9ab8-3a684cef7266", "address": "fa:16:3e:3f:be:14", "network": {"id": "4e0c5236-b30d-4213-b1b4-26c0f1a636eb", "bridge": "br-int", "label": "tempest-device-tagging-net1-1640959147", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.128", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc2df74bf-22", "ovs_interfaceid": "c2df74bf-22ce-48b1-9ab8-3a684cef7266", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Converting VIF {"id": "c2df74bf-22ce-48b1-9ab8-3a684cef7266", "address": "fa:16:3e:3f:be:14", "network": {"id": "4e0c5236-b30d-4213-b1b4-26c0f1a636eb", "bridge": "br-int", "label": "tempest-device-tagging-net1-1640959147", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.128", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc2df74bf-22", "ovs_interfaceid": "c2df74bf-22ce-48b1-9ab8-3a684cef7266", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:3f:be:14,bridge_name='br-int',has_traffic_filtering=True,id=c2df74bf-22ce-48b1-9ab8-3a684cef7266,network=Network(4e0c5236-b30d-4213-b1b4-26c0f1a636eb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc2df74bf-22') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-08-09T02:18:24Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-180184049',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-180184049',id=9,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNlyDeWP+c+bzipwXZAMO2Z2+8MHps3hDZ0m7Lqwq/td4x0E+p/3ch4cYsc0Ot8OOMGBR+QCJGss0jSgkeH83ijl4d9uevYtC5oVznEoGx5YCooCmb7aiHDwnwmo8gkNlA==',key_name='tempest-keypair-1051869033',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='6908d784cc1a41dbbb4bee47e70c259b',ramdisk_id='',reservation_id='r-rl5gtt0e',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-1303879303',owner_user_name='tempest-TaggedBootDevicesTest-1303879303-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:18:27Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='c74c3939709643c8915c7d1678935e9a',uuid=629e177e-afbb-4765-a0b5-7a3f66f72ebf,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "d1b9314b-1800-41fb-a34b-0186f8e746da", "address": "fa:16:3e:37:27:3b", "network": {"id": "b4ebdbb0-e697-4a42-93f5-946522a3dbce", "bridge": "br-int", "label": "tempest-device-tagging-net2-356335140", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd1b9314b-18", "ovs_interfaceid": "d1b9314b-1800-41fb-a34b-0186f8e746da", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Converting VIF {"id": "d1b9314b-1800-41fb-a34b-0186f8e746da", "address": "fa:16:3e:37:27:3b", "network": {"id": "b4ebdbb0-e697-4a42-93f5-946522a3dbce", "bridge": "br-int", "label": "tempest-device-tagging-net2-356335140", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd1b9314b-18", "ovs_interfaceid": "d1b9314b-1800-41fb-a34b-0186f8e746da", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:37:27:3b,bridge_name='br-int',has_traffic_filtering=True,id=d1b9314b-1800-41fb-a34b-0186f8e746da,network=Network(b4ebdbb0-e697-4a42-93f5-946522a3dbce),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd1b9314b-18') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-08-09T02:18:24Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-180184049',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-180184049',id=9,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNlyDeWP+c+bzipwXZAMO2Z2+8MHps3hDZ0m7Lqwq/td4x0E+p/3ch4cYsc0Ot8OOMGBR+QCJGss0jSgkeH83ijl4d9uevYtC5oVznEoGx5YCooCmb7aiHDwnwmo8gkNlA==',key_name='tempest-keypair-1051869033',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='6908d784cc1a41dbbb4bee47e70c259b',ramdisk_id='',reservation_id='r-rl5gtt0e',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-1303879303',owner_user_name='tempest-TaggedBootDevicesTest-1303879303-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:18:27Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='c74c3939709643c8915c7d1678935e9a',uuid=629e177e-afbb-4765-a0b5-7a3f66f72ebf,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "c7c195c0-af76-445e-8a18-8169f0cdf5fc", "address": "fa:16:3e:f7:2d:31", "network": {"id": "b4ebdbb0-e697-4a42-93f5-946522a3dbce", "bridge": "br-int", "label": "tempest-device-tagging-net2-356335140", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc7c195c0-af", "ovs_interfaceid": "c7c195c0-af76-445e-8a18-8169f0cdf5fc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Converting VIF {"id": "c7c195c0-af76-445e-8a18-8169f0cdf5fc", "address": "fa:16:3e:f7:2d:31", "network": {"id": "b4ebdbb0-e697-4a42-93f5-946522a3dbce", "bridge": "br-int", "label": "tempest-device-tagging-net2-356335140", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc7c195c0-af", "ovs_interfaceid": "c7c195c0-af76-445e-8a18-8169f0cdf5fc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:f7:2d:31,bridge_name='br-int',has_traffic_filtering=True,id=c7c195c0-af76-445e-8a18-8169f0cdf5fc,network=Network(b4ebdbb0-e697-4a42-93f5-946522a3dbce),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc7c195c0-af') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.objects.instance [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Lazy-loading 'pci_devices' on Instance uuid 629e177e-afbb-4765-a0b5-7a3f66f72ebf {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[51a99680-9710-4a77-872c-19e04747a554]: (4, ('path checker states:\nup                  5\n\npaths: 0\nbusy: False\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.network.neutron [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdc {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:19:05 user nova-compute[72351]:   <uuid>629e177e-afbb-4765-a0b5-7a3f66f72ebf</uuid>
Aug 09 02:19:05 user nova-compute[72351]:   <name>instance-00000009</name>
Aug 09 02:19:05 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:19:05 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:19:05 user nova-compute[72351]:   <metadata>
Aug 09 02:19:05 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:19:05 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:19:05 user nova-compute[72351]:       <nova:name>tempest-device-tagging-server-180184049</nova:name>
Aug 09 02:19:05 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:18:57</nova:creationTime>
Aug 09 02:19:05 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:19:05 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:19:05 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:19:05 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:19:05 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:19:05 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:19:05 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:19:05 user nova-compute[72351]:       <nova:owner>
Aug 09 02:19:05 user nova-compute[72351]:         <nova:user uuid="c74c3939709643c8915c7d1678935e9a">tempest-TaggedBootDevicesTest-1303879303-project-member</nova:user>
Aug 09 02:19:05 user nova-compute[72351]:         <nova:project uuid="6908d784cc1a41dbbb4bee47e70c259b">tempest-TaggedBootDevicesTest-1303879303</nova:project>
Aug 09 02:19:05 user nova-compute[72351]:       </nova:owner>
Aug 09 02:19:05 user nova-compute[72351]:       <nova:root type="image" uuid="18ccda7f-31d3-4978-ab9d-4ed2f74eff94"/>
Aug 09 02:19:05 user nova-compute[72351]:       <nova:ports>
Aug 09 02:19:05 user nova-compute[72351]:         <nova:port uuid="e021d67e-09ab-4779-b639-dff1ba39798a">
Aug 09 02:19:05 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.13" ipVersion="4"/>
Aug 09 02:19:05 user nova-compute[72351]:         </nova:port>
Aug 09 02:19:05 user nova-compute[72351]:         <nova:port uuid="f5ecb594-7ac3-496b-9bb0-9f908b91c8fb">
Aug 09 02:19:05 user nova-compute[72351]:           <nova:ip type="fixed" address="10.1.1.122" ipVersion="4"/>
Aug 09 02:19:05 user nova-compute[72351]:         </nova:port>
Aug 09 02:19:05 user nova-compute[72351]:         <nova:port uuid="2e9c8a9c-2d81-4b84-b481-9a6b513e730c">
Aug 09 02:19:05 user nova-compute[72351]:           <nova:ip type="fixed" address="10.1.1.75" ipVersion="4"/>
Aug 09 02:19:05 user nova-compute[72351]:         </nova:port>
Aug 09 02:19:05 user nova-compute[72351]:         <nova:port uuid="5d0bb297-60b1-4e66-8f43-3bf1c32ae4ef">
Aug 09 02:19:05 user nova-compute[72351]:           <nova:ip type="fixed" address="10.1.1.192" ipVersion="4"/>
Aug 09 02:19:05 user nova-compute[72351]:         </nova:port>
Aug 09 02:19:05 user nova-compute[72351]:         <nova:port uuid="c2df74bf-22ce-48b1-9ab8-3a684cef7266">
Aug 09 02:19:05 user nova-compute[72351]:           <nova:ip type="fixed" address="10.1.1.128" ipVersion="4"/>
Aug 09 02:19:05 user nova-compute[72351]:         </nova:port>
Aug 09 02:19:05 user nova-compute[72351]:         <nova:port uuid="d1b9314b-1800-41fb-a34b-0186f8e746da">
Aug 09 02:19:05 user nova-compute[72351]:           <nova:ip type="fixed" address="10.2.2.100" ipVersion="4"/>
Aug 09 02:19:05 user nova-compute[72351]:         </nova:port>
Aug 09 02:19:05 user nova-compute[72351]:         <nova:port uuid="c7c195c0-af76-445e-8a18-8169f0cdf5fc">
Aug 09 02:19:05 user nova-compute[72351]:           <nova:ip type="fixed" address="10.2.2.200" ipVersion="4"/>
Aug 09 02:19:05 user nova-compute[72351]:         </nova:port>
Aug 09 02:19:05 user nova-compute[72351]:       </nova:ports>
Aug 09 02:19:05 user nova-compute[72351]:     </nova:instance>
Aug 09 02:19:05 user nova-compute[72351]:   </metadata>
Aug 09 02:19:05 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:19:05 user nova-compute[72351]:     <system>
Aug 09 02:19:05 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:19:05 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:19:05 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:19:05 user nova-compute[72351]:       <entry name="serial">629e177e-afbb-4765-a0b5-7a3f66f72ebf</entry>
Aug 09 02:19:05 user nova-compute[72351]:       <entry name="uuid">629e177e-afbb-4765-a0b5-7a3f66f72ebf</entry>
Aug 09 02:19:05 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:19:05 user nova-compute[72351]:     </system>
Aug 09 02:19:05 user nova-compute[72351]:   </sysinfo>
Aug 09 02:19:05 user nova-compute[72351]:   <os>
Aug 09 02:19:05 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:19:05 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:19:05 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:19:05 user nova-compute[72351]:   </os>
Aug 09 02:19:05 user nova-compute[72351]:   <features>
Aug 09 02:19:05 user nova-compute[72351]:     <acpi/>
Aug 09 02:19:05 user nova-compute[72351]:     <apic/>
Aug 09 02:19:05 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:19:05 user nova-compute[72351]:   </features>
Aug 09 02:19:05 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:19:05 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:19:05 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:19:05 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:19:05 user nova-compute[72351]:   </clock>
Aug 09 02:19:05 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:19:05 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:19:05 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:19:05 user nova-compute[72351]:   </cpu>
Aug 09 02:19:05 user nova-compute[72351]:   <devices>
Aug 09 02:19:05 user nova-compute[72351]:     <disk type="file" device="cdrom">
Aug 09 02:19:05 user nova-compute[72351]:       <driver name="qemu" type="raw" cache="none"/>
Aug 09 02:19:05 user nova-compute[72351]:       <source file="/opt/stack/data/nova/instances/629e177e-afbb-4765-a0b5-7a3f66f72ebf/disk.config"/>
Aug 09 02:19:05 user nova-compute[72351]:       <target dev="hda" bus="ide"/>
Aug 09 02:19:05 user nova-compute[72351]:     </disk>
Aug 09 02:19:05 user nova-compute[72351]:     <disk type="block" device="disk">
Aug 09 02:19:05 user nova-compute[72351]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:19:05 user nova-compute[72351]:       <source dev="/dev/sdb"/>
Aug 09 02:19:05 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:19:05 user nova-compute[72351]:       <serial>a8ce86ce-79f7-4138-a49b-c45c9db38c97</serial>
Aug 09 02:19:05 user nova-compute[72351]:     </disk>
Aug 09 02:19:05 user nova-compute[72351]:     <disk type="block" device="disk">
Aug 09 02:19:05 user nova-compute[72351]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:19:05 user nova-compute[72351]:       <source dev="/dev/sdd"/>
Aug 09 02:19:05 user nova-compute[72351]:       <target dev="vdb" bus="virtio"/>
Aug 09 02:19:05 user nova-compute[72351]:       <serial>d1fa2bd8-1c39-4730-b60c-35dcbf24b18b</serial>
Aug 09 02:19:05 user nova-compute[72351]:     </disk>
Aug 09 02:19:05 user nova-compute[72351]:     <disk type="block" device="disk">
Aug 09 02:19:05 user nova-compute[72351]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:19:05 user nova-compute[72351]:       <source dev="/dev/sdf"/>
Aug 09 02:19:05 user nova-compute[72351]:       <target dev="vdc" bus="virtio"/>
Aug 09 02:19:05 user nova-compute[72351]:       <serial>a3218523-2c3f-4787-b0ea-f589160d940c</serial>
Aug 09 02:19:05 user nova-compute[72351]:     </disk>
Aug 09 02:19:05 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:19:05 user nova-compute[72351]:       <mac address="fa:16:3e:60:29:2e"/>
Aug 09 02:19:05 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:19:05 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:19:05 user nova-compute[72351]:       <target dev="tape021d67e-09"/>
Aug 09 02:19:05 user nova-compute[72351]:     </interface>
Aug 09 02:19:05 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:19:05 user nova-compute[72351]:       <mac address="fa:16:3e:30:31:a4"/>
Aug 09 02:19:05 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:19:05 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:19:05 user nova-compute[72351]:       <target dev="tapf5ecb594-7a"/>
Aug 09 02:19:05 user nova-compute[72351]:     </interface>
Aug 09 02:19:05 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:19:05 user nova-compute[72351]:       <mac address="fa:16:3e:39:9b:9d"/>
Aug 09 02:19:05 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:19:05 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:19:05 user nova-compute[72351]:       <target dev="tap2e9c8a9c-2d"/>
Aug 09 02:19:05 user nova-compute[72351]:     </interface>
Aug 09 02:19:05 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:19:05 user nova-compute[72351]:       <mac address="fa:16:3e:9b:17:d2"/>
Aug 09 02:19:05 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:19:05 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:19:05 user nova-compute[72351]:       <target dev="tap5d0bb297-60"/>
Aug 09 02:19:05 user nova-compute[72351]:     </interface>
Aug 09 02:19:05 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:19:05 user nova-compute[72351]:       <mac address="fa:16:3e:3f:be:14"/>
Aug 09 02:19:05 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:19:05 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:19:05 user nova-compute[72351]:       <target dev="tapc2df74bf-22"/>
Aug 09 02:19:05 user nova-compute[72351]:     </interface>
Aug 09 02:19:05 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:19:05 user nova-compute[72351]:       <mac address="fa:16:3e:37:27:3b"/>
Aug 09 02:19:05 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:19:05 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:19:05 user nova-compute[72351]:       <target dev="tapd1b9314b-18"/>
Aug 09 02:19:05 user nova-compute[72351]:     </interface>
Aug 09 02:19:05 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:19:05 user nova-compute[72351]:       <mac address="fa:16:3e:f7:2d:31"/>
Aug 09 02:19:05 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:19:05 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:19:05 user nova-compute[72351]:       <target dev="tapc7c195c0-af"/>
Aug 09 02:19:05 user nova-compute[72351]:     </interface>
Aug 09 02:19:05 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:19:05 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/629e177e-afbb-4765-a0b5-7a3f66f72ebf/console.log" append="off"/>
Aug 09 02:19:05 user nova-compute[72351]:     </serial>
Aug 09 02:19:05 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:19:05 user nova-compute[72351]:     <video>
Aug 09 02:19:05 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:19:05 user nova-compute[72351]:     </video>
Aug 09 02:19:05 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:19:05 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:19:05 user nova-compute[72351]:     </rng>
Aug 09 02:19:05 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:19:05 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:19:05 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:19:05 user nova-compute[72351]:     </memballoon>
Aug 09 02:19:05 user nova-compute[72351]:   </devices>
Aug 09 02:19:05 user nova-compute[72351]: </domain>
Aug 09 02:19:05 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-08-09T02:18:24Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-180184049',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-180184049',id=9,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNlyDeWP+c+bzipwXZAMO2Z2+8MHps3hDZ0m7Lqwq/td4x0E+p/3ch4cYsc0Ot8OOMGBR+QCJGss0jSgkeH83ijl4d9uevYtC5oVznEoGx5YCooCmb7aiHDwnwmo8gkNlA==',key_name='tempest-keypair-1051869033',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='6908d784cc1a41dbbb4bee47e70c259b',ramdisk_id='',reservation_id='r-rl5gtt0e',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-1303879303',owner_user_name='tempest-TaggedBootDevicesTest-1303879303-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:18:27Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='c74c3939709643c8915c7d1678935e9a',uuid=629e177e-afbb-4765-a0b5-7a3f66f72ebf,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "e021d67e-09ab-4779-b639-dff1ba39798a", "address": "fa:16:3e:60:29:2e", "network": {"id": "9d2c7895-3144-4087-8a69-40739a321574", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest-566420926-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape021d67e-09", "ovs_interfaceid": "e021d67e-09ab-4779-b639-dff1ba39798a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Converting VIF {"id": "e021d67e-09ab-4779-b639-dff1ba39798a", "address": "fa:16:3e:60:29:2e", "network": {"id": "9d2c7895-3144-4087-8a69-40739a321574", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest-566420926-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape021d67e-09", "ovs_interfaceid": "e021d67e-09ab-4779-b639-dff1ba39798a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:60:29:2e,bridge_name='br-int',has_traffic_filtering=True,id=e021d67e-09ab-4779-b639-dff1ba39798a,network=Network(9d2c7895-3144-4087-8a69-40739a321574),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape021d67e-09') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG os_vif [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:60:29:2e,bridge_name='br-int',has_traffic_filtering=True,id=e021d67e-09ab-4779-b639-dff1ba39798a,network=Network(9d2c7895-3144-4087-8a69-40739a321574),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape021d67e-09') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdc" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[5ae1e5e2-a07a-493c-99b7-004c1d97e774]: (4, ('ok\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-45ab9351-d624-4fdc-ade5-5a49e57667ac tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Flushing IO for device /dev/sdc {{(pid=72351) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tape021d67e-09, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tape021d67e-09, col_values=(('external_ids', {'iface-id': 'e021d67e-09ab-4779-b639-dff1ba39798a', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:60:29:2e', 'vm-uuid': '629e177e-afbb-4765-a0b5-7a3f66f72ebf'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdc {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdc" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[8099f37d-00e4-4eab-8eed-6aea107c92a3]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-45ab9351-d624-4fdc-ade5-5a49e57667ac tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Remove SCSI device /dev/sdc with /sys/block/sdc/device/delete {{(pid=72351) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdc/device/delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdc/device/delete" returned: 0 in 0.037s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:05 user nova-compute[72351]: INFO os_vif [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:60:29:2e,bridge_name='br-int',has_traffic_filtering=True,id=e021d67e-09ab-4779-b639-dff1ba39798a,network=Network(9d2c7895-3144-4087-8a69-40739a321574),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape021d67e-09')
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-08-09T02:18:24Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-180184049',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-180184049',id=9,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNlyDeWP+c+bzipwXZAMO2Z2+8MHps3hDZ0m7Lqwq/td4x0E+p/3ch4cYsc0Ot8OOMGBR+QCJGss0jSgkeH83ijl4d9uevYtC5oVznEoGx5YCooCmb7aiHDwnwmo8gkNlA==',key_name='tempest-keypair-1051869033',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='6908d784cc1a41dbbb4bee47e70c259b',ramdisk_id='',reservation_id='r-rl5gtt0e',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-1303879303',owner_user_name='tempest-TaggedBootDevicesTest-1303879303-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:18:27Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='c74c3939709643c8915c7d1678935e9a',uuid=629e177e-afbb-4765-a0b5-7a3f66f72ebf,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "f5ecb594-7ac3-496b-9bb0-9f908b91c8fb", "address": "fa:16:3e:30:31:a4", "network": {"id": "4e0c5236-b30d-4213-b1b4-26c0f1a636eb", "bridge": "br-int", "label": "tempest-device-tagging-net1-1640959147", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.122", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf5ecb594-7a", "ovs_interfaceid": "f5ecb594-7ac3-496b-9bb0-9f908b91c8fb", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Converting VIF {"id": "f5ecb594-7ac3-496b-9bb0-9f908b91c8fb", "address": "fa:16:3e:30:31:a4", "network": {"id": "4e0c5236-b30d-4213-b1b4-26c0f1a636eb", "bridge": "br-int", "label": "tempest-device-tagging-net1-1640959147", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.122", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf5ecb594-7a", "ovs_interfaceid": "f5ecb594-7ac3-496b-9bb0-9f908b91c8fb", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:30:31:a4,bridge_name='br-int',has_traffic_filtering=True,id=f5ecb594-7ac3-496b-9bb0-9f908b91c8fb,network=Network(4e0c5236-b30d-4213-b1b4-26c0f1a636eb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tapf5ecb594-7a') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG os_vif [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:30:31:a4,bridge_name='br-int',has_traffic_filtering=True,id=f5ecb594-7ac3-496b-9bb0-9f908b91c8fb,network=Network(4e0c5236-b30d-4213-b1b4-26c0f1a636eb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tapf5ecb594-7a') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[aa398f34-9c35-45bd-ac32-db30198015c2]: (4, ('1', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-45ab9351-d624-4fdc-ade5-5a49e57667ac tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Checking to see if SCSI volumes sdc have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-45ab9351-d624-4fdc-ade5-5a49e57667ac tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] SCSI volumes sdc have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-45ab9351-d624-4fdc-ade5-5a49e57667ac tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d')] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapf5ecb594-7a, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapf5ecb594-7a, col_values=(('external_ids', {'iface-id': 'f5ecb594-7ac3-496b-9bb0-9f908b91c8fb', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:30:31:a4', 'vm-uuid': '629e177e-afbb-4765-a0b5-7a3f66f72ebf'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[cfe33e13-68ae-47b2-9726-cd671633caaa]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-45ab9351-d624-4fdc-ade5-5a49e57667ac tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:05 user nova-compute[72351]: INFO os_vif [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:30:31:a4,bridge_name='br-int',has_traffic_filtering=True,id=f5ecb594-7ac3-496b-9bb0-9f908b91c8fb,network=Network(4e0c5236-b30d-4213-b1b4-26c0f1a636eb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tapf5ecb594-7a')
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-08-09T02:18:24Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-180184049',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-180184049',id=9,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNlyDeWP+c+bzipwXZAMO2Z2+8MHps3hDZ0m7Lqwq/td4x0E+p/3ch4cYsc0Ot8OOMGBR+QCJGss0jSgkeH83ijl4d9uevYtC5oVznEoGx5YCooCmb7aiHDwnwmo8gkNlA==',key_name='tempest-keypair-1051869033',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='6908d784cc1a41dbbb4bee47e70c259b',ramdisk_id='',reservation_id='r-rl5gtt0e',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-1303879303',owner_user_name='tempest-TaggedBootDevicesTest-1303879303-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:18:27Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='c74c3939709643c8915c7d1678935e9a',uuid=629e177e-afbb-4765-a0b5-7a3f66f72ebf,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "2e9c8a9c-2d81-4b84-b481-9a6b513e730c", "address": "fa:16:3e:39:9b:9d", "network": {"id": "4e0c5236-b30d-4213-b1b4-26c0f1a636eb", "bridge": "br-int", "label": "tempest-device-tagging-net1-1640959147", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.75", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2e9c8a9c-2d", "ovs_interfaceid": "2e9c8a9c-2d81-4b84-b481-9a6b513e730c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Converting VIF {"id": "2e9c8a9c-2d81-4b84-b481-9a6b513e730c", "address": "fa:16:3e:39:9b:9d", "network": {"id": "4e0c5236-b30d-4213-b1b4-26c0f1a636eb", "bridge": "br-int", "label": "tempest-device-tagging-net1-1640959147", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.75", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2e9c8a9c-2d", "ovs_interfaceid": "2e9c8a9c-2d81-4b84-b481-9a6b513e730c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:39:9b:9d,bridge_name='br-int',has_traffic_filtering=True,id=2e9c8a9c-2d81-4b84-b481-9a6b513e730c,network=Network(4e0c5236-b30d-4213-b1b4-26c0f1a636eb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap2e9c8a9c-2d') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG os_vif [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:39:9b:9d,bridge_name='br-int',has_traffic_filtering=True,id=2e9c8a9c-2d81-4b84-b481-9a6b513e730c,network=Network(4e0c5236-b30d-4213-b1b4-26c0f1a636eb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap2e9c8a9c-2d') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d -p 172.16.0.220:3260 --logout {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap2e9c8a9c-2d, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap2e9c8a9c-2d, col_values=(('external_ids', {'iface-id': '2e9c8a9c-2d81-4b84-b481-9a6b513e730c', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:39:9b:9d', 'vm-uuid': '629e177e-afbb-4765-a0b5-7a3f66f72ebf'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:05 user nova-compute[72351]: INFO os_vif [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:39:9b:9d,bridge_name='br-int',has_traffic_filtering=True,id=2e9c8a9c-2d81-4b84-b481-9a6b513e730c,network=Network(4e0c5236-b30d-4213-b1b4-26c0f1a636eb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap2e9c8a9c-2d')
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-08-09T02:18:24Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-180184049',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-180184049',id=9,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNlyDeWP+c+bzipwXZAMO2Z2+8MHps3hDZ0m7Lqwq/td4x0E+p/3ch4cYsc0Ot8OOMGBR+QCJGss0jSgkeH83ijl4d9uevYtC5oVznEoGx5YCooCmb7aiHDwnwmo8gkNlA==',key_name='tempest-keypair-1051869033',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='6908d784cc1a41dbbb4bee47e70c259b',ramdisk_id='',reservation_id='r-rl5gtt0e',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-1303879303',owner_user_name='tempest-TaggedBootDevicesTest-1303879303-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:18:27Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='c74c3939709643c8915c7d1678935e9a',uuid=629e177e-afbb-4765-a0b5-7a3f66f72ebf,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "5d0bb297-60b1-4e66-8f43-3bf1c32ae4ef", "address": "fa:16:3e:9b:17:d2", "network": {"id": "4e0c5236-b30d-4213-b1b4-26c0f1a636eb", "bridge": "br-int", "label": "tempest-device-tagging-net1-1640959147", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.192", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5d0bb297-60", "ovs_interfaceid": "5d0bb297-60b1-4e66-8f43-3bf1c32ae4ef", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Converting VIF {"id": "5d0bb297-60b1-4e66-8f43-3bf1c32ae4ef", "address": "fa:16:3e:9b:17:d2", "network": {"id": "4e0c5236-b30d-4213-b1b4-26c0f1a636eb", "bridge": "br-int", "label": "tempest-device-tagging-net1-1640959147", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.192", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5d0bb297-60", "ovs_interfaceid": "5d0bb297-60b1-4e66-8f43-3bf1c32ae4ef", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:9b:17:d2,bridge_name='br-int',has_traffic_filtering=True,id=5d0bb297-60b1-4e66-8f43-3bf1c32ae4ef,network=Network(4e0c5236-b30d-4213-b1b4-26c0f1a636eb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap5d0bb297-60') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG os_vif [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:9b:17:d2,bridge_name='br-int',has_traffic_filtering=True,id=5d0bb297-60b1-4e66-8f43-3bf1c32ae4ef,network=Network(4e0c5236-b30d-4213-b1b4-26c0f1a636eb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap5d0bb297-60') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap5d0bb297-60, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap5d0bb297-60, col_values=(('external_ids', {'iface-id': '5d0bb297-60b1-4e66-8f43-3bf1c32ae4ef', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:9b:17:d2', 'vm-uuid': '629e177e-afbb-4765-a0b5-7a3f66f72ebf'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d -p 172.16.0.220:3260 --logout" returned: 0 in 0.081s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[46b0f299-273f-40e3-8088-a097cfeae62f]: (4, ('Logging out of session [sid: 3, target: iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d, portal: 172.16.0.220,3260]\nLogout of [sid: 3, target: iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-45ab9351-d624-4fdc-ade5-5a49e57667ac tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 3, target: iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d, portal: 172.16.0.220,3260]
Aug 09 02:19:05 user nova-compute[72351]: Logout of [sid: 3, target: iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d, portal: 172.16.0.220,3260] successful.
Aug 09 02:19:05 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d -p 172.16.0.220:3260 --op delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a266475f-f65f-4f28-bc8b-724378d46b7d -p 172.16.0.220:3260 --op delete" returned: 0 in 0.053s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:05 user nova-compute[72351]: INFO os_vif [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:9b:17:d2,bridge_name='br-int',has_traffic_filtering=True,id=5d0bb297-60b1-4e66-8f43-3bf1c32ae4ef,network=Network(4e0c5236-b30d-4213-b1b4-26c0f1a636eb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap5d0bb297-60')
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-08-09T02:18:24Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-180184049',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-180184049',id=9,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNlyDeWP+c+bzipwXZAMO2Z2+8MHps3hDZ0m7Lqwq/td4x0E+p/3ch4cYsc0Ot8OOMGBR+QCJGss0jSgkeH83ijl4d9uevYtC5oVznEoGx5YCooCmb7aiHDwnwmo8gkNlA==',key_name='tempest-keypair-1051869033',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='6908d784cc1a41dbbb4bee47e70c259b',ramdisk_id='',reservation_id='r-rl5gtt0e',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-1303879303',owner_user_name='tempest-TaggedBootDevicesTest-1303879303-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:18:27Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='c74c3939709643c8915c7d1678935e9a',uuid=629e177e-afbb-4765-a0b5-7a3f66f72ebf,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "c2df74bf-22ce-48b1-9ab8-3a684cef7266", "address": "fa:16:3e:3f:be:14", "network": {"id": "4e0c5236-b30d-4213-b1b4-26c0f1a636eb", "bridge": "br-int", "label": "tempest-device-tagging-net1-1640959147", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.128", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc2df74bf-22", "ovs_interfaceid": "c2df74bf-22ce-48b1-9ab8-3a684cef7266", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Converting VIF {"id": "c2df74bf-22ce-48b1-9ab8-3a684cef7266", "address": "fa:16:3e:3f:be:14", "network": {"id": "4e0c5236-b30d-4213-b1b4-26c0f1a636eb", "bridge": "br-int", "label": "tempest-device-tagging-net1-1640959147", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.128", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc2df74bf-22", "ovs_interfaceid": "c2df74bf-22ce-48b1-9ab8-3a684cef7266", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:3f:be:14,bridge_name='br-int',has_traffic_filtering=True,id=c2df74bf-22ce-48b1-9ab8-3a684cef7266,network=Network(4e0c5236-b30d-4213-b1b4-26c0f1a636eb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc2df74bf-22') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG os_vif [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:3f:be:14,bridge_name='br-int',has_traffic_filtering=True,id=c2df74bf-22ce-48b1-9ab8-3a684cef7266,network=Network(4e0c5236-b30d-4213-b1b4-26c0f1a636eb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc2df74bf-22') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a4a0066b-9394-4303-aca7-7734dac2cb69]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-45ab9351-d624-4fdc-ade5-5a49e57667ac tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-45ab9351-d624-4fdc-ade5-5a49e57667ac tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.391s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-45ab9351-d624-4fdc-ade5-5a49e57667ac tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] <== disconnect_volume: return (2044ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-45ab9351-d624-4fdc-ade5-5a49e57667ac tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 02:19:05 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-45ab9351-d624-4fdc-ade5-5a49e57667ac tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Deleting instance files /opt/stack/data/nova/instances/f45b57fa-9264-45f0-9573-e5fb58848d1f_del
Aug 09 02:19:05 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-45ab9351-d624-4fdc-ade5-5a49e57667ac tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Deletion of /opt/stack/data/nova/instances/f45b57fa-9264-45f0-9573-e5fb58848d1f_del complete
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapc2df74bf-22, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapc2df74bf-22, col_values=(('external_ids', {'iface-id': 'c2df74bf-22ce-48b1-9ab8-3a684cef7266', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:3f:be:14', 'vm-uuid': '629e177e-afbb-4765-a0b5-7a3f66f72ebf'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:19:05 user nova-compute[72351]: INFO nova.compute.manager [None req-45ab9351-d624-4fdc-ade5-5a49e57667ac tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Took 3.02 seconds to destroy the instance on the hypervisor.
Aug 09 02:19:05 user nova-compute[72351]: DEBUG oslo.service.loopingcall [None req-45ab9351-d624-4fdc-ade5-5a49e57667ac tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72351) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.compute.manager [-] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Deallocating network for instance {{(pid=72351) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2263}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] deallocate_for_instance() {{(pid=72351) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1795}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:05 user nova-compute[72351]: INFO os_vif [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:3f:be:14,bridge_name='br-int',has_traffic_filtering=True,id=c2df74bf-22ce-48b1-9ab8-3a684cef7266,network=Network(4e0c5236-b30d-4213-b1b4-26c0f1a636eb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc2df74bf-22')
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-08-09T02:18:24Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-180184049',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-180184049',id=9,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNlyDeWP+c+bzipwXZAMO2Z2+8MHps3hDZ0m7Lqwq/td4x0E+p/3ch4cYsc0Ot8OOMGBR+QCJGss0jSgkeH83ijl4d9uevYtC5oVznEoGx5YCooCmb7aiHDwnwmo8gkNlA==',key_name='tempest-keypair-1051869033',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='6908d784cc1a41dbbb4bee47e70c259b',ramdisk_id='',reservation_id='r-rl5gtt0e',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-1303879303',owner_user_name='tempest-TaggedBootDevicesTest-1303879303-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:18:27Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='c74c3939709643c8915c7d1678935e9a',uuid=629e177e-afbb-4765-a0b5-7a3f66f72ebf,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "d1b9314b-1800-41fb-a34b-0186f8e746da", "address": "fa:16:3e:37:27:3b", "network": {"id": "b4ebdbb0-e697-4a42-93f5-946522a3dbce", "bridge": "br-int", "label": "tempest-device-tagging-net2-356335140", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd1b9314b-18", "ovs_interfaceid": "d1b9314b-1800-41fb-a34b-0186f8e746da", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Converting VIF {"id": "d1b9314b-1800-41fb-a34b-0186f8e746da", "address": "fa:16:3e:37:27:3b", "network": {"id": "b4ebdbb0-e697-4a42-93f5-946522a3dbce", "bridge": "br-int", "label": "tempest-device-tagging-net2-356335140", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd1b9314b-18", "ovs_interfaceid": "d1b9314b-1800-41fb-a34b-0186f8e746da", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:37:27:3b,bridge_name='br-int',has_traffic_filtering=True,id=d1b9314b-1800-41fb-a34b-0186f8e746da,network=Network(b4ebdbb0-e697-4a42-93f5-946522a3dbce),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd1b9314b-18') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG os_vif [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:37:27:3b,bridge_name='br-int',has_traffic_filtering=True,id=d1b9314b-1800-41fb-a34b-0186f8e746da,network=Network(b4ebdbb0-e697-4a42-93f5-946522a3dbce),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd1b9314b-18') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapd1b9314b-18, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapd1b9314b-18, col_values=(('external_ids', {'iface-id': 'd1b9314b-1800-41fb-a34b-0186f8e746da', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:37:27:3b', 'vm-uuid': '629e177e-afbb-4765-a0b5-7a3f66f72ebf'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:05 user nova-compute[72351]: INFO os_vif [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:37:27:3b,bridge_name='br-int',has_traffic_filtering=True,id=d1b9314b-1800-41fb-a34b-0186f8e746da,network=Network(b4ebdbb0-e697-4a42-93f5-946522a3dbce),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd1b9314b-18')
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-08-09T02:18:24Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-180184049',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-180184049',id=9,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNlyDeWP+c+bzipwXZAMO2Z2+8MHps3hDZ0m7Lqwq/td4x0E+p/3ch4cYsc0Ot8OOMGBR+QCJGss0jSgkeH83ijl4d9uevYtC5oVznEoGx5YCooCmb7aiHDwnwmo8gkNlA==',key_name='tempest-keypair-1051869033',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='6908d784cc1a41dbbb4bee47e70c259b',ramdisk_id='',reservation_id='r-rl5gtt0e',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-1303879303',owner_user_name='tempest-TaggedBootDevicesTest-1303879303-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:18:27Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='c74c3939709643c8915c7d1678935e9a',uuid=629e177e-afbb-4765-a0b5-7a3f66f72ebf,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "c7c195c0-af76-445e-8a18-8169f0cdf5fc", "address": "fa:16:3e:f7:2d:31", "network": {"id": "b4ebdbb0-e697-4a42-93f5-946522a3dbce", "bridge": "br-int", "label": "tempest-device-tagging-net2-356335140", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc7c195c0-af", "ovs_interfaceid": "c7c195c0-af76-445e-8a18-8169f0cdf5fc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Converting VIF {"id": "c7c195c0-af76-445e-8a18-8169f0cdf5fc", "address": "fa:16:3e:f7:2d:31", "network": {"id": "b4ebdbb0-e697-4a42-93f5-946522a3dbce", "bridge": "br-int", "label": "tempest-device-tagging-net2-356335140", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc7c195c0-af", "ovs_interfaceid": "c7c195c0-af76-445e-8a18-8169f0cdf5fc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:f7:2d:31,bridge_name='br-int',has_traffic_filtering=True,id=c7c195c0-af76-445e-8a18-8169f0cdf5fc,network=Network(b4ebdbb0-e697-4a42-93f5-946522a3dbce),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc7c195c0-af') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG os_vif [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:f7:2d:31,bridge_name='br-int',has_traffic_filtering=True,id=c7c195c0-af76-445e-8a18-8169f0cdf5fc,network=Network(b4ebdbb0-e697-4a42-93f5-946522a3dbce),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc7c195c0-af') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapc7c195c0-af, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapc7c195c0-af, col_values=(('external_ids', {'iface-id': 'c7c195c0-af76-445e-8a18-8169f0cdf5fc', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:f7:2d:31', 'vm-uuid': '629e177e-afbb-4765-a0b5-7a3f66f72ebf'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.network.neutron [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Updating instance_info_cache with network_info: [{"id": "6bfee618-fc3b-43ba-b96f-c75db4b7a4e9", "address": "fa:16:3e:e4:67:dd", "network": {"id": "66067153-d668-48d7-9851-b33abf90f2e1", "bridge": "br-int", "label": "tempest-TestServerVolumeAttachScenarioOldVersion-1194345640-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5fd127f53fc54fdbbf8ae524f3578f91", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6bfee618-fc", "ovs_interfaceid": "6bfee618-fc3b-43ba-b96f-c75db4b7a4e9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Releasing lock "refresh_cache-822e25c4-8dd1-4e21-b0ed-f340d59c6c98" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.compute.manager [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Instance network_info: |[{"id": "6bfee618-fc3b-43ba-b96f-c75db4b7a4e9", "address": "fa:16:3e:e4:67:dd", "network": {"id": "66067153-d668-48d7-9851-b33abf90f2e1", "bridge": "br-int", "label": "tempest-TestServerVolumeAttachScenarioOldVersion-1194345640-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5fd127f53fc54fdbbf8ae524f3578f91", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6bfee618-fc", "ovs_interfaceid": "6bfee618-fc3b-43ba-b96f-c75db4b7a4e9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1971}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-33f60766-b235-4848-9982-70674147dc46 req-87c2e634-cd5b-4d0d-8ad4-e47f1d0a8358 service nova] Acquired lock "refresh_cache-822e25c4-8dd1-4e21-b0ed-f340d59c6c98" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.network.neutron [req-33f60766-b235-4848-9982-70674147dc46 req-87c2e634-cd5b-4d0d-8ad4-e47f1d0a8358 service nova] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Refreshing network info cache for port 6bfee618-fc3b-43ba-b96f-c75db4b7a4e9 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Start _get_guest_xml network_info=[{"id": "6bfee618-fc3b-43ba-b96f-c75db4b7a4e9", "address": "fa:16:3e:e4:67:dd", "network": {"id": "66067153-d668-48d7-9851-b33abf90f2e1", "bridge": "br-int", "label": "tempest-TestServerVolumeAttachScenarioOldVersion-1194345640-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5fd127f53fc54fdbbf8ae524f3578f91", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6bfee618-fc", "ovs_interfaceid": "6bfee618-fc3b-43ba-b96f-c75db4b7a4e9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'size': 0, 'device_type': 'disk', 'encryption_secret_uuid': None, 'guest_format': None, 'encryption_options': None, 'disk_bus': 'virtio', 'encrypted': False, 'encryption_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'image_id': '18ccda7f-31d3-4978-ab9d-4ed2f74eff94'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:19:05 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:19:05 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:19:01Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestServerVolumeAttachScenarioOldVersion-server-1011938981',display_name='tempest-TestServerVolumeAttachScenarioOldVersion-server-1011938981',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testservervolumeattachscenariooldversion-server-1011938',id=11,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGPTAtBLvv+eQnb1GxApCCwfO/b9dFAlv4Z9d8CIpZZkoHUa1BvykzJdDs1KfKyHfsGzGGa6sqh+JXq3NyAXht/DjjCN8xgzJjWBqtfbK66i+iGwapUmiS6V9Np3Mgd9og==',key_name='tempest-keypair-1739372982',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='5fd127f53fc54fdbbf8ae524f3578f91',ramdisk_id='',reservation_id='r-cat7ueqw',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestServerVolumeAttachScenarioOldVersion-59067075',owner_user_name='tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:19:03Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='761c64dbddf443dbb05b2c1211b9a91d',uuid=822e25c4-8dd1-4e21-b0ed-f340d59c6c98,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "6bfee618-fc3b-43ba-b96f-c75db4b7a4e9", "address": "fa:16:3e:e4:67:dd", "network": {"id": "66067153-d668-48d7-9851-b33abf90f2e1", "bridge": "br-int", "label": "tempest-TestServerVolumeAttachScenarioOldVersion-1194345640-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5fd127f53fc54fdbbf8ae524f3578f91", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6bfee618-fc", "ovs_interfaceid": "6bfee618-fc3b-43ba-b96f-c75db4b7a4e9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Converting VIF {"id": "6bfee618-fc3b-43ba-b96f-c75db4b7a4e9", "address": "fa:16:3e:e4:67:dd", "network": {"id": "66067153-d668-48d7-9851-b33abf90f2e1", "bridge": "br-int", "label": "tempest-TestServerVolumeAttachScenarioOldVersion-1194345640-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5fd127f53fc54fdbbf8ae524f3578f91", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6bfee618-fc", "ovs_interfaceid": "6bfee618-fc3b-43ba-b96f-c75db4b7a4e9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:e4:67:dd,bridge_name='br-int',has_traffic_filtering=True,id=6bfee618-fc3b-43ba-b96f-c75db4b7a4e9,network=Network(66067153-d668-48d7-9851-b33abf90f2e1),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6bfee618-fc') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.objects.instance [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Lazy-loading 'pci_devices' on Instance uuid 822e25c4-8dd1-4e21-b0ed-f340d59c6c98 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:05 user nova-compute[72351]: INFO os_vif [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:f7:2d:31,bridge_name='br-int',has_traffic_filtering=True,id=c7c195c0-af76-445e-8a18-8169f0cdf5fc,network=Network(b4ebdbb0-e697-4a42-93f5-946522a3dbce),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc7c195c0-af')
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:19:05 user nova-compute[72351]:   <uuid>822e25c4-8dd1-4e21-b0ed-f340d59c6c98</uuid>
Aug 09 02:19:05 user nova-compute[72351]:   <name>instance-0000000b</name>
Aug 09 02:19:05 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:19:05 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:19:05 user nova-compute[72351]:   <metadata>
Aug 09 02:19:05 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:19:05 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:19:05 user nova-compute[72351]:       <nova:name>tempest-TestServerVolumeAttachScenarioOldVersion-server-1011938981</nova:name>
Aug 09 02:19:05 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:19:05</nova:creationTime>
Aug 09 02:19:05 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:19:05 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:19:05 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:19:05 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:19:05 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:19:05 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:19:05 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:19:05 user nova-compute[72351]:       <nova:owner>
Aug 09 02:19:05 user nova-compute[72351]:         <nova:user uuid="761c64dbddf443dbb05b2c1211b9a91d">tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member</nova:user>
Aug 09 02:19:05 user nova-compute[72351]:         <nova:project uuid="5fd127f53fc54fdbbf8ae524f3578f91">tempest-TestServerVolumeAttachScenarioOldVersion-59067075</nova:project>
Aug 09 02:19:05 user nova-compute[72351]:       </nova:owner>
Aug 09 02:19:05 user nova-compute[72351]:       <nova:root type="image" uuid="18ccda7f-31d3-4978-ab9d-4ed2f74eff94"/>
Aug 09 02:19:05 user nova-compute[72351]:       <nova:ports>
Aug 09 02:19:05 user nova-compute[72351]:         <nova:port uuid="6bfee618-fc3b-43ba-b96f-c75db4b7a4e9">
Aug 09 02:19:05 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.12" ipVersion="4"/>
Aug 09 02:19:05 user nova-compute[72351]:         </nova:port>
Aug 09 02:19:05 user nova-compute[72351]:       </nova:ports>
Aug 09 02:19:05 user nova-compute[72351]:     </nova:instance>
Aug 09 02:19:05 user nova-compute[72351]:   </metadata>
Aug 09 02:19:05 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:19:05 user nova-compute[72351]:     <system>
Aug 09 02:19:05 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:19:05 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:19:05 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:19:05 user nova-compute[72351]:       <entry name="serial">822e25c4-8dd1-4e21-b0ed-f340d59c6c98</entry>
Aug 09 02:19:05 user nova-compute[72351]:       <entry name="uuid">822e25c4-8dd1-4e21-b0ed-f340d59c6c98</entry>
Aug 09 02:19:05 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:19:05 user nova-compute[72351]:     </system>
Aug 09 02:19:05 user nova-compute[72351]:   </sysinfo>
Aug 09 02:19:05 user nova-compute[72351]:   <os>
Aug 09 02:19:05 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:19:05 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:19:05 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:19:05 user nova-compute[72351]:   </os>
Aug 09 02:19:05 user nova-compute[72351]:   <features>
Aug 09 02:19:05 user nova-compute[72351]:     <acpi/>
Aug 09 02:19:05 user nova-compute[72351]:     <apic/>
Aug 09 02:19:05 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:19:05 user nova-compute[72351]:   </features>
Aug 09 02:19:05 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:19:05 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:19:05 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:19:05 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:19:05 user nova-compute[72351]:   </clock>
Aug 09 02:19:05 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:19:05 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:19:05 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:19:05 user nova-compute[72351]:   </cpu>
Aug 09 02:19:05 user nova-compute[72351]:   <devices>
Aug 09 02:19:05 user nova-compute[72351]:     <disk type="file" device="disk">
Aug 09 02:19:05 user nova-compute[72351]:       <driver name="qemu" type="qcow2" cache="none"/>
Aug 09 02:19:05 user nova-compute[72351]:       <source file="/opt/stack/data/nova/instances/822e25c4-8dd1-4e21-b0ed-f340d59c6c98/disk"/>
Aug 09 02:19:05 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:19:05 user nova-compute[72351]:     </disk>
Aug 09 02:19:05 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:19:05 user nova-compute[72351]:       <mac address="fa:16:3e:e4:67:dd"/>
Aug 09 02:19:05 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:19:05 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:19:05 user nova-compute[72351]:       <target dev="tap6bfee618-fc"/>
Aug 09 02:19:05 user nova-compute[72351]:     </interface>
Aug 09 02:19:05 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:19:05 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/822e25c4-8dd1-4e21-b0ed-f340d59c6c98/console.log" append="off"/>
Aug 09 02:19:05 user nova-compute[72351]:     </serial>
Aug 09 02:19:05 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:19:05 user nova-compute[72351]:     <video>
Aug 09 02:19:05 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:19:05 user nova-compute[72351]:     </video>
Aug 09 02:19:05 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:19:05 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:19:05 user nova-compute[72351]:     </rng>
Aug 09 02:19:05 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:19:05 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:19:05 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:19:05 user nova-compute[72351]:     </memballoon>
Aug 09 02:19:05 user nova-compute[72351]:   </devices>
Aug 09 02:19:05 user nova-compute[72351]: </domain>
Aug 09 02:19:05 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:19:01Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestServerVolumeAttachScenarioOldVersion-server-1011938981',display_name='tempest-TestServerVolumeAttachScenarioOldVersion-server-1011938981',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testservervolumeattachscenariooldversion-server-1011938',id=11,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGPTAtBLvv+eQnb1GxApCCwfO/b9dFAlv4Z9d8CIpZZkoHUa1BvykzJdDs1KfKyHfsGzGGa6sqh+JXq3NyAXht/DjjCN8xgzJjWBqtfbK66i+iGwapUmiS6V9Np3Mgd9og==',key_name='tempest-keypair-1739372982',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='5fd127f53fc54fdbbf8ae524f3578f91',ramdisk_id='',reservation_id='r-cat7ueqw',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestServerVolumeAttachScenarioOldVersion-59067075',owner_user_name='tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:19:03Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='761c64dbddf443dbb05b2c1211b9a91d',uuid=822e25c4-8dd1-4e21-b0ed-f340d59c6c98,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "6bfee618-fc3b-43ba-b96f-c75db4b7a4e9", "address": "fa:16:3e:e4:67:dd", "network": {"id": "66067153-d668-48d7-9851-b33abf90f2e1", "bridge": "br-int", "label": "tempest-TestServerVolumeAttachScenarioOldVersion-1194345640-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5fd127f53fc54fdbbf8ae524f3578f91", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6bfee618-fc", "ovs_interfaceid": "6bfee618-fc3b-43ba-b96f-c75db4b7a4e9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Converting VIF {"id": "6bfee618-fc3b-43ba-b96f-c75db4b7a4e9", "address": "fa:16:3e:e4:67:dd", "network": {"id": "66067153-d668-48d7-9851-b33abf90f2e1", "bridge": "br-int", "label": "tempest-TestServerVolumeAttachScenarioOldVersion-1194345640-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5fd127f53fc54fdbbf8ae524f3578f91", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6bfee618-fc", "ovs_interfaceid": "6bfee618-fc3b-43ba-b96f-c75db4b7a4e9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:e4:67:dd,bridge_name='br-int',has_traffic_filtering=True,id=6bfee618-fc3b-43ba-b96f-c75db4b7a4e9,network=Network(66067153-d668-48d7-9851-b33abf90f2e1),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6bfee618-fc') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG os_vif [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:e4:67:dd,bridge_name='br-int',has_traffic_filtering=True,id=6bfee618-fc3b-43ba-b96f-c75db4b7a4e9,network=Network(66067153-d668-48d7-9851-b33abf90f2e1),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6bfee618-fc') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap6bfee618-fc, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap6bfee618-fc, col_values=(('external_ids', {'iface-id': '6bfee618-fc3b-43ba-b96f-c75db4b7a4e9', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:e4:67:dd', 'vm-uuid': '822e25c4-8dd1-4e21-b0ed-f340d59c6c98'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] No BDM found with device name hda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] No BDM found with device name vdc, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] No VIF found with MAC fa:16:3e:60:29:2e, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:19:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] No VIF found with MAC fa:16:3e:3f:be:14, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:19:05 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Using config drive
Aug 09 02:19:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:06 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:06 user nova-compute[72351]: INFO os_vif [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:e4:67:dd,bridge_name='br-int',has_traffic_filtering=True,id=6bfee618-fc3b-43ba-b96f-c75db4b7a4e9,network=Network(66067153-d668-48d7-9851-b33abf90f2e1),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6bfee618-fc')
Aug 09 02:19:06 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:19:06 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] No VIF found with MAC fa:16:3e:e4:67:dd, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:19:06 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Creating config drive at /opt/stack/data/nova/instances/629e177e-afbb-4765-a0b5-7a3f66f72ebf/disk.config
Aug 09 02:19:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Running cmd (subprocess): genisoimage -o /opt/stack/data/nova/instances/629e177e-afbb-4765-a0b5-7a3f66f72ebf/disk.config -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 0.0.0 -quiet -J -r -V config-2 /tmp/tmpp3g8d960 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] CMD "genisoimage -o /opt/stack/data/nova/instances/629e177e-afbb-4765-a0b5-7a3f66f72ebf/disk.config -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 0.0.0 -quiet -J -r -V config-2 /tmp/tmpp3g8d960" returned: 0 in 0.059s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:06 user nova-compute[72351]: DEBUG nova.compute.manager [req-f0a3fe3d-dfa0-4b0c-9cec-34302690b4cd req-1ada9faf-fc15-4425-9bd6-449ffd2c079c service nova] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Received event network-vif-deleted-e9a7bda3-90fa-4274-acc9-3597835503e9 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:19:06 user nova-compute[72351]: INFO nova.compute.manager [req-f0a3fe3d-dfa0-4b0c-9cec-34302690b4cd req-1ada9faf-fc15-4425-9bd6-449ffd2c079c service nova] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Neutron deleted interface e9a7bda3-90fa-4274-acc9-3597835503e9; detaching it from the instance and deleting it from the info cache
Aug 09 02:19:06 user nova-compute[72351]: DEBUG nova.network.neutron [req-f0a3fe3d-dfa0-4b0c-9cec-34302690b4cd req-1ada9faf-fc15-4425-9bd6-449ffd2c079c service nova] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:19:06 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:19:06 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Took 0.76 seconds to deallocate network for instance.
Aug 09 02:19:06 user nova-compute[72351]: DEBUG nova.compute.manager [req-f0a3fe3d-dfa0-4b0c-9cec-34302690b4cd req-1ada9faf-fc15-4425-9bd6-449ffd2c079c service nova] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Detach interface failed, port_id=e9a7bda3-90fa-4274-acc9-3597835503e9, reason: Instance f45b57fa-9264-45f0-9573-e5fb58848d1f could not be found. {{(pid=72351) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10866}}
Aug 09 02:19:06 user nova-compute[72351]: DEBUG nova.network.neutron [req-33f60766-b235-4848-9982-70674147dc46 req-87c2e634-cd5b-4d0d-8ad4-e47f1d0a8358 service nova] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Updated VIF entry in instance network info cache for port 6bfee618-fc3b-43ba-b96f-c75db4b7a4e9. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:19:06 user nova-compute[72351]: DEBUG nova.network.neutron [req-33f60766-b235-4848-9982-70674147dc46 req-87c2e634-cd5b-4d0d-8ad4-e47f1d0a8358 service nova] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Updating instance_info_cache with network_info: [{"id": "6bfee618-fc3b-43ba-b96f-c75db4b7a4e9", "address": "fa:16:3e:e4:67:dd", "network": {"id": "66067153-d668-48d7-9851-b33abf90f2e1", "bridge": "br-int", "label": "tempest-TestServerVolumeAttachScenarioOldVersion-1194345640-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5fd127f53fc54fdbbf8ae524f3578f91", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6bfee618-fc", "ovs_interfaceid": "6bfee618-fc3b-43ba-b96f-c75db4b7a4e9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:19:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-33f60766-b235-4848-9982-70674147dc46 req-87c2e634-cd5b-4d0d-8ad4-e47f1d0a8358 service nova] Releasing lock "refresh_cache-822e25c4-8dd1-4e21-b0ed-f340d59c6c98" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:19:06 user nova-compute[72351]: DEBUG nova.compute.manager [req-5d67548c-8d43-4d20-a26e-9a672bf51855 req-a6a6af75-696b-4974-bf65-c87314439c47 service nova] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Received event network-vif-plugged-49802a57-3939-4cc2-956b-3d396b7c738e {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:19:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-5d67548c-8d43-4d20-a26e-9a672bf51855 req-a6a6af75-696b-4974-bf65-c87314439c47 service nova] Acquiring lock "a5ca6400-1125-431e-8a91-057e746d40b5-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-5d67548c-8d43-4d20-a26e-9a672bf51855 req-a6a6af75-696b-4974-bf65-c87314439c47 service nova] Lock "a5ca6400-1125-431e-8a91-057e746d40b5-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-5d67548c-8d43-4d20-a26e-9a672bf51855 req-a6a6af75-696b-4974-bf65-c87314439c47 service nova] Lock "a5ca6400-1125-431e-8a91-057e746d40b5-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:06 user nova-compute[72351]: DEBUG nova.compute.manager [req-5d67548c-8d43-4d20-a26e-9a672bf51855 req-a6a6af75-696b-4974-bf65-c87314439c47 service nova] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] No waiting events found dispatching network-vif-plugged-49802a57-3939-4cc2-956b-3d396b7c738e {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:19:06 user nova-compute[72351]: WARNING nova.compute.manager [req-5d67548c-8d43-4d20-a26e-9a672bf51855 req-a6a6af75-696b-4974-bf65-c87314439c47 service nova] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Received unexpected event network-vif-plugged-49802a57-3939-4cc2-956b-3d396b7c738e for instance with vm_state building and task_state spawning.
Aug 09 02:19:06 user nova-compute[72351]: DEBUG nova.compute.manager [req-5d67548c-8d43-4d20-a26e-9a672bf51855 req-a6a6af75-696b-4974-bf65-c87314439c47 service nova] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Received event network-vif-plugged-49802a57-3939-4cc2-956b-3d396b7c738e {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:19:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-5d67548c-8d43-4d20-a26e-9a672bf51855 req-a6a6af75-696b-4974-bf65-c87314439c47 service nova] Acquiring lock "a5ca6400-1125-431e-8a91-057e746d40b5-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-5d67548c-8d43-4d20-a26e-9a672bf51855 req-a6a6af75-696b-4974-bf65-c87314439c47 service nova] Lock "a5ca6400-1125-431e-8a91-057e746d40b5-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-5d67548c-8d43-4d20-a26e-9a672bf51855 req-a6a6af75-696b-4974-bf65-c87314439c47 service nova] Lock "a5ca6400-1125-431e-8a91-057e746d40b5-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:06 user nova-compute[72351]: DEBUG nova.compute.manager [req-5d67548c-8d43-4d20-a26e-9a672bf51855 req-a6a6af75-696b-4974-bf65-c87314439c47 service nova] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] No waiting events found dispatching network-vif-plugged-49802a57-3939-4cc2-956b-3d396b7c738e {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:19:06 user nova-compute[72351]: WARNING nova.compute.manager [req-5d67548c-8d43-4d20-a26e-9a672bf51855 req-a6a6af75-696b-4974-bf65-c87314439c47 service nova] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Received unexpected event network-vif-plugged-49802a57-3939-4cc2-956b-3d396b7c738e for instance with vm_state building and task_state spawning.
Aug 09 02:19:07 user nova-compute[72351]: INFO nova.compute.manager [None req-45ab9351-d624-4fdc-ade5-5a49e57667ac tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Took 0.86 seconds to detach 1 volumes for instance.
Aug 09 02:19:07 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-45ab9351-d624-4fdc-ade5-5a49e57667ac tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:07 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-45ab9351-d624-4fdc-ade5-5a49e57667ac tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:07 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547547.3485217, a5ca6400-1125-431e-8a91-057e746d40b5 => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:19:07 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] VM Resumed (Lifecycle Event)
Aug 09 02:19:07 user nova-compute[72351]: DEBUG nova.compute.manager [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:19:07 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Guest created on hypervisor {{(pid=72351) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4411}}
Aug 09 02:19:07 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:19:07 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Instance spawned successfully.
Aug 09 02:19:07 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:894}}
Aug 09 02:19:07 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:19:07 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Found default for hw_cdrom_bus of ide {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:19:07 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Found default for hw_disk_bus of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:19:07 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Found default for hw_input_bus of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:19:07 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Found default for hw_pointer_model of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:19:07 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Found default for hw_video_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:19:07 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Found default for hw_vif_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:19:07 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:19:07 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547547.351403, a5ca6400-1125-431e-8a91-057e746d40b5 => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:19:07 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] VM Started (Lifecycle Event)
Aug 09 02:19:07 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:19:07 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:19:07 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:07 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:19:07 user nova-compute[72351]: INFO nova.compute.manager [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Took 7.56 seconds to spawn the instance on the hypervisor.
Aug 09 02:19:07 user nova-compute[72351]: DEBUG nova.compute.manager [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:19:07 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-45ab9351-d624-4fdc-ade5-5a49e57667ac tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:19:07 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-45ab9351-d624-4fdc-ade5-5a49e57667ac tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:19:07 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-45ab9351-d624-4fdc-ade5-5a49e57667ac tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.370s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:07 user nova-compute[72351]: INFO nova.compute.manager [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Took 23.63 seconds to build instance.
Aug 09 02:19:07 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-096c43b3-c74c-4da6-a070-5e4a032c631b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "a5ca6400-1125-431e-8a91-057e746d40b5" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 23.729s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:07 user nova-compute[72351]: INFO nova.scheduler.client.report [None req-45ab9351-d624-4fdc-ade5-5a49e57667ac tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Deleted allocations for instance f45b57fa-9264-45f0-9573-e5fb58848d1f
Aug 09 02:19:07 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-45ab9351-d624-4fdc-ade5-5a49e57667ac tempest-DeleteServersTestJSON-845954460 tempest-DeleteServersTestJSON-845954460-project-member] Lock "f45b57fa-9264-45f0-9573-e5fb58848d1f" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 5.251s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:08 user nova-compute[72351]: DEBUG nova.compute.manager [req-7ca45e16-af5a-445f-a341-92118a94fcba req-1ee368ca-608c-4819-a19f-6868af560f58 service nova] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Received event network-vif-plugged-6bfee618-fc3b-43ba-b96f-c75db4b7a4e9 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:19:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-7ca45e16-af5a-445f-a341-92118a94fcba req-1ee368ca-608c-4819-a19f-6868af560f58 service nova] Acquiring lock "822e25c4-8dd1-4e21-b0ed-f340d59c6c98-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-7ca45e16-af5a-445f-a341-92118a94fcba req-1ee368ca-608c-4819-a19f-6868af560f58 service nova] Lock "822e25c4-8dd1-4e21-b0ed-f340d59c6c98-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-7ca45e16-af5a-445f-a341-92118a94fcba req-1ee368ca-608c-4819-a19f-6868af560f58 service nova] Lock "822e25c4-8dd1-4e21-b0ed-f340d59c6c98-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:08 user nova-compute[72351]: DEBUG nova.compute.manager [req-7ca45e16-af5a-445f-a341-92118a94fcba req-1ee368ca-608c-4819-a19f-6868af560f58 service nova] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] No waiting events found dispatching network-vif-plugged-6bfee618-fc3b-43ba-b96f-c75db4b7a4e9 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:19:08 user nova-compute[72351]: WARNING nova.compute.manager [req-7ca45e16-af5a-445f-a341-92118a94fcba req-1ee368ca-608c-4819-a19f-6868af560f58 service nova] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Received unexpected event network-vif-plugged-6bfee618-fc3b-43ba-b96f-c75db4b7a4e9 for instance with vm_state building and task_state spawning.
Aug 09 02:19:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:09 user nova-compute[72351]: DEBUG nova.compute.manager [req-238a9988-7363-41a8-b1c8-90b2b1ecb99b req-66753f52-4af6-4400-b045-297634b33028 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received event network-vif-plugged-e021d67e-09ab-4779-b639-dff1ba39798a {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:19:09 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-238a9988-7363-41a8-b1c8-90b2b1ecb99b req-66753f52-4af6-4400-b045-297634b33028 service nova] Acquiring lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:09 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-238a9988-7363-41a8-b1c8-90b2b1ecb99b req-66753f52-4af6-4400-b045-297634b33028 service nova] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:09 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-238a9988-7363-41a8-b1c8-90b2b1ecb99b req-66753f52-4af6-4400-b045-297634b33028 service nova] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:09 user nova-compute[72351]: DEBUG nova.compute.manager [req-238a9988-7363-41a8-b1c8-90b2b1ecb99b req-66753f52-4af6-4400-b045-297634b33028 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] No waiting events found dispatching network-vif-plugged-e021d67e-09ab-4779-b639-dff1ba39798a {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:19:09 user nova-compute[72351]: WARNING nova.compute.manager [req-238a9988-7363-41a8-b1c8-90b2b1ecb99b req-66753f52-4af6-4400-b045-297634b33028 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received unexpected event network-vif-plugged-e021d67e-09ab-4779-b639-dff1ba39798a for instance with vm_state building and task_state spawning.
Aug 09 02:19:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:10 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:10 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:10 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:10 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:10 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:10 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:10 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:10 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:10 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:10 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:10 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:10 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:10 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:10 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:10 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:11 user nova-compute[72351]: DEBUG nova.compute.manager [req-67c8a96a-f9ec-45ac-a666-ef50852756f4 req-bc01fd68-6636-4ccc-8606-43c645908bc9 service nova] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Received event network-vif-plugged-6bfee618-fc3b-43ba-b96f-c75db4b7a4e9 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:19:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-67c8a96a-f9ec-45ac-a666-ef50852756f4 req-bc01fd68-6636-4ccc-8606-43c645908bc9 service nova] Acquiring lock "822e25c4-8dd1-4e21-b0ed-f340d59c6c98-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-67c8a96a-f9ec-45ac-a666-ef50852756f4 req-bc01fd68-6636-4ccc-8606-43c645908bc9 service nova] Lock "822e25c4-8dd1-4e21-b0ed-f340d59c6c98-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-67c8a96a-f9ec-45ac-a666-ef50852756f4 req-bc01fd68-6636-4ccc-8606-43c645908bc9 service nova] Lock "822e25c4-8dd1-4e21-b0ed-f340d59c6c98-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:11 user nova-compute[72351]: DEBUG nova.compute.manager [req-67c8a96a-f9ec-45ac-a666-ef50852756f4 req-bc01fd68-6636-4ccc-8606-43c645908bc9 service nova] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] No waiting events found dispatching network-vif-plugged-6bfee618-fc3b-43ba-b96f-c75db4b7a4e9 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:19:11 user nova-compute[72351]: WARNING nova.compute.manager [req-67c8a96a-f9ec-45ac-a666-ef50852756f4 req-bc01fd68-6636-4ccc-8606-43c645908bc9 service nova] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Received unexpected event network-vif-plugged-6bfee618-fc3b-43ba-b96f-c75db4b7a4e9 for instance with vm_state building and task_state spawning.
Aug 09 02:19:11 user nova-compute[72351]: DEBUG nova.compute.manager [req-67c8a96a-f9ec-45ac-a666-ef50852756f4 req-bc01fd68-6636-4ccc-8606-43c645908bc9 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received event network-vif-plugged-f5ecb594-7ac3-496b-9bb0-9f908b91c8fb {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:19:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-67c8a96a-f9ec-45ac-a666-ef50852756f4 req-bc01fd68-6636-4ccc-8606-43c645908bc9 service nova] Acquiring lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-67c8a96a-f9ec-45ac-a666-ef50852756f4 req-bc01fd68-6636-4ccc-8606-43c645908bc9 service nova] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-67c8a96a-f9ec-45ac-a666-ef50852756f4 req-bc01fd68-6636-4ccc-8606-43c645908bc9 service nova] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:11 user nova-compute[72351]: DEBUG nova.compute.manager [req-67c8a96a-f9ec-45ac-a666-ef50852756f4 req-bc01fd68-6636-4ccc-8606-43c645908bc9 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] No waiting events found dispatching network-vif-plugged-f5ecb594-7ac3-496b-9bb0-9f908b91c8fb {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:19:11 user nova-compute[72351]: WARNING nova.compute.manager [req-67c8a96a-f9ec-45ac-a666-ef50852756f4 req-bc01fd68-6636-4ccc-8606-43c645908bc9 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received unexpected event network-vif-plugged-f5ecb594-7ac3-496b-9bb0-9f908b91c8fb for instance with vm_state building and task_state spawning.
Aug 09 02:19:11 user nova-compute[72351]: DEBUG nova.compute.manager [req-67c8a96a-f9ec-45ac-a666-ef50852756f4 req-bc01fd68-6636-4ccc-8606-43c645908bc9 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received event network-vif-plugged-f5ecb594-7ac3-496b-9bb0-9f908b91c8fb {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:19:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-67c8a96a-f9ec-45ac-a666-ef50852756f4 req-bc01fd68-6636-4ccc-8606-43c645908bc9 service nova] Acquiring lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-67c8a96a-f9ec-45ac-a666-ef50852756f4 req-bc01fd68-6636-4ccc-8606-43c645908bc9 service nova] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-67c8a96a-f9ec-45ac-a666-ef50852756f4 req-bc01fd68-6636-4ccc-8606-43c645908bc9 service nova] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:11 user nova-compute[72351]: DEBUG nova.compute.manager [req-67c8a96a-f9ec-45ac-a666-ef50852756f4 req-bc01fd68-6636-4ccc-8606-43c645908bc9 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] No waiting events found dispatching network-vif-plugged-f5ecb594-7ac3-496b-9bb0-9f908b91c8fb {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:19:11 user nova-compute[72351]: WARNING nova.compute.manager [req-67c8a96a-f9ec-45ac-a666-ef50852756f4 req-bc01fd68-6636-4ccc-8606-43c645908bc9 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received unexpected event network-vif-plugged-f5ecb594-7ac3-496b-9bb0-9f908b91c8fb for instance with vm_state building and task_state spawning.
Aug 09 02:19:11 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:11 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:11 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:11 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:11 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:11 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:11 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:11 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:11 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:11 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:11 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:11 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:11 user nova-compute[72351]: DEBUG nova.compute.manager [req-fea6cbfe-ede0-46e4-9819-23f6f912b2a4 req-e39eb74a-7015-4984-a0d5-62eda2692d42 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received event network-vif-plugged-e021d67e-09ab-4779-b639-dff1ba39798a {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:19:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-fea6cbfe-ede0-46e4-9819-23f6f912b2a4 req-e39eb74a-7015-4984-a0d5-62eda2692d42 service nova] Acquiring lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-fea6cbfe-ede0-46e4-9819-23f6f912b2a4 req-e39eb74a-7015-4984-a0d5-62eda2692d42 service nova] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-fea6cbfe-ede0-46e4-9819-23f6f912b2a4 req-e39eb74a-7015-4984-a0d5-62eda2692d42 service nova] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:11 user nova-compute[72351]: DEBUG nova.compute.manager [req-fea6cbfe-ede0-46e4-9819-23f6f912b2a4 req-e39eb74a-7015-4984-a0d5-62eda2692d42 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] No waiting events found dispatching network-vif-plugged-e021d67e-09ab-4779-b639-dff1ba39798a {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:19:11 user nova-compute[72351]: WARNING nova.compute.manager [req-fea6cbfe-ede0-46e4-9819-23f6f912b2a4 req-e39eb74a-7015-4984-a0d5-62eda2692d42 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received unexpected event network-vif-plugged-e021d67e-09ab-4779-b639-dff1ba39798a for instance with vm_state building and task_state spawning.
Aug 09 02:19:11 user nova-compute[72351]: DEBUG nova.compute.manager [req-fea6cbfe-ede0-46e4-9819-23f6f912b2a4 req-e39eb74a-7015-4984-a0d5-62eda2692d42 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received event network-vif-plugged-2e9c8a9c-2d81-4b84-b481-9a6b513e730c {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:19:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-fea6cbfe-ede0-46e4-9819-23f6f912b2a4 req-e39eb74a-7015-4984-a0d5-62eda2692d42 service nova] Acquiring lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-fea6cbfe-ede0-46e4-9819-23f6f912b2a4 req-e39eb74a-7015-4984-a0d5-62eda2692d42 service nova] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-fea6cbfe-ede0-46e4-9819-23f6f912b2a4 req-e39eb74a-7015-4984-a0d5-62eda2692d42 service nova] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:11 user nova-compute[72351]: DEBUG nova.compute.manager [req-fea6cbfe-ede0-46e4-9819-23f6f912b2a4 req-e39eb74a-7015-4984-a0d5-62eda2692d42 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] No waiting events found dispatching network-vif-plugged-2e9c8a9c-2d81-4b84-b481-9a6b513e730c {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:19:11 user nova-compute[72351]: WARNING nova.compute.manager [req-fea6cbfe-ede0-46e4-9819-23f6f912b2a4 req-e39eb74a-7015-4984-a0d5-62eda2692d42 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received unexpected event network-vif-plugged-2e9c8a9c-2d81-4b84-b481-9a6b513e730c for instance with vm_state building and task_state spawning.
Aug 09 02:19:11 user nova-compute[72351]: DEBUG nova.compute.manager [req-fea6cbfe-ede0-46e4-9819-23f6f912b2a4 req-e39eb74a-7015-4984-a0d5-62eda2692d42 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received event network-vif-plugged-2e9c8a9c-2d81-4b84-b481-9a6b513e730c {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:19:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-fea6cbfe-ede0-46e4-9819-23f6f912b2a4 req-e39eb74a-7015-4984-a0d5-62eda2692d42 service nova] Acquiring lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-fea6cbfe-ede0-46e4-9819-23f6f912b2a4 req-e39eb74a-7015-4984-a0d5-62eda2692d42 service nova] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-fea6cbfe-ede0-46e4-9819-23f6f912b2a4 req-e39eb74a-7015-4984-a0d5-62eda2692d42 service nova] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:11 user nova-compute[72351]: DEBUG nova.compute.manager [req-fea6cbfe-ede0-46e4-9819-23f6f912b2a4 req-e39eb74a-7015-4984-a0d5-62eda2692d42 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] No waiting events found dispatching network-vif-plugged-2e9c8a9c-2d81-4b84-b481-9a6b513e730c {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:19:11 user nova-compute[72351]: WARNING nova.compute.manager [req-fea6cbfe-ede0-46e4-9819-23f6f912b2a4 req-e39eb74a-7015-4984-a0d5-62eda2692d42 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received unexpected event network-vif-plugged-2e9c8a9c-2d81-4b84-b481-9a6b513e730c for instance with vm_state building and task_state spawning.
Aug 09 02:19:11 user nova-compute[72351]: DEBUG nova.compute.manager [req-fea6cbfe-ede0-46e4-9819-23f6f912b2a4 req-e39eb74a-7015-4984-a0d5-62eda2692d42 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received event network-vif-plugged-5d0bb297-60b1-4e66-8f43-3bf1c32ae4ef {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:19:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-fea6cbfe-ede0-46e4-9819-23f6f912b2a4 req-e39eb74a-7015-4984-a0d5-62eda2692d42 service nova] Acquiring lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-fea6cbfe-ede0-46e4-9819-23f6f912b2a4 req-e39eb74a-7015-4984-a0d5-62eda2692d42 service nova] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-fea6cbfe-ede0-46e4-9819-23f6f912b2a4 req-e39eb74a-7015-4984-a0d5-62eda2692d42 service nova] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:11 user nova-compute[72351]: DEBUG nova.compute.manager [req-fea6cbfe-ede0-46e4-9819-23f6f912b2a4 req-e39eb74a-7015-4984-a0d5-62eda2692d42 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] No waiting events found dispatching network-vif-plugged-5d0bb297-60b1-4e66-8f43-3bf1c32ae4ef {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:19:11 user nova-compute[72351]: WARNING nova.compute.manager [req-fea6cbfe-ede0-46e4-9819-23f6f912b2a4 req-e39eb74a-7015-4984-a0d5-62eda2692d42 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received unexpected event network-vif-plugged-5d0bb297-60b1-4e66-8f43-3bf1c32ae4ef for instance with vm_state building and task_state spawning.
Aug 09 02:19:11 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547551.7777596, 822e25c4-8dd1-4e21-b0ed-f340d59c6c98 => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:19:11 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] VM Resumed (Lifecycle Event)
Aug 09 02:19:11 user nova-compute[72351]: DEBUG nova.compute.manager [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:19:11 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Guest created on hypervisor {{(pid=72351) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4411}}
Aug 09 02:19:11 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Instance spawned successfully.
Aug 09 02:19:11 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:894}}
Aug 09 02:19:11 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:19:11 user nova-compute[72351]: DEBUG nova.compute.manager [req-11d9bf94-14fe-4ebd-9a1b-bfdbd444ecaf req-a63f15dd-c742-4e01-ac71-018f15baf76f service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received event network-vif-plugged-c2df74bf-22ce-48b1-9ab8-3a684cef7266 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:19:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-11d9bf94-14fe-4ebd-9a1b-bfdbd444ecaf req-a63f15dd-c742-4e01-ac71-018f15baf76f service nova] Acquiring lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-11d9bf94-14fe-4ebd-9a1b-bfdbd444ecaf req-a63f15dd-c742-4e01-ac71-018f15baf76f service nova] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-11d9bf94-14fe-4ebd-9a1b-bfdbd444ecaf req-a63f15dd-c742-4e01-ac71-018f15baf76f service nova] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:11 user nova-compute[72351]: DEBUG nova.compute.manager [req-11d9bf94-14fe-4ebd-9a1b-bfdbd444ecaf req-a63f15dd-c742-4e01-ac71-018f15baf76f service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] No waiting events found dispatching network-vif-plugged-c2df74bf-22ce-48b1-9ab8-3a684cef7266 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:19:11 user nova-compute[72351]: WARNING nova.compute.manager [req-11d9bf94-14fe-4ebd-9a1b-bfdbd444ecaf req-a63f15dd-c742-4e01-ac71-018f15baf76f service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received unexpected event network-vif-plugged-c2df74bf-22ce-48b1-9ab8-3a684cef7266 for instance with vm_state building and task_state spawning.
Aug 09 02:19:11 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:19:11 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Found default for hw_cdrom_bus of ide {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:19:11 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Found default for hw_disk_bus of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:19:11 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Found default for hw_input_bus of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:19:11 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Found default for hw_pointer_model of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:19:11 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Found default for hw_video_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:19:11 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Found default for hw_vif_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:19:11 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:11 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:19:11 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547551.7821667, 822e25c4-8dd1-4e21-b0ed-f340d59c6c98 => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:19:11 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] VM Started (Lifecycle Event)
Aug 09 02:19:11 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:11 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:19:11 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:19:11 user nova-compute[72351]: INFO nova.compute.manager [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Took 8.59 seconds to spawn the instance on the hypervisor.
Aug 09 02:19:11 user nova-compute[72351]: DEBUG nova.compute.manager [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:19:11 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:19:12 user nova-compute[72351]: INFO nova.compute.manager [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Took 9.62 seconds to build instance.
Aug 09 02:19:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c594fffc-9774-4b15-84b1-f17ba940404e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Lock "822e25c4-8dd1-4e21-b0ed-f340d59c6c98" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 9.725s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:12 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:12 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:12 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:12 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:12 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:13 user nova-compute[72351]: DEBUG nova.compute.manager [req-7718a720-489d-4635-833f-276da0a2f653 req-3a359a99-ad4a-4792-afbc-c6f2d24ddf04 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received event network-vif-plugged-d1b9314b-1800-41fb-a34b-0186f8e746da {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:19:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-7718a720-489d-4635-833f-276da0a2f653 req-3a359a99-ad4a-4792-afbc-c6f2d24ddf04 service nova] Acquiring lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-7718a720-489d-4635-833f-276da0a2f653 req-3a359a99-ad4a-4792-afbc-c6f2d24ddf04 service nova] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-7718a720-489d-4635-833f-276da0a2f653 req-3a359a99-ad4a-4792-afbc-c6f2d24ddf04 service nova] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:13 user nova-compute[72351]: DEBUG nova.compute.manager [req-7718a720-489d-4635-833f-276da0a2f653 req-3a359a99-ad4a-4792-afbc-c6f2d24ddf04 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] No waiting events found dispatching network-vif-plugged-d1b9314b-1800-41fb-a34b-0186f8e746da {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:19:13 user nova-compute[72351]: WARNING nova.compute.manager [req-7718a720-489d-4635-833f-276da0a2f653 req-3a359a99-ad4a-4792-afbc-c6f2d24ddf04 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received unexpected event network-vif-plugged-d1b9314b-1800-41fb-a34b-0186f8e746da for instance with vm_state building and task_state spawning.
Aug 09 02:19:13 user nova-compute[72351]: DEBUG nova.compute.manager [req-7718a720-489d-4635-833f-276da0a2f653 req-3a359a99-ad4a-4792-afbc-c6f2d24ddf04 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received event network-vif-plugged-d1b9314b-1800-41fb-a34b-0186f8e746da {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:19:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-7718a720-489d-4635-833f-276da0a2f653 req-3a359a99-ad4a-4792-afbc-c6f2d24ddf04 service nova] Acquiring lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-7718a720-489d-4635-833f-276da0a2f653 req-3a359a99-ad4a-4792-afbc-c6f2d24ddf04 service nova] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-7718a720-489d-4635-833f-276da0a2f653 req-3a359a99-ad4a-4792-afbc-c6f2d24ddf04 service nova] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:13 user nova-compute[72351]: DEBUG nova.compute.manager [req-7718a720-489d-4635-833f-276da0a2f653 req-3a359a99-ad4a-4792-afbc-c6f2d24ddf04 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] No waiting events found dispatching network-vif-plugged-d1b9314b-1800-41fb-a34b-0186f8e746da {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:19:13 user nova-compute[72351]: WARNING nova.compute.manager [req-7718a720-489d-4635-833f-276da0a2f653 req-3a359a99-ad4a-4792-afbc-c6f2d24ddf04 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received unexpected event network-vif-plugged-d1b9314b-1800-41fb-a34b-0186f8e746da for instance with vm_state building and task_state spawning.
Aug 09 02:19:13 user nova-compute[72351]: DEBUG nova.compute.manager [req-7718a720-489d-4635-833f-276da0a2f653 req-3a359a99-ad4a-4792-afbc-c6f2d24ddf04 service nova] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Received event network-changed-a309cd5b-1679-48c6-a596-168b3af2d284 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:19:13 user nova-compute[72351]: DEBUG nova.compute.manager [req-7718a720-489d-4635-833f-276da0a2f653 req-3a359a99-ad4a-4792-afbc-c6f2d24ddf04 service nova] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Refreshing instance network info cache due to event network-changed-a309cd5b-1679-48c6-a596-168b3af2d284. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:19:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-7718a720-489d-4635-833f-276da0a2f653 req-3a359a99-ad4a-4792-afbc-c6f2d24ddf04 service nova] Acquiring lock "refresh_cache-11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:19:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-7718a720-489d-4635-833f-276da0a2f653 req-3a359a99-ad4a-4792-afbc-c6f2d24ddf04 service nova] Acquired lock "refresh_cache-11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:19:13 user nova-compute[72351]: DEBUG nova.network.neutron [req-7718a720-489d-4635-833f-276da0a2f653 req-3a359a99-ad4a-4792-afbc-c6f2d24ddf04 service nova] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Refreshing network info cache for port a309cd5b-1679-48c6-a596-168b3af2d284 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:19:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-8a1e58e1-dcd8-4708-b13e-b997dfbd1229 tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Acquiring lock "11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-8a1e58e1-dcd8-4708-b13e-b997dfbd1229 tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Lock "11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:13 user nova-compute[72351]: DEBUG nova.objects.instance [None req-8a1e58e1-dcd8-4708-b13e-b997dfbd1229 tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Lazy-loading 'flavor' on Instance uuid 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:19:13 user nova-compute[72351]: DEBUG nova.compute.manager [req-0f2781e1-3558-4820-b3f3-1502aeb9c2f7 req-51c4cff7-28b9-41cc-908c-295be7b1c074 service nova] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Received event network-changed-3718df01-0e23-424c-8ab1-bf0fd9c7e121 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:19:13 user nova-compute[72351]: DEBUG nova.compute.manager [req-0f2781e1-3558-4820-b3f3-1502aeb9c2f7 req-51c4cff7-28b9-41cc-908c-295be7b1c074 service nova] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Refreshing instance network info cache due to event network-changed-3718df01-0e23-424c-8ab1-bf0fd9c7e121. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:19:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-0f2781e1-3558-4820-b3f3-1502aeb9c2f7 req-51c4cff7-28b9-41cc-908c-295be7b1c074 service nova] Acquiring lock "refresh_cache-74728ca8-34d4-45e6-a1f2-21c0952ff835" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:19:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-0f2781e1-3558-4820-b3f3-1502aeb9c2f7 req-51c4cff7-28b9-41cc-908c-295be7b1c074 service nova] Acquired lock "refresh_cache-74728ca8-34d4-45e6-a1f2-21c0952ff835" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:19:13 user nova-compute[72351]: DEBUG nova.network.neutron [req-0f2781e1-3558-4820-b3f3-1502aeb9c2f7 req-51c4cff7-28b9-41cc-908c-295be7b1c074 service nova] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Refreshing network info cache for port 3718df01-0e23-424c-8ab1-bf0fd9c7e121 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:19:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-8a1e58e1-dcd8-4708-b13e-b997dfbd1229 tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Lock "11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.125s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:14 user nova-compute[72351]: DEBUG nova.compute.manager [req-949fcf0d-e77b-4801-a596-dfb0e3aec204 req-9509a352-b831-456c-9d29-a1ae130c8f6a service nova] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Received event network-changed-e5ba6e8d-49a5-4761-ab57-21cee6b0fbdd {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:19:14 user nova-compute[72351]: DEBUG nova.compute.manager [req-949fcf0d-e77b-4801-a596-dfb0e3aec204 req-9509a352-b831-456c-9d29-a1ae130c8f6a service nova] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Refreshing instance network info cache due to event network-changed-e5ba6e8d-49a5-4761-ab57-21cee6b0fbdd. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:19:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-949fcf0d-e77b-4801-a596-dfb0e3aec204 req-9509a352-b831-456c-9d29-a1ae130c8f6a service nova] Acquiring lock "refresh_cache-50302d76-f6b9-4fe3-b55a-4a01e483e2e3" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:19:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-949fcf0d-e77b-4801-a596-dfb0e3aec204 req-9509a352-b831-456c-9d29-a1ae130c8f6a service nova] Acquired lock "refresh_cache-50302d76-f6b9-4fe3-b55a-4a01e483e2e3" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:19:14 user nova-compute[72351]: DEBUG nova.network.neutron [req-949fcf0d-e77b-4801-a596-dfb0e3aec204 req-9509a352-b831-456c-9d29-a1ae130c8f6a service nova] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Refreshing network info cache for port e5ba6e8d-49a5-4761-ab57-21cee6b0fbdd {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:19:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-8a1e58e1-dcd8-4708-b13e-b997dfbd1229 tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Acquiring lock "11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-8a1e58e1-dcd8-4708-b13e-b997dfbd1229 tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Lock "11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:14 user nova-compute[72351]: INFO nova.compute.manager [None req-8a1e58e1-dcd8-4708-b13e-b997dfbd1229 tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Attaching volume 0203626e-04e6-4768-b2c2-e7dbaeed4c6c to /dev/vdb
Aug 09 02:19:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:14 user nova-compute[72351]: DEBUG nova.network.neutron [req-7718a720-489d-4635-833f-276da0a2f653 req-3a359a99-ad4a-4792-afbc-c6f2d24ddf04 service nova] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Updated VIF entry in instance network info cache for port a309cd5b-1679-48c6-a596-168b3af2d284. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:19:14 user nova-compute[72351]: DEBUG nova.network.neutron [req-7718a720-489d-4635-833f-276da0a2f653 req-3a359a99-ad4a-4792-afbc-c6f2d24ddf04 service nova] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Updating instance_info_cache with network_info: [{"id": "a309cd5b-1679-48c6-a596-168b3af2d284", "address": "fa:16:3e:dc:f0:3c", "network": {"id": "da9c66d8-08c8-4c9c-aec7-c9ad980b89e4", "bridge": "br-int", "label": "tempest-TestVolumeSwap-1327534036-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.16", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "1277988bc35041249b8e315df5259f98", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa309cd5b-16", "ovs_interfaceid": "a309cd5b-1679-48c6-a596-168b3af2d284", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:19:14 user nova-compute[72351]: DEBUG os_brick.utils [None req-8a1e58e1-dcd8-4708-b13e-b997dfbd1229 tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:19:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-7718a720-489d-4635-833f-276da0a2f653 req-3a359a99-ad4a-4792-afbc-c6f2d24ddf04 service nova] Releasing lock "refresh_cache-11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:19:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.016s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:14 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[462105b4-b1f4-41ee-b868-3fcf2e24781f]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:a52269602ff4\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:14 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[426deeb6-9f8a-47b3-b0cf-f975baa827ca]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.024s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:14 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[7bf7dc30-725f-4048-b84a-4ecd4d547885]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:14 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[634b3b1c-e61b-4aaf-bc25-30c7c631eb82]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-8a1e58e1-dcd8-4708-b13e-b997dfbd1229 tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Running cmd (subprocess): nvme version {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-8a1e58e1-dcd8-4708-b13e-b997dfbd1229 tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] 'nvme version' failed. Not Retrying. {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:19:14 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.nvmeof [None req-8a1e58e1-dcd8-4708-b13e-b997dfbd1229 tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] nvme not present on system {{(pid=72351) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Aug 09 02:19:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:19:14 user nova-compute[72351]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Aug 09 02:19:14 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e143f379-6cf9-4eb9-9ac9-d55157e57bb8]: (4, '') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:14 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-8a1e58e1-dcd8-4708-b13e-b997dfbd1229 tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72351) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Aug 09 02:19:14 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-8a1e58e1-dcd8-4708-b13e-b997dfbd1229 tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Aug 09 02:19:14 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-8a1e58e1-dcd8-4708-b13e-b997dfbd1229 tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] LIGHTOS: no hostnqn found. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Aug 09 02:19:14 user nova-compute[72351]: DEBUG os_brick.utils [None req-8a1e58e1-dcd8-4708-b13e-b997dfbd1229 tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] <== get_connector_properties: return (177ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:a52269602ff4', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:19:14 user nova-compute[72351]: DEBUG nova.virt.block_device [None req-8a1e58e1-dcd8-4708-b13e-b997dfbd1229 tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Updating existing volume attachment record: 869b9765-6f9b-4c0d-8c39-6e4ce1e6e842 {{(pid=72351) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Aug 09 02:19:14 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547554.5181177, 629e177e-afbb-4765-a0b5-7a3f66f72ebf => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:19:14 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] VM Resumed (Lifecycle Event)
Aug 09 02:19:14 user nova-compute[72351]: DEBUG nova.compute.manager [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:19:14 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Guest created on hypervisor {{(pid=72351) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4411}}
Aug 09 02:19:14 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Instance spawned successfully.
Aug 09 02:19:14 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:894}}
Aug 09 02:19:14 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:19:14 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:19:14 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:19:14 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547554.5213428, 629e177e-afbb-4765-a0b5-7a3f66f72ebf => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:19:14 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] VM Started (Lifecycle Event)
Aug 09 02:19:14 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Found default for hw_cdrom_bus of ide {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:19:14 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Found default for hw_disk_bus of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:19:14 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Found default for hw_input_bus of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:19:14 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Found default for hw_pointer_model of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:19:14 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Found default for hw_video_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:19:14 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Found default for hw_vif_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:19:14 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:19:14 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:19:14 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:19:14 user nova-compute[72351]: DEBUG nova.network.neutron [req-0f2781e1-3558-4820-b3f3-1502aeb9c2f7 req-51c4cff7-28b9-41cc-908c-295be7b1c074 service nova] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Updated VIF entry in instance network info cache for port 3718df01-0e23-424c-8ab1-bf0fd9c7e121. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:19:14 user nova-compute[72351]: DEBUG nova.network.neutron [req-0f2781e1-3558-4820-b3f3-1502aeb9c2f7 req-51c4cff7-28b9-41cc-908c-295be7b1c074 service nova] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Updating instance_info_cache with network_info: [{"id": "3718df01-0e23-424c-8ab1-bf0fd9c7e121", "address": "fa:16:3e:7c:0b:0d", "network": {"id": "21efbfa4-f3a2-4317-89fc-3cc7b60fde97", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1919568535-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.239", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bc5e05fbd7f44f85bc3b1cc169dd03a9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3718df01-0e", "ovs_interfaceid": "3718df01-0e23-424c-8ab1-bf0fd9c7e121", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:19:14 user nova-compute[72351]: INFO nova.compute.manager [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Took 17.07 seconds to spawn the instance on the hypervisor.
Aug 09 02:19:14 user nova-compute[72351]: DEBUG nova.compute.manager [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:19:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-0f2781e1-3558-4820-b3f3-1502aeb9c2f7 req-51c4cff7-28b9-41cc-908c-295be7b1c074 service nova] Releasing lock "refresh_cache-74728ca8-34d4-45e6-a1f2-21c0952ff835" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:19:14 user nova-compute[72351]: DEBUG nova.compute.manager [req-0f2781e1-3558-4820-b3f3-1502aeb9c2f7 req-51c4cff7-28b9-41cc-908c-295be7b1c074 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received event network-vif-plugged-5d0bb297-60b1-4e66-8f43-3bf1c32ae4ef {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:19:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-0f2781e1-3558-4820-b3f3-1502aeb9c2f7 req-51c4cff7-28b9-41cc-908c-295be7b1c074 service nova] Acquiring lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-0f2781e1-3558-4820-b3f3-1502aeb9c2f7 req-51c4cff7-28b9-41cc-908c-295be7b1c074 service nova] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-0f2781e1-3558-4820-b3f3-1502aeb9c2f7 req-51c4cff7-28b9-41cc-908c-295be7b1c074 service nova] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:14 user nova-compute[72351]: DEBUG nova.compute.manager [req-0f2781e1-3558-4820-b3f3-1502aeb9c2f7 req-51c4cff7-28b9-41cc-908c-295be7b1c074 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] No waiting events found dispatching network-vif-plugged-5d0bb297-60b1-4e66-8f43-3bf1c32ae4ef {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:19:14 user nova-compute[72351]: WARNING nova.compute.manager [req-0f2781e1-3558-4820-b3f3-1502aeb9c2f7 req-51c4cff7-28b9-41cc-908c-295be7b1c074 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received unexpected event network-vif-plugged-5d0bb297-60b1-4e66-8f43-3bf1c32ae4ef for instance with vm_state building and task_state spawning.
Aug 09 02:19:14 user nova-compute[72351]: DEBUG nova.compute.manager [req-0f2781e1-3558-4820-b3f3-1502aeb9c2f7 req-51c4cff7-28b9-41cc-908c-295be7b1c074 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received event network-vif-plugged-c7c195c0-af76-445e-8a18-8169f0cdf5fc {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:19:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-0f2781e1-3558-4820-b3f3-1502aeb9c2f7 req-51c4cff7-28b9-41cc-908c-295be7b1c074 service nova] Acquiring lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-0f2781e1-3558-4820-b3f3-1502aeb9c2f7 req-51c4cff7-28b9-41cc-908c-295be7b1c074 service nova] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-0f2781e1-3558-4820-b3f3-1502aeb9c2f7 req-51c4cff7-28b9-41cc-908c-295be7b1c074 service nova] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:14 user nova-compute[72351]: DEBUG nova.compute.manager [req-0f2781e1-3558-4820-b3f3-1502aeb9c2f7 req-51c4cff7-28b9-41cc-908c-295be7b1c074 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] No waiting events found dispatching network-vif-plugged-c7c195c0-af76-445e-8a18-8169f0cdf5fc {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:19:14 user nova-compute[72351]: WARNING nova.compute.manager [req-0f2781e1-3558-4820-b3f3-1502aeb9c2f7 req-51c4cff7-28b9-41cc-908c-295be7b1c074 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received unexpected event network-vif-plugged-c7c195c0-af76-445e-8a18-8169f0cdf5fc for instance with vm_state building and task_state spawning.
Aug 09 02:19:14 user nova-compute[72351]: DEBUG nova.compute.manager [req-0f2781e1-3558-4820-b3f3-1502aeb9c2f7 req-51c4cff7-28b9-41cc-908c-295be7b1c074 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received event network-vif-plugged-c7c195c0-af76-445e-8a18-8169f0cdf5fc {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:19:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-0f2781e1-3558-4820-b3f3-1502aeb9c2f7 req-51c4cff7-28b9-41cc-908c-295be7b1c074 service nova] Acquiring lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-0f2781e1-3558-4820-b3f3-1502aeb9c2f7 req-51c4cff7-28b9-41cc-908c-295be7b1c074 service nova] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-0f2781e1-3558-4820-b3f3-1502aeb9c2f7 req-51c4cff7-28b9-41cc-908c-295be7b1c074 service nova] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:14 user nova-compute[72351]: DEBUG nova.compute.manager [req-0f2781e1-3558-4820-b3f3-1502aeb9c2f7 req-51c4cff7-28b9-41cc-908c-295be7b1c074 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] No waiting events found dispatching network-vif-plugged-c7c195c0-af76-445e-8a18-8169f0cdf5fc {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:19:14 user nova-compute[72351]: WARNING nova.compute.manager [req-0f2781e1-3558-4820-b3f3-1502aeb9c2f7 req-51c4cff7-28b9-41cc-908c-295be7b1c074 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received unexpected event network-vif-plugged-c7c195c0-af76-445e-8a18-8169f0cdf5fc for instance with vm_state building and task_state spawning.
Aug 09 02:19:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-32af3847-2f53-49f4-a53d-30a86edfd7d0 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] Acquiring lock "7c5e633d-a32f-4c5e-bb9d-1e40da964bd0" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-32af3847-2f53-49f4-a53d-30a86edfd7d0 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] Lock "7c5e633d-a32f-4c5e-bb9d-1e40da964bd0" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:14 user nova-compute[72351]: DEBUG nova.objects.instance [None req-32af3847-2f53-49f4-a53d-30a86edfd7d0 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] Lazy-loading 'flavor' on Instance uuid 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:19:14 user nova-compute[72351]: INFO nova.compute.manager [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Took 48.79 seconds to build instance.
Aug 09 02:19:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c7c2ac7e-926f-443a-9c6a-38b5ea0a2dd6 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 48.929s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-32af3847-2f53-49f4-a53d-30a86edfd7d0 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] Lock "7c5e633d-a32f-4c5e-bb9d-1e40da964bd0" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.122s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:15 user nova-compute[72351]: DEBUG nova.network.neutron [req-949fcf0d-e77b-4801-a596-dfb0e3aec204 req-9509a352-b831-456c-9d29-a1ae130c8f6a service nova] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Updated VIF entry in instance network info cache for port e5ba6e8d-49a5-4761-ab57-21cee6b0fbdd. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:19:15 user nova-compute[72351]: DEBUG nova.network.neutron [req-949fcf0d-e77b-4801-a596-dfb0e3aec204 req-9509a352-b831-456c-9d29-a1ae130c8f6a service nova] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Updating instance_info_cache with network_info: [{"id": "e5ba6e8d-49a5-4761-ab57-21cee6b0fbdd", "address": "fa:16:3e:6e:b8:a5", "network": {"id": "1acf710e-e626-43a0-a32f-389a87f9858e", "bridge": "br-int", "label": "tempest-AttachSCSIVolumeTestJSON-1052335889-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.121", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3f04dea5c1d84aaf94feb47b8ea84f0e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape5ba6e8d-49", "ovs_interfaceid": "e5ba6e8d-49a5-4761-ab57-21cee6b0fbdd", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:19:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-949fcf0d-e77b-4801-a596-dfb0e3aec204 req-9509a352-b831-456c-9d29-a1ae130c8f6a service nova] Releasing lock "refresh_cache-50302d76-f6b9-4fe3-b55a-4a01e483e2e3" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:19:15 user nova-compute[72351]: DEBUG nova.compute.manager [req-949fcf0d-e77b-4801-a596-dfb0e3aec204 req-9509a352-b831-456c-9d29-a1ae130c8f6a service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received event network-vif-plugged-c2df74bf-22ce-48b1-9ab8-3a684cef7266 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:19:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-949fcf0d-e77b-4801-a596-dfb0e3aec204 req-9509a352-b831-456c-9d29-a1ae130c8f6a service nova] Acquiring lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-949fcf0d-e77b-4801-a596-dfb0e3aec204 req-9509a352-b831-456c-9d29-a1ae130c8f6a service nova] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-949fcf0d-e77b-4801-a596-dfb0e3aec204 req-9509a352-b831-456c-9d29-a1ae130c8f6a service nova] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:15 user nova-compute[72351]: DEBUG nova.compute.manager [req-949fcf0d-e77b-4801-a596-dfb0e3aec204 req-9509a352-b831-456c-9d29-a1ae130c8f6a service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] No waiting events found dispatching network-vif-plugged-c2df74bf-22ce-48b1-9ab8-3a684cef7266 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:19:15 user nova-compute[72351]: WARNING nova.compute.manager [req-949fcf0d-e77b-4801-a596-dfb0e3aec204 req-9509a352-b831-456c-9d29-a1ae130c8f6a service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received unexpected event network-vif-plugged-c2df74bf-22ce-48b1-9ab8-3a684cef7266 for instance with vm_state building and task_state spawning.
Aug 09 02:19:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-32af3847-2f53-49f4-a53d-30a86edfd7d0 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] Acquiring lock "7c5e633d-a32f-4c5e-bb9d-1e40da964bd0" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-32af3847-2f53-49f4-a53d-30a86edfd7d0 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] Lock "7c5e633d-a32f-4c5e-bb9d-1e40da964bd0" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:15 user nova-compute[72351]: INFO nova.compute.manager [None req-32af3847-2f53-49f4-a53d-30a86edfd7d0 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Attaching volume 3fbc4f5c-6a7d-4387-bd79-861227a814d3 to /dev/vdb
Aug 09 02:19:15 user nova-compute[72351]: DEBUG os_brick.utils [None req-32af3847-2f53-49f4-a53d-30a86edfd7d0 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:19:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:15 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[7c075b7d-49cf-4a65-945e-5c9521c5be68]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:a52269602ff4\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:15 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a5a11b83-3776-4f51-a835-48cf36b4e3bd]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.015s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:15 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[0f3e7ed6-e639-417f-9d6d-dc9fa539cbbf]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:15 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[b77209c8-12c8-41af-88a6-30c8bd63e47a]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-32af3847-2f53-49f4-a53d-30a86edfd7d0 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] Running cmd (subprocess): nvme version {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-32af3847-2f53-49f4-a53d-30a86edfd7d0 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] 'nvme version' failed. Not Retrying. {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:19:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.nvmeof [None req-32af3847-2f53-49f4-a53d-30a86edfd7d0 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] nvme not present on system {{(pid=72351) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Aug 09 02:19:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:19:15 user nova-compute[72351]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Aug 09 02:19:15 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[8da251dd-15ad-47da-ac96-3e2a72469856]: (4, '') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-32af3847-2f53-49f4-a53d-30a86edfd7d0 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72351) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Aug 09 02:19:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-32af3847-2f53-49f4-a53d-30a86edfd7d0 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Aug 09 02:19:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-32af3847-2f53-49f4-a53d-30a86edfd7d0 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] LIGHTOS: no hostnqn found. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Aug 09 02:19:15 user nova-compute[72351]: DEBUG os_brick.utils [None req-32af3847-2f53-49f4-a53d-30a86edfd7d0 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] <== get_connector_properties: return (128ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:a52269602ff4', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:19:15 user nova-compute[72351]: DEBUG nova.virt.block_device [None req-32af3847-2f53-49f4-a53d-30a86edfd7d0 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Updating existing volume attachment record: 1319bb3d-b2ae-454a-8596-e75b4fbbd33a {{(pid=72351) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Aug 09 02:19:15 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:16 user nova-compute[72351]: DEBUG nova.compute.manager [req-5b6a0489-b9bc-43c5-a2df-836cd210f05d req-d174bb4c-8c1e-4966-aacb-b5eb6737f25d service nova] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Received event network-changed-7e7b4229-7c23-4a73-9294-4c4fc9f6a01a {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:19:16 user nova-compute[72351]: DEBUG nova.compute.manager [req-5b6a0489-b9bc-43c5-a2df-836cd210f05d req-d174bb4c-8c1e-4966-aacb-b5eb6737f25d service nova] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Refreshing instance network info cache due to event network-changed-7e7b4229-7c23-4a73-9294-4c4fc9f6a01a. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:19:16 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-5b6a0489-b9bc-43c5-a2df-836cd210f05d req-d174bb4c-8c1e-4966-aacb-b5eb6737f25d service nova] Acquiring lock "refresh_cache-7c5e633d-a32f-4c5e-bb9d-1e40da964bd0" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:19:16 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-5b6a0489-b9bc-43c5-a2df-836cd210f05d req-d174bb4c-8c1e-4966-aacb-b5eb6737f25d service nova] Acquired lock "refresh_cache-7c5e633d-a32f-4c5e-bb9d-1e40da964bd0" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:19:16 user nova-compute[72351]: DEBUG nova.network.neutron [req-5b6a0489-b9bc-43c5-a2df-836cd210f05d req-d174bb4c-8c1e-4966-aacb-b5eb6737f25d service nova] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Refreshing network info cache for port 7e7b4229-7c23-4a73-9294-4c4fc9f6a01a {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:19:16 user nova-compute[72351]: DEBUG nova.network.neutron [req-5b6a0489-b9bc-43c5-a2df-836cd210f05d req-d174bb4c-8c1e-4966-aacb-b5eb6737f25d service nova] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Updated VIF entry in instance network info cache for port 7e7b4229-7c23-4a73-9294-4c4fc9f6a01a. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:19:16 user nova-compute[72351]: DEBUG nova.network.neutron [req-5b6a0489-b9bc-43c5-a2df-836cd210f05d req-d174bb4c-8c1e-4966-aacb-b5eb6737f25d service nova] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Updating instance_info_cache with network_info: [{"id": "7e7b4229-7c23-4a73-9294-4c4fc9f6a01a", "address": "fa:16:3e:d9:0b:0f", "network": {"id": "49cd7d7c-b635-484c-9ef3-268529d7344f", "bridge": "br-int", "label": "tempest-VolumesAssistedSnapshotsTest-24910214-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.168", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "26827058be4b48a3a7aed21aad7c5e77", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7e7b4229-7c", "ovs_interfaceid": "7e7b4229-7c23-4a73-9294-4c4fc9f6a01a", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:19:16 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-5b6a0489-b9bc-43c5-a2df-836cd210f05d req-d174bb4c-8c1e-4966-aacb-b5eb6737f25d service nova] Releasing lock "refresh_cache-7c5e633d-a32f-4c5e-bb9d-1e40da964bd0" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:19:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-06db8269-2459-48b6-acef-2c376c658ce3 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Acquiring lock "74728ca8-34d4-45e6-a1f2-21c0952ff835" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-06db8269-2459-48b6-acef-2c376c658ce3 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Lock "74728ca8-34d4-45e6-a1f2-21c0952ff835" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:17 user nova-compute[72351]: DEBUG nova.objects.instance [None req-06db8269-2459-48b6-acef-2c376c658ce3 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Lazy-loading 'flavor' on Instance uuid 74728ca8-34d4-45e6-a1f2-21c0952ff835 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:19:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-06db8269-2459-48b6-acef-2c376c658ce3 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Lock "74728ca8-34d4-45e6-a1f2-21c0952ff835" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.080s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-06db8269-2459-48b6-acef-2c376c658ce3 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Acquiring lock "74728ca8-34d4-45e6-a1f2-21c0952ff835" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-06db8269-2459-48b6-acef-2c376c658ce3 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Lock "74728ca8-34d4-45e6-a1f2-21c0952ff835" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:17 user nova-compute[72351]: INFO nova.compute.manager [None req-06db8269-2459-48b6-acef-2c376c658ce3 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Attaching volume 3e072274-962d-4e11-985a-5ef6973a3780 to /dev/vdb
Aug 09 02:19:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-4f194355-115f-4a7e-bc98-987e987f1cf0 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Acquiring lock "50302d76-f6b9-4fe3-b55a-4a01e483e2e3" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-4f194355-115f-4a7e-bc98-987e987f1cf0 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Lock "50302d76-f6b9-4fe3-b55a-4a01e483e2e3" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:17 user nova-compute[72351]: DEBUG nova.objects.instance [None req-4f194355-115f-4a7e-bc98-987e987f1cf0 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Lazy-loading 'flavor' on Instance uuid 50302d76-f6b9-4fe3-b55a-4a01e483e2e3 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:19:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-4f194355-115f-4a7e-bc98-987e987f1cf0 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Lock "50302d76-f6b9-4fe3-b55a-4a01e483e2e3" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.074s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:17 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:17 user nova-compute[72351]: DEBUG os_brick.utils [None req-06db8269-2459-48b6-acef-2c376c658ce3 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:19:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:17 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[caeaf2f0-eba7-4029-b64f-9e3c12960d4d]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:a52269602ff4\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:17 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[be78b9bc-3e79-41c7-8af4-79c94fe5186a]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:17 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ed3a7164-99a0-4e8c-b94f-3fb94313f74e]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:17 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[7a3bc2e7-8afc-4638-9c64-c109fd0bc5f7]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-06db8269-2459-48b6-acef-2c376c658ce3 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Running cmd (subprocess): nvme version {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-06db8269-2459-48b6-acef-2c376c658ce3 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] 'nvme version' failed. Not Retrying. {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:19:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.nvmeof [None req-06db8269-2459-48b6-acef-2c376c658ce3 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] nvme not present on system {{(pid=72351) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Aug 09 02:19:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:19:17 user nova-compute[72351]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Aug 09 02:19:17 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[f9848c44-77f0-443a-af8d-2ba956cab165]: (4, '') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-06db8269-2459-48b6-acef-2c376c658ce3 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72351) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Aug 09 02:19:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-06db8269-2459-48b6-acef-2c376c658ce3 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Aug 09 02:19:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-06db8269-2459-48b6-acef-2c376c658ce3 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] LIGHTOS: no hostnqn found. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Aug 09 02:19:17 user nova-compute[72351]: DEBUG os_brick.utils [None req-06db8269-2459-48b6-acef-2c376c658ce3 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] <== get_connector_properties: return (158ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:a52269602ff4', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:19:17 user nova-compute[72351]: DEBUG nova.virt.block_device [None req-06db8269-2459-48b6-acef-2c376c658ce3 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Updating existing volume attachment record: bd6aebc4-5984-434f-8267-5e6bf0d39521 {{(pid=72351) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Aug 09 02:19:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-4f194355-115f-4a7e-bc98-987e987f1cf0 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Acquiring lock "50302d76-f6b9-4fe3-b55a-4a01e483e2e3" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-4f194355-115f-4a7e-bc98-987e987f1cf0 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Lock "50302d76-f6b9-4fe3-b55a-4a01e483e2e3" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:17 user nova-compute[72351]: INFO nova.compute.manager [None req-4f194355-115f-4a7e-bc98-987e987f1cf0 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Attaching volume 643c3e80-8b8a-40c2-a979-579d65ba653a to /dev/sdc
Aug 09 02:19:18 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691547543.248214, f45b57fa-9264-45f0-9573-e5fb58848d1f => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:19:18 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] VM Stopped (Lifecycle Event)
Aug 09 02:19:18 user nova-compute[72351]: DEBUG nova.compute.manager [None req-c9dfcd2d-b05b-4678-b521-f06dd97200b8 None None] [instance: f45b57fa-9264-45f0-9573-e5fb58848d1f] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:19:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-32af3847-2f53-49f4-a53d-30a86edfd7d0 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-32af3847-2f53-49f4-a53d-30a86edfd7d0 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:3fbc4f5c-6a7d-4387-bd79-861227a814d3'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'XfchnPRD', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:19:25 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-32af3847-2f53-49f4-a53d-30a86edfd7d0 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:19:25 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-32af3847-2f53-49f4-a53d-30a86edfd7d0 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-32af3847-2f53-49f4-a53d-30a86edfd7d0 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:19:25 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-32af3847-2f53-49f4-a53d-30a86edfd7d0 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:19:25 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-32af3847-2f53-49f4-a53d-30a86edfd7d0 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:3fbc4f5c-6a7d-4387-bd79-861227a814d3" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-32af3847-2f53-49f4-a53d-30a86edfd7d0 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:3fbc4f5c-6a7d-4387-bd79-861227a814d3" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:19:25 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-32af3847-2f53-49f4-a53d-30a86edfd7d0 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 02:19:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3fbc4f5c-6a7d-4387-bd79-861227a814d3 -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3fbc4f5c-6a7d-4387-bd79-861227a814d3 -p 172.16.0.220:3260" returned: 21 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[7cd5b32a-8b08-4bbe-8fcc-f3f912bf9a28]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-32af3847-2f53-49f4-a53d-30a86edfd7d0 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] iscsiadm (): stdout= stderr=iscsiadm: No records found
Aug 09 02:19:25 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3fbc4f5c-6a7d-4387-bd79-861227a814d3 -p 172.16.0.220:3260 --interface default --op new {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3fbc4f5c-6a7d-4387-bd79-861227a814d3 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[920d41eb-9177-425d-b5f8-d38ba6bfeb6f]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:3fbc4f5c-6a7d-4387-bd79-861227a814d3] added\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-32af3847-2f53-49f4-a53d-30a86edfd7d0 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:3fbc4f5c-6a7d-4387-bd79-861227a814d3] added
Aug 09 02:19:25 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3fbc4f5c-6a7d-4387-bd79-861227a814d3 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3fbc4f5c-6a7d-4387-bd79-861227a814d3 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[125f2b7e-ec5f-4b83-a0ae-77c7235f3381]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-32af3847-2f53-49f4-a53d-30a86edfd7d0 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3fbc4f5c-6a7d-4387-bd79-861227a814d3 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3fbc4f5c-6a7d-4387-bd79-861227a814d3 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[0616f5d3-e1cd-41ca-a4e6-45aff48a2192]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-32af3847-2f53-49f4-a53d-30a86edfd7d0 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3fbc4f5c-6a7d-4387-bd79-861227a814d3 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v XfchnPRD {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3fbc4f5c-6a7d-4387-bd79-861227a814d3 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v XfchnPRD" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[247e3856-3ed6-473a-a356-5f46208a8c7b]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-32af3847-2f53-49f4-a53d-30a86edfd7d0 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'XfchnPRD'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3fbc4f5c-6a7d-4387-bd79-861227a814d3 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3fbc4f5c-6a7d-4387-bd79-861227a814d3 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[02d96dcf-6ed0-433b-ac25-ecdac2b8b63b]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-32af3847-2f53-49f4-a53d-30a86edfd7d0 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[350b18ab-a14d-4bd3-83d5-5b43d8dca4a4]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-32af3847-2f53-49f4-a53d-30a86edfd7d0 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:19:25 user nova-compute[72351]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)
Aug 09 02:19:25 user nova-compute[72351]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:19:25 user nova-compute[72351]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)
Aug 09 02:19:25 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-32af3847-2f53-49f4-a53d-30a86edfd7d0 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:19:25 user nova-compute[72351]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)
Aug 09 02:19:25 user nova-compute[72351]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:19:25 user nova-compute[72351]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)
Aug 09 02:19:25 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3fbc4f5c-6a7d-4387-bd79-861227a814d3 -p 172.16.0.220:3260 --login {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG os_brick.utils [None req-4f194355-115f-4a7e-bc98-987e987f1cf0 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[b19cddd4-fe67-4758-96a0-1b22521bf04c]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:a52269602ff4\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3fbc4f5c-6a7d-4387-bd79-861227a814d3 -p 172.16.0.220:3260 --login" returned: 0 in 0.177s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[89b2a35e-4300-476a-9aef-f67fa0436d18]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:3fbc4f5c-6a7d-4387-bd79-861227a814d3, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:3fbc4f5c-6a7d-4387-bd79-861227a814d3, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-32af3847-2f53-49f4-a53d-30a86edfd7d0 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:3fbc4f5c-6a7d-4387-bd79-861227a814d3, portal: 172.16.0.220,3260]
Aug 09 02:19:25 user nova-compute[72351]: Login to [iface: default, target: iqn.2010-10.org.openstack:3fbc4f5c-6a7d-4387-bd79-861227a814d3, portal: 172.16.0.220,3260] successful.
Aug 09 02:19:25 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3fbc4f5c-6a7d-4387-bd79-861227a814d3 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.049s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[2f2e5625-8f32-4735-9a45-175668c0209c]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3fbc4f5c-6a7d-4387-bd79-861227a814d3 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[cd51c45c-6c16-4c73-a5f9-2478d7d47a38]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-32af3847-2f53-49f4-a53d-30a86edfd7d0 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.026s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[30d124ab-70e5-41a3-b1cf-66fe569bb09d]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[f2484792-9286-4112-8cf3-c460b497919b]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-4f194355-115f-4a7e-bc98-987e987f1cf0 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Running cmd (subprocess): nvme version {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-4f194355-115f-4a7e-bc98-987e987f1cf0 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] 'nvme version' failed. Not Retrying. {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.039s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[9c64c892-c9da-4eb9-af9a-f91d64d283e7]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)\ntcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3fbc4f5c-6a7d-4387-bd79-861227a814d3 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.nvmeof [None req-4f194355-115f-4a7e-bc98-987e987f1cf0 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] nvme not present on system {{(pid=72351) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-32af3847-2f53-49f4-a53d-30a86edfd7d0 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:19:25 user nova-compute[72351]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)
Aug 09 02:19:25 user nova-compute[72351]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:19:25 user nova-compute[72351]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)
Aug 09 02:19:25 user nova-compute[72351]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3fbc4f5c-6a7d-4387-bd79-861227a814d3 (non-flash)
Aug 09 02:19:25 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-32af3847-2f53-49f4-a53d-30a86edfd7d0 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:19:25 user nova-compute[72351]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)
Aug 09 02:19:25 user nova-compute[72351]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:19:25 user nova-compute[72351]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)
Aug 09 02:19:25 user nova-compute[72351]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3fbc4f5c-6a7d-4387-bd79-861227a814d3 (non-flash)
Aug 09 02:19:25 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-32af3847-2f53-49f4-a53d-30a86edfd7d0 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:3fbc4f5c-6a7d-4387-bd79-861227a814d3" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.454s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-32af3847-2f53-49f4-a53d-30a86edfd7d0 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-32af3847-2f53-49f4-a53d-30a86edfd7d0 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] HCTL ('34', '-', '-', 0) found on session 7 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-32af3847-2f53-49f4-a53d-30a86edfd7d0 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] Scanning host 34 c: -, t: -, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:19:25 user nova-compute[72351]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Aug 09 02:19:25 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[2b5bf93a-368e-4185-9e2e-36ad56265062]: (4, '') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-4f194355-115f-4a7e-bc98-987e987f1cf0 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72351) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-4f194355-115f-4a7e-bc98-987e987f1cf0 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-4f194355-115f-4a7e-bc98-987e987f1cf0 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] LIGHTOS: no hostnqn found. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG os_brick.utils [None req-4f194355-115f-4a7e-bc98-987e987f1cf0 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] <== get_connector_properties: return (202ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:a52269602ff4', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG nova.virt.block_device [None req-4f194355-115f-4a7e-bc98-987e987f1cf0 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Updating existing volume attachment record: f9abc1fd-346c-4ab9-843b-2f19d2c2f107 {{(pid=72351) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d4367eb1-610e-48dd-bd60-4a59b6ad264a]: (4, ('- - 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-32af3847-2f53-49f4-a53d-30a86edfd7d0 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] Searching for a device in session 7 and hctl ['34', '*', '*', 0] yield: sdc {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:19:25 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-32af3847-2f53-49f4-a53d-30a86edfd7d0 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] Connected to sdc using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'XfchnPRD', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 02:19:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-8a1e58e1-dcd8-4708-b13e-b997dfbd1229 tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 02:19:26 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-8a1e58e1-dcd8-4708-b13e-b997dfbd1229 tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'sZqwozDV', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:19:26 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-8a1e58e1-dcd8-4708-b13e-b997dfbd1229 tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:19:26 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-8a1e58e1-dcd8-4708-b13e-b997dfbd1229 tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:19:26 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-32af3847-2f53-49f4-a53d-30a86edfd7d0 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.511s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:19:26 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-32af3847-2f53-49f4-a53d-30a86edfd7d0 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] <== connect_volume: return (1513ms) {'type': 'block', 'scsi_wwn': '26330313566663632', 'path': '/dev/sdc'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:19:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-32af3847-2f53-49f4-a53d-30a86edfd7d0 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26330313566663632', 'path': '/dev/sdc'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 02:19:26 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-8a1e58e1-dcd8-4708-b13e-b997dfbd1229 tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.965s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:19:26 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-8a1e58e1-dcd8-4708-b13e-b997dfbd1229 tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:19:26 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-8a1e58e1-dcd8-4708-b13e-b997dfbd1229 tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:19:26 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-8a1e58e1-dcd8-4708-b13e-b997dfbd1229 tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:19:26 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-8a1e58e1-dcd8-4708-b13e-b997dfbd1229 tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 02:19:26 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:27 user nova-compute[72351]: DEBUG nova.objects.instance [None req-32af3847-2f53-49f4-a53d-30a86edfd7d0 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] Lazy-loading 'flavor' on Instance uuid 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:19:27 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c -p 172.16.0.220:3260" returned: 21 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:27 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[2e613052-098b-4ef9-93e6-7de12db5ab6f]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:27 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-8a1e58e1-dcd8-4708-b13e-b997dfbd1229 tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Aug 09 02:19:27 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:27 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c -p 172.16.0.220:3260 --interface default --op new {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:27 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.025s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:27 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[7074f8d5-2851-46bb-91fc-35d388d9beaf]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c] added\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:27 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-8a1e58e1-dcd8-4708-b13e-b997dfbd1229 tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c] added
Aug 09 02:19:27 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:27 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:27 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-32af3847-2f53-49f4-a53d-30a86edfd7d0 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] attach device xml: <disk type="block" device="disk">
Aug 09 02:19:27 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:19:27 user nova-compute[72351]:   <source dev="/dev/sdc"/>
Aug 09 02:19:27 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:19:27 user nova-compute[72351]:   <serial>3fbc4f5c-6a7d-4387-bd79-861227a814d3</serial>
Aug 09 02:19:27 user nova-compute[72351]: </disk>
Aug 09 02:19:27 user nova-compute[72351]:  {{(pid=72351) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Aug 09 02:19:27 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:27 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a9576228-e7a1-49e3-a571-01179b2cb9f0]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:27 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-8a1e58e1-dcd8-4708-b13e-b997dfbd1229 tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:27 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:27 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.024s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:27 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[29b4476f-5605-473d-ba01-4027155b586b]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:27 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-8a1e58e1-dcd8-4708-b13e-b997dfbd1229 tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:27 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c -p 172.16.0.220:3260 --op update -n node.session.auth.username -v sZqwozDV {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:27 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c -p 172.16.0.220:3260 --op update -n node.session.auth.username -v sZqwozDV" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:27 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[4be1e100-e30e-48c1-a49f-4cffe5be9690]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:27 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-8a1e58e1-dcd8-4708-b13e-b997dfbd1229 tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'sZqwozDV'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:27 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:27 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.025s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:27 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[b899032a-e99b-4921-bff4-fae38c317715]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:27 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-8a1e58e1-dcd8-4708-b13e-b997dfbd1229 tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:27 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:27 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.035s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:27 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e17e7ced-2983-4869-af70-7502a1e9c714]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)\ntcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3fbc4f5c-6a7d-4387-bd79-861227a814d3 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:27 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-8a1e58e1-dcd8-4708-b13e-b997dfbd1229 tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:19:27 user nova-compute[72351]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)
Aug 09 02:19:27 user nova-compute[72351]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:19:27 user nova-compute[72351]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)
Aug 09 02:19:27 user nova-compute[72351]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3fbc4f5c-6a7d-4387-bd79-861227a814d3 (non-flash)
Aug 09 02:19:27 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:19:27 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-8a1e58e1-dcd8-4708-b13e-b997dfbd1229 tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:19:27 user nova-compute[72351]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)
Aug 09 02:19:27 user nova-compute[72351]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:19:27 user nova-compute[72351]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)
Aug 09 02:19:27 user nova-compute[72351]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3fbc4f5c-6a7d-4387-bd79-861227a814d3 (non-flash)
Aug 09 02:19:27 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:19:27 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c -p 172.16.0.220:3260 --login {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:27 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c -p 172.16.0.220:3260 --login" returned: 0 in 0.089s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:27 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[750e1b1a-db1a-4795-b65e-a20d8303af2a]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:27 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-8a1e58e1-dcd8-4708-b13e-b997dfbd1229 tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c, portal: 172.16.0.220,3260]
Aug 09 02:19:27 user nova-compute[72351]: Login to [iface: default, target: iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c, portal: 172.16.0.220,3260] successful.
Aug 09 02:19:27 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:27 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:27 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.025s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:27 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a2a1a6b8-60bf-4969-9976-f33012d8d3a7]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:27 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-8a1e58e1-dcd8-4708-b13e-b997dfbd1229 tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:27 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:27 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.028s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:27 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[18c11234-5e93-4580-9572-726a5f5cb287]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)\ntcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3fbc4f5c-6a7d-4387-bd79-861227a814d3 (non-flash)\ntcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:27 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-8a1e58e1-dcd8-4708-b13e-b997dfbd1229 tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:19:27 user nova-compute[72351]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)
Aug 09 02:19:27 user nova-compute[72351]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:19:27 user nova-compute[72351]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)
Aug 09 02:19:27 user nova-compute[72351]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3fbc4f5c-6a7d-4387-bd79-861227a814d3 (non-flash)
Aug 09 02:19:27 user nova-compute[72351]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c (non-flash)
Aug 09 02:19:27 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:19:27 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-8a1e58e1-dcd8-4708-b13e-b997dfbd1229 tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:19:27 user nova-compute[72351]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)
Aug 09 02:19:27 user nova-compute[72351]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:19:27 user nova-compute[72351]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)
Aug 09 02:19:27 user nova-compute[72351]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3fbc4f5c-6a7d-4387-bd79-861227a814d3 (non-flash)
Aug 09 02:19:27 user nova-compute[72351]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c (non-flash)
Aug 09 02:19:27 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:19:27 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-8a1e58e1-dcd8-4708-b13e-b997dfbd1229 tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.357s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:19:27 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-8a1e58e1-dcd8-4708-b13e-b997dfbd1229 tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 02:19:27 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-8a1e58e1-dcd8-4708-b13e-b997dfbd1229 tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] HCTL ('38', '-', '-', 0) found on session 8 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 02:19:27 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-8a1e58e1-dcd8-4708-b13e-b997dfbd1229 tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Scanning host 38 c: -, t: -, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 02:19:27 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host38/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:27 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host38/scan" returned: 0 in 0.024s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:27 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[842d38f2-f5a4-4039-99ee-8f8a988b7743]: (4, ('- - 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:27 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-8a1e58e1-dcd8-4708-b13e-b997dfbd1229 tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Searching for a device in session 8 and hctl ['38', '*', '*', 0] yield: sdg {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:19:27 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-8a1e58e1-dcd8-4708-b13e-b997dfbd1229 tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Connected to sdg using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'sZqwozDV', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 02:19:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:27 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-32af3847-2f53-49f4-a53d-30a86edfd7d0 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:19:27 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-32af3847-2f53-49f4-a53d-30a86edfd7d0 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] No BDM found with device name vdb, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:19:27 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-32af3847-2f53-49f4-a53d-30a86edfd7d0 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] No VIF found with MAC fa:16:3e:d9:0b:0f, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:19:27 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-06db8269-2459-48b6-acef-2c376c658ce3 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 02:19:27 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-06db8269-2459-48b6-acef-2c376c658ce3 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:3e072274-962d-4e11-985a-5ef6973a3780'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'qbhgRIds', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:19:27 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-06db8269-2459-48b6-acef-2c376c658ce3 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:19:27 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-06db8269-2459-48b6-acef-2c376c658ce3 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:19:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-32af3847-2f53-49f4-a53d-30a86edfd7d0 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] Lock "7c5e633d-a32f-4c5e-bb9d-1e40da964bd0" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 12.837s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:28 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-8a1e58e1-dcd8-4708-b13e-b997dfbd1229 tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.416s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:19:28 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-8a1e58e1-dcd8-4708-b13e-b997dfbd1229 tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] <== connect_volume: return (2380ms) {'type': 'block', 'scsi_wwn': '23732393866386262', 'path': '/dev/sdg'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:19:28 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-8a1e58e1-dcd8-4708-b13e-b997dfbd1229 tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23732393866386262', 'path': '/dev/sdg'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 02:19:28 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-06db8269-2459-48b6-acef-2c376c658ce3 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.529s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:19:28 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-06db8269-2459-48b6-acef-2c376c658ce3 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:19:28 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-06db8269-2459-48b6-acef-2c376c658ce3 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:3e072274-962d-4e11-985a-5ef6973a3780" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:19:28 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-06db8269-2459-48b6-acef-2c376c658ce3 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:3e072274-962d-4e11-985a-5ef6973a3780" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:19:28 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-06db8269-2459-48b6-acef-2c376c658ce3 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 02:19:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3e072274-962d-4e11-985a-5ef6973a3780 -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:28 user nova-compute[72351]: DEBUG nova.objects.instance [None req-8a1e58e1-dcd8-4708-b13e-b997dfbd1229 tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Lazy-loading 'flavor' on Instance uuid 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:19:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3e072274-962d-4e11-985a-5ef6973a3780 -p 172.16.0.220:3260" returned: 21 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:28 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e0a56a7f-e964-4ee7-8309-7df66a640377]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:28 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-06db8269-2459-48b6-acef-2c376c658ce3 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Aug 09 02:19:28 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3e072274-962d-4e11-985a-5ef6973a3780 -p 172.16.0.220:3260 --interface default --op new {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3e072274-962d-4e11-985a-5ef6973a3780 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:28 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[99ffd25b-ecb8-4874-a8df-0471c2e8240e]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:3e072274-962d-4e11-985a-5ef6973a3780] added\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:28 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-06db8269-2459-48b6-acef-2c376c658ce3 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:3e072274-962d-4e11-985a-5ef6973a3780] added
Aug 09 02:19:28 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:28 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-8a1e58e1-dcd8-4708-b13e-b997dfbd1229 tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] attach device xml: <disk type="block" device="disk">
Aug 09 02:19:28 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:19:28 user nova-compute[72351]:   <source dev="/dev/sdg"/>
Aug 09 02:19:28 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:19:28 user nova-compute[72351]:   <serial>0203626e-04e6-4768-b2c2-e7dbaeed4c6c</serial>
Aug 09 02:19:28 user nova-compute[72351]: </disk>
Aug 09 02:19:28 user nova-compute[72351]:  {{(pid=72351) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Aug 09 02:19:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3e072274-962d-4e11-985a-5ef6973a3780 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3e072274-962d-4e11-985a-5ef6973a3780 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:28 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[6be052c0-7fb3-4953-aa01-a4ff398d1687]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:28 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-06db8269-2459-48b6-acef-2c376c658ce3 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3e072274-962d-4e11-985a-5ef6973a3780 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3e072274-962d-4e11-985a-5ef6973a3780 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:28 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[0c15ac8b-cf24-454a-b229-6f086a1e1b9d]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:28 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-06db8269-2459-48b6-acef-2c376c658ce3 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3e072274-962d-4e11-985a-5ef6973a3780 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v qbhgRIds {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3e072274-962d-4e11-985a-5ef6973a3780 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v qbhgRIds" returned: 0 in 0.024s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:28 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[3048de1f-21ea-4413-b0bd-17cadd1d72c9]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:28 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-06db8269-2459-48b6-acef-2c376c658ce3 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'qbhgRIds'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3e072274-962d-4e11-985a-5ef6973a3780 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3e072274-962d-4e11-985a-5ef6973a3780 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:28 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e17dbf6e-9831-4116-b4bc-a41bc31ab338]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:28 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-06db8269-2459-48b6-acef-2c376c658ce3 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.026s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:28 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[b4e0de0f-fd10-437d-a584-aab40bb92bd8]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)\ntcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3fbc4f5c-6a7d-4387-bd79-861227a814d3 (non-flash)\ntcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:28 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-06db8269-2459-48b6-acef-2c376c658ce3 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:19:28 user nova-compute[72351]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)
Aug 09 02:19:28 user nova-compute[72351]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:19:28 user nova-compute[72351]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)
Aug 09 02:19:28 user nova-compute[72351]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3fbc4f5c-6a7d-4387-bd79-861227a814d3 (non-flash)
Aug 09 02:19:28 user nova-compute[72351]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c (non-flash)
Aug 09 02:19:28 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:19:28 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-06db8269-2459-48b6-acef-2c376c658ce3 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:19:28 user nova-compute[72351]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)
Aug 09 02:19:28 user nova-compute[72351]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:19:28 user nova-compute[72351]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)
Aug 09 02:19:28 user nova-compute[72351]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3fbc4f5c-6a7d-4387-bd79-861227a814d3 (non-flash)
Aug 09 02:19:28 user nova-compute[72351]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c (non-flash)
Aug 09 02:19:28 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:19:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3e072274-962d-4e11-985a-5ef6973a3780 -p 172.16.0.220:3260 --login {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3e072274-962d-4e11-985a-5ef6973a3780 -p 172.16.0.220:3260 --login" returned: 0 in 0.077s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:28 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[875fe6f9-787e-4ea8-9f76-890ac33192f3]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:3e072274-962d-4e11-985a-5ef6973a3780, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:3e072274-962d-4e11-985a-5ef6973a3780, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:28 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-06db8269-2459-48b6-acef-2c376c658ce3 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:3e072274-962d-4e11-985a-5ef6973a3780, portal: 172.16.0.220,3260]
Aug 09 02:19:28 user nova-compute[72351]: Login to [iface: default, target: iqn.2010-10.org.openstack:3e072274-962d-4e11-985a-5ef6973a3780, portal: 172.16.0.220,3260] successful.
Aug 09 02:19:28 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3e072274-962d-4e11-985a-5ef6973a3780 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3e072274-962d-4e11-985a-5ef6973a3780 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:28 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[60f93cd9-0789-4042-b42f-b6a26cd096cf]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:28 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-06db8269-2459-48b6-acef-2c376c658ce3 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.028s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:28 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a5e1940c-ddca-49fd-9983-e52ce617a49b]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)\ntcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3fbc4f5c-6a7d-4387-bd79-861227a814d3 (non-flash)\ntcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c (non-flash)\ntcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3e072274-962d-4e11-985a-5ef6973a3780 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:28 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-06db8269-2459-48b6-acef-2c376c658ce3 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:19:28 user nova-compute[72351]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)
Aug 09 02:19:28 user nova-compute[72351]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:19:28 user nova-compute[72351]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)
Aug 09 02:19:28 user nova-compute[72351]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3fbc4f5c-6a7d-4387-bd79-861227a814d3 (non-flash)
Aug 09 02:19:28 user nova-compute[72351]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c (non-flash)
Aug 09 02:19:28 user nova-compute[72351]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3e072274-962d-4e11-985a-5ef6973a3780 (non-flash)
Aug 09 02:19:28 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:19:28 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-06db8269-2459-48b6-acef-2c376c658ce3 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:19:28 user nova-compute[72351]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)
Aug 09 02:19:28 user nova-compute[72351]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:19:28 user nova-compute[72351]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)
Aug 09 02:19:28 user nova-compute[72351]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3fbc4f5c-6a7d-4387-bd79-861227a814d3 (non-flash)
Aug 09 02:19:28 user nova-compute[72351]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c (non-flash)
Aug 09 02:19:28 user nova-compute[72351]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3e072274-962d-4e11-985a-5ef6973a3780 (non-flash)
Aug 09 02:19:28 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:19:28 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-06db8269-2459-48b6-acef-2c376c658ce3 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:3e072274-962d-4e11-985a-5ef6973a3780" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.305s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:19:28 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-06db8269-2459-48b6-acef-2c376c658ce3 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 02:19:28 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-06db8269-2459-48b6-acef-2c376c658ce3 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] HCTL ('39', '-', '-', 0) found on session 9 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 02:19:28 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-06db8269-2459-48b6-acef-2c376c658ce3 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Scanning host 39 c: -, t: -, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 02:19:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host39/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host39/scan" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:28 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[aaac8ea0-a3dd-47f4-80ba-81a5030002c0]: (4, ('- - 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:28 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-06db8269-2459-48b6-acef-2c376c658ce3 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Searching for a device in session 9 and hctl ['39', '*', '*', 0] yield: sdh {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:19:28 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-06db8269-2459-48b6-acef-2c376c658ce3 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Connected to sdh using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'qbhgRIds', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 02:19:28 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-8a1e58e1-dcd8-4708-b13e-b997dfbd1229 tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:19:28 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-8a1e58e1-dcd8-4708-b13e-b997dfbd1229 tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:19:28 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-8a1e58e1-dcd8-4708-b13e-b997dfbd1229 tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] No VIF found with MAC fa:16:3e:dc:f0:3c, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:19:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-8a1e58e1-dcd8-4708-b13e-b997dfbd1229 tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Lock "11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 15.044s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:29 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b3264373-6424-4dec-9cb8-4c8bea35179b tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] volume_snapshot_create: create_info: {'snapshot_id': 'e8abd541-0c94-412d-8f2f-83ac8bcbd4e4', 'type': 'qcow2', 'new_file': 'new_file'} {{(pid=72351) volume_snapshot_create /opt/stack/nova/nova/virt/libvirt/driver.py:3566}}
Aug 09 02:19:29 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b3264373-6424-4dec-9cb8-4c8bea35179b tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] snap xml: <domainsnapshot>
Aug 09 02:19:29 user nova-compute[72351]:   <disks>
Aug 09 02:19:29 user nova-compute[72351]:     <disk name="/dev/sdc" snapshot="external" type="file">
Aug 09 02:19:29 user nova-compute[72351]:       <source file="/dev/new_file"/>
Aug 09 02:19:29 user nova-compute[72351]:     </disk>
Aug 09 02:19:29 user nova-compute[72351]:     <disk name="vda" snapshot="no"/>
Aug 09 02:19:29 user nova-compute[72351]:   </disks>
Aug 09 02:19:29 user nova-compute[72351]: </domainsnapshot>
Aug 09 02:19:29 user nova-compute[72351]:  {{(pid=72351) _volume_snapshot_create /opt/stack/nova/nova/virt/libvirt/driver.py:3507}}
Aug 09 02:19:29 user nova-compute[72351]: DEBUG nova.objects.instance [None req-b3264373-6424-4dec-9cb8-4c8bea35179b tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] Lazy-loading 'system_metadata' on Instance uuid 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:19:29 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-b3264373-6424-4dec-9cb8-4c8bea35179b tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Skipping quiescing instance: QEMU guest agent is not enabled.
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver [None req-b3264373-6424-4dec-9cb8-4c8bea35179b tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Unable to create VM snapshot, failing volume_snapshot operation.: libvirt.libvirtError: missing existing file for disk vdb: /dev/new_file: No such file or directory
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Traceback (most recent call last):
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0]   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 3534, in _volume_snapshot_create
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0]     guest.snapshot(snapshot, no_metadata=True, disk_only=True,
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0]   File "/opt/stack/nova/nova/virt/libvirt/guest.py", line 569, in snapshot
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0]     self._domain.snapshotCreateXML(device_xml, flags=flags)
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0]   File "/usr/local/lib/python3.10/dist-packages/eventlet/tpool.py", line 193, in doit
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0]     result = proxy_call(self._autowrap, f, *args, **kwargs)
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0]   File "/usr/local/lib/python3.10/dist-packages/eventlet/tpool.py", line 151, in proxy_call
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0]     rv = execute(f, *args, **kwargs)
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0]   File "/usr/local/lib/python3.10/dist-packages/eventlet/tpool.py", line 132, in execute
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0]     six.reraise(c, e, tb)
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0]   File "/usr/lib/python3/dist-packages/six.py", line 719, in reraise
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0]     raise value
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0]   File "/usr/local/lib/python3.10/dist-packages/eventlet/tpool.py", line 86, in tworker
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0]     rv = meth(*args, **kwargs)
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0]   File "/usr/lib/python3/dist-packages/libvirt.py", line 3070, in snapshotCreateXML
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0]     raise libvirtError('virDomainSnapshotCreateXML() failed')
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] libvirt.libvirtError: missing existing file for disk vdb: /dev/new_file: No such file or directory
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] 
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver [None req-b3264373-6424-4dec-9cb8-4c8bea35179b tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Error occurred during volume_snapshot_create, sending error status to Cinder.: libvirt.libvirtError: missing existing file for disk vdb: /dev/new_file: No such file or directory
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Traceback (most recent call last):
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0]   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 3584, in volume_snapshot_create
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0]     self._volume_snapshot_create(context, instance, guest,
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0]   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 3534, in _volume_snapshot_create
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0]     guest.snapshot(snapshot, no_metadata=True, disk_only=True,
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0]   File "/opt/stack/nova/nova/virt/libvirt/guest.py", line 569, in snapshot
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0]     self._domain.snapshotCreateXML(device_xml, flags=flags)
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0]   File "/usr/local/lib/python3.10/dist-packages/eventlet/tpool.py", line 193, in doit
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0]     result = proxy_call(self._autowrap, f, *args, **kwargs)
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0]   File "/usr/local/lib/python3.10/dist-packages/eventlet/tpool.py", line 151, in proxy_call
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0]     rv = execute(f, *args, **kwargs)
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0]   File "/usr/local/lib/python3.10/dist-packages/eventlet/tpool.py", line 132, in execute
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0]     six.reraise(c, e, tb)
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0]   File "/usr/lib/python3/dist-packages/six.py", line 719, in reraise
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0]     raise value
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0]   File "/usr/local/lib/python3.10/dist-packages/eventlet/tpool.py", line 86, in tworker
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0]     rv = meth(*args, **kwargs)
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0]   File "/usr/lib/python3/dist-packages/libvirt.py", line 3070, in snapshotCreateXML
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0]     raise libvirtError('virDomainSnapshotCreateXML() failed')
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] libvirt.libvirtError: missing existing file for disk vdb: /dev/new_file: No such file or directory
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] 
Aug 09 02:19:29 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c63454b4-c1e6-4455-9c28-9f3d3fcbc5a7 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] volume_snapshot_delete: delete_info: {'volume_id': '3fbc4f5c-6a7d-4387-bd79-861227a814d3'} {{(pid=72351) _volume_snapshot_delete /opt/stack/nova/nova/virt/libvirt/driver.py:3667}}
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver [None req-c63454b4-c1e6-4455-9c28-9f3d3fcbc5a7 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Error occurred during volume_snapshot_delete, sending error status to Cinder.: KeyError: 'type'
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Traceback (most recent call last):
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0]   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 3840, in volume_snapshot_delete
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0]     self._volume_snapshot_delete(context, instance, volume_id,
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0]   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 3670, in _volume_snapshot_delete
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0]     if delete_info['type'] != 'qcow2':
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] KeyError: 'type'
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] 
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver [None req-b3264373-6424-4dec-9cb8-4c8bea35179b tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] Failed to send updated snapshot status to volume service.: nova.exception.SnapshotNotFound: Snapshot e8abd541-0c94-412d-8f2f-83ac8bcbd4e4 could not be found.
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver Traceback (most recent call last):
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 3584, in volume_snapshot_create
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver     self._volume_snapshot_create(context, instance, guest,
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 3534, in _volume_snapshot_create
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver     guest.snapshot(snapshot, no_metadata=True, disk_only=True,
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver   File "/opt/stack/nova/nova/virt/libvirt/guest.py", line 569, in snapshot
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver     self._domain.snapshotCreateXML(device_xml, flags=flags)
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver   File "/usr/local/lib/python3.10/dist-packages/eventlet/tpool.py", line 193, in doit
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver     result = proxy_call(self._autowrap, f, *args, **kwargs)
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver   File "/usr/local/lib/python3.10/dist-packages/eventlet/tpool.py", line 151, in proxy_call
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver     rv = execute(f, *args, **kwargs)
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver   File "/usr/local/lib/python3.10/dist-packages/eventlet/tpool.py", line 132, in execute
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver     six.reraise(c, e, tb)
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver   File "/usr/lib/python3/dist-packages/six.py", line 719, in reraise
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver     raise value
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver   File "/usr/local/lib/python3.10/dist-packages/eventlet/tpool.py", line 86, in tworker
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver     rv = meth(*args, **kwargs)
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver   File "/usr/lib/python3/dist-packages/libvirt.py", line 3070, in snapshotCreateXML
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver     raise libvirtError('virDomainSnapshotCreateXML() failed')
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver libvirt.libvirtError: missing existing file for disk vdb: /dev/new_file: No such file or directory
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver 
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver During handling of the above exception, another exception occurred:
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver 
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver Traceback (most recent call last):
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver   File "/opt/stack/nova/nova/volume/cinder.py", line 466, in wrapper
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver     res = method(self, ctx, snapshot_id, *args, **kwargs)
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver   File "/opt/stack/nova/nova/volume/cinder.py", line 761, in update_snapshot_status
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver     vs.update_snapshot_status(
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver   File "/usr/local/lib/python3.10/dist-packages/cinderclient/v3/volume_snapshots.py", line 225, in update_snapshot_status
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver     return self._action('os-update_snapshot_status',
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver   File "/usr/local/lib/python3.10/dist-packages/cinderclient/v3/volume_snapshots.py", line 221, in _action
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver     resp, body = self.api.client.post(url, body=body)
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver   File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 223, in post
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver     return self._cs_request(url, 'POST', **kwargs)
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver   File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 211, in _cs_request
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver     return self.request(url, method, **kwargs)
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver   File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 197, in request
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver     raise exceptions.from_response(resp, body)
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver cinderclient.exceptions.NotFound: Snapshot e8abd541-0c94-412d-8f2f-83ac8bcbd4e4 could not be found. (HTTP 404) (Request-ID: req-b02caf82-d84f-456b-a9b6-3840a4e582f4)
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver 
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver During handling of the above exception, another exception occurred:
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver 
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver Traceback (most recent call last):
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 3406, in _volume_snapshot_update_status
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver     self._volume_api.update_snapshot_status(context,
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver   File "/opt/stack/nova/nova/volume/cinder.py", line 397, in wrapper
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver     res = method(self, ctx, *args, **kwargs)
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver   File "/opt/stack/nova/nova/volume/cinder.py", line 468, in wrapper
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver     _reraise(exception.SnapshotNotFound(snapshot_id=snapshot_id))
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver   File "/opt/stack/nova/nova/volume/cinder.py", line 488, in _reraise
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver     raise desired_exc.with_traceback(sys.exc_info()[2])
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver   File "/opt/stack/nova/nova/volume/cinder.py", line 466, in wrapper
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver     res = method(self, ctx, snapshot_id, *args, **kwargs)
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver   File "/opt/stack/nova/nova/volume/cinder.py", line 761, in update_snapshot_status
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver     vs.update_snapshot_status(
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver   File "/usr/local/lib/python3.10/dist-packages/cinderclient/v3/volume_snapshots.py", line 225, in update_snapshot_status
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver     return self._action('os-update_snapshot_status',
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver   File "/usr/local/lib/python3.10/dist-packages/cinderclient/v3/volume_snapshots.py", line 221, in _action
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver     resp, body = self.api.client.post(url, body=body)
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver   File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 223, in post
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver     return self._cs_request(url, 'POST', **kwargs)
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver   File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 211, in _cs_request
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver     return self.request(url, method, **kwargs)
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver   File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 197, in request
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver     raise exceptions.from_response(resp, body)
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver nova.exception.SnapshotNotFound: Snapshot e8abd541-0c94-412d-8f2f-83ac8bcbd4e4 could not be found.
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver 
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver [None req-c63454b4-c1e6-4455-9c28-9f3d3fcbc5a7 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] Failed to send updated snapshot status to volume service.: nova.exception.SnapshotNotFound: Snapshot None could not be found.
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver Traceback (most recent call last):
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 3840, in volume_snapshot_delete
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver     self._volume_snapshot_delete(context, instance, volume_id,
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 3670, in _volume_snapshot_delete
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver     if delete_info['type'] != 'qcow2':
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver KeyError: 'type'
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver 
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver During handling of the above exception, another exception occurred:
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver 
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver Traceback (most recent call last):
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver   File "/opt/stack/nova/nova/volume/cinder.py", line 466, in wrapper
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver     res = method(self, ctx, snapshot_id, *args, **kwargs)
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver   File "/opt/stack/nova/nova/volume/cinder.py", line 761, in update_snapshot_status
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver     vs.update_snapshot_status(
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver   File "/usr/local/lib/python3.10/dist-packages/cinderclient/v3/volume_snapshots.py", line 225, in update_snapshot_status
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver     return self._action('os-update_snapshot_status',
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver   File "/usr/local/lib/python3.10/dist-packages/cinderclient/v3/volume_snapshots.py", line 221, in _action
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver     resp, body = self.api.client.post(url, body=body)
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver   File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 223, in post
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver     return self._cs_request(url, 'POST', **kwargs)
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver   File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 211, in _cs_request
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver     return self.request(url, method, **kwargs)
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver   File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 197, in request
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver     raise exceptions.from_response(resp, body)
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver cinderclient.exceptions.NotFound: Snapshot None could not be found. (HTTP 404) (Request-ID: req-6ffe9795-a3cc-43de-9de7-f0ebfed059f5)
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver 
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver During handling of the above exception, another exception occurred:
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver 
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver Traceback (most recent call last):
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 3406, in _volume_snapshot_update_status
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver     self._volume_api.update_snapshot_status(context,
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver   File "/opt/stack/nova/nova/volume/cinder.py", line 397, in wrapper
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver     res = method(self, ctx, *args, **kwargs)
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver   File "/opt/stack/nova/nova/volume/cinder.py", line 468, in wrapper
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver     _reraise(exception.SnapshotNotFound(snapshot_id=snapshot_id))
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver   File "/opt/stack/nova/nova/volume/cinder.py", line 488, in _reraise
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver     raise desired_exc.with_traceback(sys.exc_info()[2])
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver   File "/opt/stack/nova/nova/volume/cinder.py", line 466, in wrapper
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver     res = method(self, ctx, snapshot_id, *args, **kwargs)
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver   File "/opt/stack/nova/nova/volume/cinder.py", line 761, in update_snapshot_status
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver     vs.update_snapshot_status(
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver   File "/usr/local/lib/python3.10/dist-packages/cinderclient/v3/volume_snapshots.py", line 225, in update_snapshot_status
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver     return self._action('os-update_snapshot_status',
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver   File "/usr/local/lib/python3.10/dist-packages/cinderclient/v3/volume_snapshots.py", line 221, in _action
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver     resp, body = self.api.client.post(url, body=body)
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver   File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 223, in post
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver     return self._cs_request(url, 'POST', **kwargs)
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver   File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 211, in _cs_request
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver     return self.request(url, method, **kwargs)
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver   File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 197, in request
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver     raise exceptions.from_response(resp, body)
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver nova.exception.SnapshotNotFound: Snapshot None could not be found.
Aug 09 02:19:29 user nova-compute[72351]: ERROR nova.virt.libvirt.driver 
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server [None req-b3264373-6424-4dec-9cb8-4c8bea35179b tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] Exception during message handling: libvirt.libvirtError: missing existing file for disk vdb: /dev/new_file: No such file or directory
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server     return func(*args, **kwargs)
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/exception_wrapper.py", line 65, in wrapped
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server     with excutils.save_and_reraise_exception():
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server     self.force_reraise()
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server     raise self.value
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server     return f(self, context, *args, **kw)
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/compute/manager.py", line 4425, in volume_snapshot_create
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server     self.driver.volume_snapshot_create(context, instance, volume_id,
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 3587, in volume_snapshot_create
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server     with excutils.save_and_reraise_exception():
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server     self.force_reraise()
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server     raise self.value
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 3584, in volume_snapshot_create
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server     self._volume_snapshot_create(context, instance, guest,
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 3534, in _volume_snapshot_create
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server     guest.snapshot(snapshot, no_metadata=True, disk_only=True,
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/virt/libvirt/guest.py", line 569, in snapshot
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server     self._domain.snapshotCreateXML(device_xml, flags=flags)
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python3.10/dist-packages/eventlet/tpool.py", line 193, in doit
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server     result = proxy_call(self._autowrap, f, *args, **kwargs)
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python3.10/dist-packages/eventlet/tpool.py", line 151, in proxy_call
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server     rv = execute(f, *args, **kwargs)
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python3.10/dist-packages/eventlet/tpool.py", line 132, in execute
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server     six.reraise(c, e, tb)
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server   File "/usr/lib/python3/dist-packages/six.py", line 719, in reraise
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server     raise value
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python3.10/dist-packages/eventlet/tpool.py", line 86, in tworker
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server     rv = meth(*args, **kwargs)
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server   File "/usr/lib/python3/dist-packages/libvirt.py", line 3070, in snapshotCreateXML
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server     raise libvirtError('virDomainSnapshotCreateXML() failed')
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server libvirt.libvirtError: missing existing file for disk vdb: /dev/new_file: No such file or directory
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server 
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server [None req-c63454b4-c1e6-4455-9c28-9f3d3fcbc5a7 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] Exception during message handling: KeyError: 'type'
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server     return func(*args, **kwargs)
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/exception_wrapper.py", line 65, in wrapped
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server     with excutils.save_and_reraise_exception():
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server     self.force_reraise()
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server     raise self.value
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server     return f(self, context, *args, **kw)
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/compute/manager.py", line 4437, in volume_snapshot_delete
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server     self.driver.volume_snapshot_delete(context, instance, volume_id,
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 3843, in volume_snapshot_delete
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server     with excutils.save_and_reraise_exception():
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server     self.force_reraise()
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server     raise self.value
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 3840, in volume_snapshot_delete
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server     self._volume_snapshot_delete(context, instance, volume_id,
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 3670, in _volume_snapshot_delete
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server     if delete_info['type'] != 'qcow2':
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server KeyError: 'type'
Aug 09 02:19:29 user nova-compute[72351]: ERROR oslo_messaging.rpc.server 
Aug 09 02:19:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-9a033c1e-3682-4cf9-a332-e42494075886 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] Acquiring lock "7c5e633d-a32f-4c5e-bb9d-1e40da964bd0" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-9a033c1e-3682-4cf9-a332-e42494075886 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] Lock "7c5e633d-a32f-4c5e-bb9d-1e40da964bd0" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:29 user nova-compute[72351]: INFO nova.compute.manager [None req-9a033c1e-3682-4cf9-a332-e42494075886 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Detaching volume 3fbc4f5c-6a7d-4387-bd79-861227a814d3
Aug 09 02:19:29 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-06db8269-2459-48b6-acef-2c376c658ce3 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.363s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:19:29 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-06db8269-2459-48b6-acef-2c376c658ce3 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] <== connect_volume: return (1892ms) {'type': 'block', 'scsi_wwn': '23338613761616436', 'path': '/dev/sdh'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:19:29 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-06db8269-2459-48b6-acef-2c376c658ce3 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23338613761616436', 'path': '/dev/sdh'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 02:19:29 user nova-compute[72351]: DEBUG nova.objects.instance [None req-06db8269-2459-48b6-acef-2c376c658ce3 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Lazy-loading 'flavor' on Instance uuid 74728ca8-34d4-45e6-a1f2-21c0952ff835 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:19:29 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-06db8269-2459-48b6-acef-2c376c658ce3 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] attach device xml: <disk type="block" device="disk">
Aug 09 02:19:29 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:19:29 user nova-compute[72351]:   <source dev="/dev/sdh"/>
Aug 09 02:19:29 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:19:29 user nova-compute[72351]:   <serial>3e072274-962d-4e11-985a-5ef6973a3780</serial>
Aug 09 02:19:29 user nova-compute[72351]: </disk>
Aug 09 02:19:29 user nova-compute[72351]:  {{(pid=72351) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Aug 09 02:19:29 user nova-compute[72351]: INFO nova.virt.block_device [None req-9a033c1e-3682-4cf9-a332-e42494075886 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Attempting to driver detach volume 3fbc4f5c-6a7d-4387-bd79-861227a814d3 from mountpoint /dev/vdb
Aug 09 02:19:29 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-9a033c1e-3682-4cf9-a332-e42494075886 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] Attempting to detach device vdb from instance 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0 from the persistent domain config. {{(pid=72351) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2477}}
Aug 09 02:19:29 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-9a033c1e-3682-4cf9-a332-e42494075886 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] detach device xml: <disk type="block" device="disk">
Aug 09 02:19:29 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:19:29 user nova-compute[72351]:   <source dev="/dev/sdc"/>
Aug 09 02:19:29 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:19:29 user nova-compute[72351]:   <serial>3fbc4f5c-6a7d-4387-bd79-861227a814d3</serial>
Aug 09 02:19:29 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Aug 09 02:19:29 user nova-compute[72351]: </disk>
Aug 09 02:19:29 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 02:19:29 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-9a033c1e-3682-4cf9-a332-e42494075886 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] Successfully detached device vdb from instance 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0 from the persistent domain config.
Aug 09 02:19:29 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-9a033c1e-3682-4cf9-a332-e42494075886 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0 from the live domain config. {{(pid=72351) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2513}}
Aug 09 02:19:29 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-9a033c1e-3682-4cf9-a332-e42494075886 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] detach device xml: <disk type="block" device="disk">
Aug 09 02:19:29 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:19:29 user nova-compute[72351]:   <source dev="/dev/sdc"/>
Aug 09 02:19:29 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:19:29 user nova-compute[72351]:   <serial>3fbc4f5c-6a7d-4387-bd79-861227a814d3</serial>
Aug 09 02:19:29 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Aug 09 02:19:29 user nova-compute[72351]: </disk>
Aug 09 02:19:29 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-06db8269-2459-48b6-acef-2c376c658ce3 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-06db8269-2459-48b6-acef-2c376c658ce3 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-06db8269-2459-48b6-acef-2c376c658ce3 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] No VIF found with MAC fa:16:3e:7c:0b:0d, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-06db8269-2459-48b6-acef-2c376c658ce3 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Lock "74728ca8-34d4-45e6-a1f2-21c0952ff835" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 13.094s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Received event <DeviceRemovedEvent: 1691547570.5086985, 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0 => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=72351) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2360}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-9a033c1e-3682-4cf9-a332-e42494075886 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0 {{(pid=72351) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2589}}
Aug 09 02:19:30 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-9a033c1e-3682-4cf9-a332-e42494075886 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] Successfully detached device vdb from instance 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0 from the live domain config.
Aug 09 02:19:30 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-9a033c1e-3682-4cf9-a332-e42494075886 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-9a033c1e-3682-4cf9-a332-e42494075886 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:3fbc4f5c-6a7d-4387-bd79-861227a814d3'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'XfchnPRD', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc'}, None), 'kwargs': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:19:30 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-9a033c1e-3682-4cf9-a332-e42494075886 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:19:30 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-9a033c1e-3682-4cf9-a332-e42494075886 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-9a033c1e-3682-4cf9-a332-e42494075886 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-9a033c1e-3682-4cf9-a332-e42494075886 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:3fbc4f5c-6a7d-4387-bd79-861227a814d3', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[5ff5ecb5-b2bf-4f41-a2e7-47e05beccddb]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:3e072274-962d-4e11-985a-5ef6973a3780\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:3fbc4f5c-6a7d-4387-bd79-861227a814d3\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.031s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[8cfac960-fb5c-44ff-98cf-ab81e0476615]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)\ntcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3fbc4f5c-6a7d-4387-bd79-861227a814d3 (non-flash)\ntcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c (non-flash)\ntcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3e072274-962d-4e11-985a-5ef6973a3780 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-9a033c1e-3682-4cf9-a332-e42494075886 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:19:30 user nova-compute[72351]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)
Aug 09 02:19:30 user nova-compute[72351]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:19:30 user nova-compute[72351]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)
Aug 09 02:19:30 user nova-compute[72351]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3fbc4f5c-6a7d-4387-bd79-861227a814d3 (non-flash)
Aug 09 02:19:30 user nova-compute[72351]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c (non-flash)
Aug 09 02:19:30 user nova-compute[72351]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3e072274-962d-4e11-985a-5ef6973a3780 (non-flash)
Aug 09 02:19:30 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-9a033c1e-3682-4cf9-a332-e42494075886 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:19:30 user nova-compute[72351]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)
Aug 09 02:19:30 user nova-compute[72351]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:19:30 user nova-compute[72351]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)
Aug 09 02:19:30 user nova-compute[72351]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3fbc4f5c-6a7d-4387-bd79-861227a814d3 (non-flash)
Aug 09 02:19:30 user nova-compute[72351]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c (non-flash)
Aug 09 02:19:30 user nova-compute[72351]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3e072274-962d-4e11-985a-5ef6973a3780 (non-flash)
Aug 09 02:19:30 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-9a033c1e-3682-4cf9-a332-e42494075886 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fcfdc18acb0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:3fbc4f5c-6a7d-4387-bd79-861227a814d3'): ({'sdc'}, set())}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-9a033c1e-3682-4cf9-a332-e42494075886 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] Removing single pathed devices sdc {{(pid=72351) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[c904b02b-bed0-47f5-9c9f-a81bae15aad1]: (4, ('path checker states:\nup                  7\n\npaths: 0\nbusy: False\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdc {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdc" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d8ba3ec5-a4c0-487c-b205-e7bed14462ce]: (4, ('ok\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-9a033c1e-3682-4cf9-a332-e42494075886 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] Flushing IO for device /dev/sdc {{(pid=72351) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdc {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdc" returned: 0 in 0.015s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[893c8953-9e0b-45c0-961c-40d5ec0ed12f]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-9a033c1e-3682-4cf9-a332-e42494075886 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] Remove SCSI device /dev/sdc with /sys/block/sdc/device/delete {{(pid=72351) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdc/device/delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdc/device/delete" returned: 0 in 0.055s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[8c177522-37d7-4e0c-9234-4de2a8079630]: (4, ('1', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-9a033c1e-3682-4cf9-a332-e42494075886 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] Checking to see if SCSI volumes sdc have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-9a033c1e-3682-4cf9-a332-e42494075886 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] SCSI volumes sdc have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-9a033c1e-3682-4cf9-a332-e42494075886 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:3fbc4f5c-6a7d-4387-bd79-861227a814d3')] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3fbc4f5c-6a7d-4387-bd79-861227a814d3 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3fbc4f5c-6a7d-4387-bd79-861227a814d3 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[583684d1-9f75-4ef6-94c9-c056582bc300]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-9a033c1e-3682-4cf9-a332-e42494075886 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3fbc4f5c-6a7d-4387-bd79-861227a814d3 -p 172.16.0.220:3260 --logout {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] Acquiring lock "11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b" by "nova.compute.manager.ComputeManager.swap_volume.<locals>._do_locked_swap_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] Lock "11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b" acquired by "nova.compute.manager.ComputeManager.swap_volume.<locals>._do_locked_swap_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG os_brick.utils [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[047a0cfe-2d4b-4ca3-98d7-e613996c86e7]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:a52269602ff4\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3fbc4f5c-6a7d-4387-bd79-861227a814d3 -p 172.16.0.220:3260 --logout" returned: 0 in 0.085s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[5c06c643-5153-44db-9436-b61bba6b57aa]: (4, ('Logging out of session [sid: 7, target: iqn.2010-10.org.openstack:3fbc4f5c-6a7d-4387-bd79-861227a814d3, portal: 172.16.0.220,3260]\nLogout of [sid: 7, target: iqn.2010-10.org.openstack:3fbc4f5c-6a7d-4387-bd79-861227a814d3, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-9a033c1e-3682-4cf9-a332-e42494075886 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] iscsiadm ('--logout',): stdout=Logging out of session [sid: 7, target: iqn.2010-10.org.openstack:3fbc4f5c-6a7d-4387-bd79-861227a814d3, portal: 172.16.0.220,3260]
Aug 09 02:19:30 user nova-compute[72351]: Logout of [sid: 7, target: iqn.2010-10.org.openstack:3fbc4f5c-6a7d-4387-bd79-861227a814d3, portal: 172.16.0.220,3260] successful.
Aug 09 02:19:30 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3fbc4f5c-6a7d-4387-bd79-861227a814d3 -p 172.16.0.220:3260 --op delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[01943928-f2cb-4f19-b365-3defedd0777d]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[09b5c048-a4ac-48ea-99f1-831928226432]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[7de09331-264b-4f4e-9e47-fab3a94eadb1]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] Running cmd (subprocess): nvme version {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] 'nvme version' failed. Not Retrying. {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3fbc4f5c-6a7d-4387-bd79-861227a814d3 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.047s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[8c16e8a2-8793-4e8b-b71b-c4e1514a512b]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.nvmeof [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] nvme not present on system {{(pid=72351) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-9a033c1e-3682-4cf9-a332-e42494075886 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-9a033c1e-3682-4cf9-a332-e42494075886 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.390s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-9a033c1e-3682-4cf9-a332-e42494075886 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] <== disconnect_volume: return (391ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-9a033c1e-3682-4cf9-a332-e42494075886 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:19:30 user nova-compute[72351]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Aug 09 02:19:30 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[c4fdae0a-b7b8-47e1-835b-d752ad7e6196]: (4, '') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72351) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] LIGHTOS: no hostnqn found. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG os_brick.utils [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] <== get_connector_properties: return (140ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:a52269602ff4', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:19:30 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:31 user nova-compute[72351]: INFO nova.compute.manager [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Swapping volume 0203626e-04e6-4768-b2c2-e7dbaeed4c6c for edeae21b-4fa4-464b-b7de-da06cf9e8f58
Aug 09 02:19:31 user nova-compute[72351]: DEBUG nova.objects.instance [None req-9a033c1e-3682-4cf9-a332-e42494075886 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] Lazy-loading 'flavor' on Instance uuid 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:19:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-9a033c1e-3682-4cf9-a332-e42494075886 tempest-VolumesAssistedSnapshotsTest-871652921 tempest-VolumesAssistedSnapshotsTest-871652921-project-admin] Lock "7c5e633d-a32f-4c5e-bb9d-1e40da964bd0" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 2.017s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:32 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:32 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-14db3182-0b29-42ac-86c2-9992762b18ce tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Acquiring lock "74728ca8-34d4-45e6-a1f2-21c0952ff835" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:32 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-14db3182-0b29-42ac-86c2-9992762b18ce tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Lock "74728ca8-34d4-45e6-a1f2-21c0952ff835" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:32 user nova-compute[72351]: INFO nova.compute.manager [None req-14db3182-0b29-42ac-86c2-9992762b18ce tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Detaching volume 3e072274-962d-4e11-985a-5ef6973a3780
Aug 09 02:19:32 user nova-compute[72351]: INFO nova.virt.block_device [None req-14db3182-0b29-42ac-86c2-9992762b18ce tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Attempting to driver detach volume 3e072274-962d-4e11-985a-5ef6973a3780 from mountpoint /dev/vdb
Aug 09 02:19:32 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-14db3182-0b29-42ac-86c2-9992762b18ce tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Attempting to detach device vdb from instance 74728ca8-34d4-45e6-a1f2-21c0952ff835 from the persistent domain config. {{(pid=72351) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2477}}
Aug 09 02:19:32 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-14db3182-0b29-42ac-86c2-9992762b18ce tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 02:19:32 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:19:32 user nova-compute[72351]:   <source dev="/dev/sdh"/>
Aug 09 02:19:32 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:19:32 user nova-compute[72351]:   <serial>3e072274-962d-4e11-985a-5ef6973a3780</serial>
Aug 09 02:19:32 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Aug 09 02:19:32 user nova-compute[72351]: </disk>
Aug 09 02:19:32 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 02:19:32 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-14db3182-0b29-42ac-86c2-9992762b18ce tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Successfully detached device vdb from instance 74728ca8-34d4-45e6-a1f2-21c0952ff835 from the persistent domain config.
Aug 09 02:19:32 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-14db3182-0b29-42ac-86c2-9992762b18ce tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 74728ca8-34d4-45e6-a1f2-21c0952ff835 from the live domain config. {{(pid=72351) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2513}}
Aug 09 02:19:32 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-14db3182-0b29-42ac-86c2-9992762b18ce tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 02:19:32 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:19:32 user nova-compute[72351]:   <source dev="/dev/sdh"/>
Aug 09 02:19:32 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:19:32 user nova-compute[72351]:   <serial>3e072274-962d-4e11-985a-5ef6973a3780</serial>
Aug 09 02:19:32 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Aug 09 02:19:32 user nova-compute[72351]: </disk>
Aug 09 02:19:32 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 02:19:33 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Received event <DeviceRemovedEvent: 1691547573.14652, 74728ca8-34d4-45e6-a1f2-21c0952ff835 => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=72351) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2360}}
Aug 09 02:19:33 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-14db3182-0b29-42ac-86c2-9992762b18ce tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 74728ca8-34d4-45e6-a1f2-21c0952ff835 {{(pid=72351) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2589}}
Aug 09 02:19:33 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-14db3182-0b29-42ac-86c2-9992762b18ce tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Successfully detached device vdb from instance 74728ca8-34d4-45e6-a1f2-21c0952ff835 from the live domain config.
Aug 09 02:19:33 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-14db3182-0b29-42ac-86c2-9992762b18ce tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 02:19:33 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-14db3182-0b29-42ac-86c2-9992762b18ce tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:3e072274-962d-4e11-985a-5ef6973a3780'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'qbhgRIds', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdh'}, None), 'kwargs': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:19:33 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-14db3182-0b29-42ac-86c2-9992762b18ce tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:19:33 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-14db3182-0b29-42ac-86c2-9992762b18ce tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:19:33 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-14db3182-0b29-42ac-86c2-9992762b18ce tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:19:33 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-14db3182-0b29-42ac-86c2-9992762b18ce tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:3e072274-962d-4e11-985a-5ef6973a3780', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 02:19:33 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:33 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.025s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:33 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[fe61c303-6c42-4a69-acc5-3acff2b9e176]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:3e072274-962d-4e11-985a-5ef6973a3780\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:33 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:33 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.030s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:33 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[6af28edd-1e56-4edb-b1c8-c3f8a3a0fc7b]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)\ntcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c (non-flash)\ntcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3e072274-962d-4e11-985a-5ef6973a3780 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:33 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-14db3182-0b29-42ac-86c2-9992762b18ce tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:19:33 user nova-compute[72351]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)
Aug 09 02:19:33 user nova-compute[72351]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:19:33 user nova-compute[72351]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)
Aug 09 02:19:33 user nova-compute[72351]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c (non-flash)
Aug 09 02:19:33 user nova-compute[72351]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3e072274-962d-4e11-985a-5ef6973a3780 (non-flash)
Aug 09 02:19:33 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:19:33 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-14db3182-0b29-42ac-86c2-9992762b18ce tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:19:33 user nova-compute[72351]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)
Aug 09 02:19:33 user nova-compute[72351]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:19:33 user nova-compute[72351]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)
Aug 09 02:19:33 user nova-compute[72351]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c (non-flash)
Aug 09 02:19:33 user nova-compute[72351]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3e072274-962d-4e11-985a-5ef6973a3780 (non-flash)
Aug 09 02:19:33 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:19:33 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-14db3182-0b29-42ac-86c2-9992762b18ce tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fcfdc4cfeb0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:3e072274-962d-4e11-985a-5ef6973a3780'): ({'sdh'}, set())}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 02:19:33 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-14db3182-0b29-42ac-86c2-9992762b18ce tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Removing single pathed devices sdh {{(pid=72351) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Aug 09 02:19:33 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:33 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:33 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[85f23624-bbc5-41af-9525-d849de9e7745]: (4, ('path checker states:\nup                  6\n\npaths: 0\nbusy: False\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:33 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdh {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:33 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdh" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:33 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[b911ba18-e045-4be5-ad03-f790b4a3ec37]: (4, ('ok\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:33 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-14db3182-0b29-42ac-86c2-9992762b18ce tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Flushing IO for device /dev/sdh {{(pid=72351) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Aug 09 02:19:33 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdh {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:33 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdh" returned: 0 in 0.013s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:33 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e8d51f22-6e55-41aa-b70a-1da4fc6b32bc]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:33 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-14db3182-0b29-42ac-86c2-9992762b18ce tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Remove SCSI device /dev/sdh with /sys/block/sdh/device/delete {{(pid=72351) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Aug 09 02:19:33 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdh/device/delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:33 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdh/device/delete" returned: 0 in 0.052s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:33 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[28d601ec-8424-4ade-a2b4-bf1060880a23]: (4, ('1', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:33 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-14db3182-0b29-42ac-86c2-9992762b18ce tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Checking to see if SCSI volumes sdh have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Aug 09 02:19:33 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-14db3182-0b29-42ac-86c2-9992762b18ce tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] SCSI volumes sdh have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Aug 09 02:19:33 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-14db3182-0b29-42ac-86c2-9992762b18ce tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:3e072274-962d-4e11-985a-5ef6973a3780')] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 02:19:33 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3e072274-962d-4e11-985a-5ef6973a3780 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:33 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3e072274-962d-4e11-985a-5ef6973a3780 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:33 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[7fd39ff0-bda6-4517-880a-b128b4c35551]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:33 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-14db3182-0b29-42ac-86c2-9992762b18ce tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:33 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3e072274-962d-4e11-985a-5ef6973a3780 -p 172.16.0.220:3260 --logout {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:33 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3e072274-962d-4e11-985a-5ef6973a3780 -p 172.16.0.220:3260 --logout" returned: 0 in 0.056s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:33 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[b28f489f-a5af-43e5-8a14-e3097dec3886]: (4, ('Logging out of session [sid: 9, target: iqn.2010-10.org.openstack:3e072274-962d-4e11-985a-5ef6973a3780, portal: 172.16.0.220,3260]\nLogout of [sid: 9, target: iqn.2010-10.org.openstack:3e072274-962d-4e11-985a-5ef6973a3780, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:33 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-14db3182-0b29-42ac-86c2-9992762b18ce tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 9, target: iqn.2010-10.org.openstack:3e072274-962d-4e11-985a-5ef6973a3780, portal: 172.16.0.220,3260]
Aug 09 02:19:33 user nova-compute[72351]: Logout of [sid: 9, target: iqn.2010-10.org.openstack:3e072274-962d-4e11-985a-5ef6973a3780, portal: 172.16.0.220,3260] successful.
Aug 09 02:19:33 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:33 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3e072274-962d-4e11-985a-5ef6973a3780 -p 172.16.0.220:3260 --op delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:33 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3e072274-962d-4e11-985a-5ef6973a3780 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.031s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:33 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[fa87e8b8-0edb-4c57-8e9d-ac786bbdfe40]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:33 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-14db3182-0b29-42ac-86c2-9992762b18ce tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:33 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-14db3182-0b29-42ac-86c2-9992762b18ce tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.315s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:19:33 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-14db3182-0b29-42ac-86c2-9992762b18ce tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] <== disconnect_volume: return (315ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:19:33 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-14db3182-0b29-42ac-86c2-9992762b18ce tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 02:19:34 user nova-compute[72351]: DEBUG nova.objects.instance [None req-14db3182-0b29-42ac-86c2-9992762b18ce tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Lazy-loading 'flavor' on Instance uuid 74728ca8-34d4-45e6-a1f2-21c0952ff835 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:19:34 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-14db3182-0b29-42ac-86c2-9992762b18ce tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Lock "74728ca8-34d4-45e6-a1f2-21c0952ff835" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.802s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-67596b5e-f47e-4ec6-9fe5-8e383162081c tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Acquiring lock "d0c7002c-8a3a-4bc5-940c-63cf4264668b" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-67596b5e-f47e-4ec6-9fe5-8e383162081c tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Lock "d0c7002c-8a3a-4bc5-940c-63cf4264668b" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-67596b5e-f47e-4ec6-9fe5-8e383162081c tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Acquiring lock "d0c7002c-8a3a-4bc5-940c-63cf4264668b-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-67596b5e-f47e-4ec6-9fe5-8e383162081c tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Lock "d0c7002c-8a3a-4bc5-940c-63cf4264668b-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-67596b5e-f47e-4ec6-9fe5-8e383162081c tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Lock "d0c7002c-8a3a-4bc5-940c-63cf4264668b-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:35 user nova-compute[72351]: INFO nova.compute.manager [None req-67596b5e-f47e-4ec6-9fe5-8e383162081c tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Terminating instance
Aug 09 02:19:35 user nova-compute[72351]: DEBUG nova.compute.manager [None req-67596b5e-f47e-4ec6-9fe5-8e383162081c tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Start destroying the instance on the hypervisor. {{(pid=72351) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3124}}
Aug 09 02:19:35 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:35 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:35 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:35 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:35 user nova-compute[72351]: DEBUG nova.compute.manager [req-d3deb4c2-2ed2-4f59-ab5a-507fa53449f5 req-0714743f-0390-44b9-9b74-1ab43789139e service nova] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Received event network-vif-unplugged-9734c099-ce77-4e2d-ab87-6ae4ab56c56b {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:19:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d3deb4c2-2ed2-4f59-ab5a-507fa53449f5 req-0714743f-0390-44b9-9b74-1ab43789139e service nova] Acquiring lock "d0c7002c-8a3a-4bc5-940c-63cf4264668b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d3deb4c2-2ed2-4f59-ab5a-507fa53449f5 req-0714743f-0390-44b9-9b74-1ab43789139e service nova] Lock "d0c7002c-8a3a-4bc5-940c-63cf4264668b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d3deb4c2-2ed2-4f59-ab5a-507fa53449f5 req-0714743f-0390-44b9-9b74-1ab43789139e service nova] Lock "d0c7002c-8a3a-4bc5-940c-63cf4264668b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:35 user nova-compute[72351]: DEBUG nova.compute.manager [req-d3deb4c2-2ed2-4f59-ab5a-507fa53449f5 req-0714743f-0390-44b9-9b74-1ab43789139e service nova] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] No waiting events found dispatching network-vif-unplugged-9734c099-ce77-4e2d-ab87-6ae4ab56c56b {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:19:35 user nova-compute[72351]: DEBUG nova.compute.manager [req-d3deb4c2-2ed2-4f59-ab5a-507fa53449f5 req-0714743f-0390-44b9-9b74-1ab43789139e service nova] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Received event network-vif-unplugged-9734c099-ce77-4e2d-ab87-6ae4ab56c56b for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:19:35 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:35 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:35 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:35 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Instance destroyed successfully.
Aug 09 02:19:35 user nova-compute[72351]: DEBUG nova.objects.instance [None req-67596b5e-f47e-4ec6-9fe5-8e383162081c tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Lazy-loading 'resources' on Instance uuid d0c7002c-8a3a-4bc5-940c-63cf4264668b {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:19:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-67596b5e-f47e-4ec6-9fe5-8e383162081c tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:17:43Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-VolumesActionsTest-instance-2017545637',display_name='tempest-VolumesActionsTest-instance-2017545637',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-volumesactionstest-instance-2017545637',id=7,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:17:52Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='5209610aaaa34f7a8ffb7757dbfac85d',ramdisk_id='',reservation_id='r-huk5pbgj',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-VolumesActionsTest-1343359891',owner_user_name='tempest-VolumesActionsTest-1343359891-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:17:53Z,user_data=None,user_id='00a45ab3cb2d43028418af1ae209da51',uuid=d0c7002c-8a3a-4bc5-940c-63cf4264668b,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "9734c099-ce77-4e2d-ab87-6ae4ab56c56b", "address": "fa:16:3e:6b:a4:c9", "network": {"id": "fc48221d-cf2e-4871-9741-c102904e10a6", "bridge": "br-int", "label": "tempest-VolumesActionsTest-857807030-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5209610aaaa34f7a8ffb7757dbfac85d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9734c099-ce", "ovs_interfaceid": "9734c099-ce77-4e2d-ab87-6ae4ab56c56b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:19:35 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-67596b5e-f47e-4ec6-9fe5-8e383162081c tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Converting VIF {"id": "9734c099-ce77-4e2d-ab87-6ae4ab56c56b", "address": "fa:16:3e:6b:a4:c9", "network": {"id": "fc48221d-cf2e-4871-9741-c102904e10a6", "bridge": "br-int", "label": "tempest-VolumesActionsTest-857807030-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5209610aaaa34f7a8ffb7757dbfac85d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9734c099-ce", "ovs_interfaceid": "9734c099-ce77-4e2d-ab87-6ae4ab56c56b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:19:35 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-67596b5e-f47e-4ec6-9fe5-8e383162081c tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:6b:a4:c9,bridge_name='br-int',has_traffic_filtering=True,id=9734c099-ce77-4e2d-ab87-6ae4ab56c56b,network=Network(fc48221d-cf2e-4871-9741-c102904e10a6),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap9734c099-ce') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:19:35 user nova-compute[72351]: DEBUG os_vif [None req-67596b5e-f47e-4ec6-9fe5-8e383162081c tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:6b:a4:c9,bridge_name='br-int',has_traffic_filtering=True,id=9734c099-ce77-4e2d-ab87-6ae4ab56c56b,network=Network(fc48221d-cf2e-4871-9741-c102904e10a6),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap9734c099-ce') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:19:35 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:35 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap9734c099-ce, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:19:35 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:19:35 user nova-compute[72351]: INFO os_vif [None req-67596b5e-f47e-4ec6-9fe5-8e383162081c tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:6b:a4:c9,bridge_name='br-int',has_traffic_filtering=True,id=9734c099-ce77-4e2d-ab87-6ae4ab56c56b,network=Network(fc48221d-cf2e-4871-9741-c102904e10a6),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap9734c099-ce')
Aug 09 02:19:35 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-67596b5e-f47e-4ec6-9fe5-8e383162081c tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Deleting instance files /opt/stack/data/nova/instances/d0c7002c-8a3a-4bc5-940c-63cf4264668b_del
Aug 09 02:19:35 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-67596b5e-f47e-4ec6-9fe5-8e383162081c tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Deletion of /opt/stack/data/nova/instances/d0c7002c-8a3a-4bc5-940c-63cf4264668b_del complete
Aug 09 02:19:35 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:35 user nova-compute[72351]: INFO nova.compute.manager [None req-67596b5e-f47e-4ec6-9fe5-8e383162081c tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Took 0.99 seconds to destroy the instance on the hypervisor.
Aug 09 02:19:35 user nova-compute[72351]: DEBUG oslo.service.loopingcall [None req-67596b5e-f47e-4ec6-9fe5-8e383162081c tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72351) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Aug 09 02:19:35 user nova-compute[72351]: DEBUG nova.compute.manager [-] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Deallocating network for instance {{(pid=72351) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2263}}
Aug 09 02:19:35 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] deallocate_for_instance() {{(pid=72351) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1795}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-4f194355-115f-4a7e-bc98-987e987f1cf0 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f194355-115f-4a7e-bc98-987e987f1cf0 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:643c3e80-8b8a-40c2-a979-579d65ba653a'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'tzELmHlW', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:19:36 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-4f194355-115f-4a7e-bc98-987e987f1cf0 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:19:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-4f194355-115f-4a7e-bc98-987e987f1cf0 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-4f194355-115f-4a7e-bc98-987e987f1cf0 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:19:36 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-4f194355-115f-4a7e-bc98-987e987f1cf0 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:19:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-4f194355-115f-4a7e-bc98-987e987f1cf0 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:643c3e80-8b8a-40c2-a979-579d65ba653a" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-4f194355-115f-4a7e-bc98-987e987f1cf0 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:643c3e80-8b8a-40c2-a979-579d65ba653a" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:19:36 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-4f194355-115f-4a7e-bc98-987e987f1cf0 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 02:19:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:643c3e80-8b8a-40c2-a979-579d65ba653a -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:643c3e80-8b8a-40c2-a979-579d65ba653a -p 172.16.0.220:3260" returned: 21 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[2f604112-e369-4b42-8be4-d411303eb407]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f194355-115f-4a7e-bc98-987e987f1cf0 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Aug 09 02:19:36 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:643c3e80-8b8a-40c2-a979-579d65ba653a -p 172.16.0.220:3260 --interface default --op new {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:643c3e80-8b8a-40c2-a979-579d65ba653a -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[336268bb-5642-4ace-a95e-908511459970]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:643c3e80-8b8a-40c2-a979-579d65ba653a] added\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f194355-115f-4a7e-bc98-987e987f1cf0 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:643c3e80-8b8a-40c2-a979-579d65ba653a] added
Aug 09 02:19:36 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:643c3e80-8b8a-40c2-a979-579d65ba653a -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:643c3e80-8b8a-40c2-a979-579d65ba653a -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.025s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[35ce2917-eaf2-41c5-a9ae-cbcd1a1bb1c5]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f194355-115f-4a7e-bc98-987e987f1cf0 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:643c3e80-8b8a-40c2-a979-579d65ba653a -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:643c3e80-8b8a-40c2-a979-579d65ba653a -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.026s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[0663844e-2b33-42e0-864d-00c3b37dbcd7]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f194355-115f-4a7e-bc98-987e987f1cf0 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:643c3e80-8b8a-40c2-a979-579d65ba653a -p 172.16.0.220:3260 --op update -n node.session.auth.username -v tzELmHlW {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:643c3e80-8b8a-40c2-a979-579d65ba653a -p 172.16.0.220:3260 --op update -n node.session.auth.username -v tzELmHlW" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a46bbbae-cbd1-496c-bf38-5bceb4d1f08b]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f194355-115f-4a7e-bc98-987e987f1cf0 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'tzELmHlW'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:643c3e80-8b8a-40c2-a979-579d65ba653a -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:643c3e80-8b8a-40c2-a979-579d65ba653a -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.028s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[6aa590b1-8661-4ba1-a13c-9a8bb10d47a5]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f194355-115f-4a7e-bc98-987e987f1cf0 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.040s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ffad3f2a-cecf-4b73-8dd9-e7d511bade18]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)\ntcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f194355-115f-4a7e-bc98-987e987f1cf0 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:19:36 user nova-compute[72351]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)
Aug 09 02:19:36 user nova-compute[72351]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:19:36 user nova-compute[72351]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)
Aug 09 02:19:36 user nova-compute[72351]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c (non-flash)
Aug 09 02:19:36 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f194355-115f-4a7e-bc98-987e987f1cf0 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:19:36 user nova-compute[72351]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)
Aug 09 02:19:36 user nova-compute[72351]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:19:36 user nova-compute[72351]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)
Aug 09 02:19:36 user nova-compute[72351]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c (non-flash)
Aug 09 02:19:36 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:643c3e80-8b8a-40c2-a979-579d65ba653a -p 172.16.0.220:3260 --login {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG nova.compute.manager [req-2860abf9-43d2-4d4f-8fa2-9e26c93cd57d req-37922111-b7c2-40f3-9f7d-bde2407b99d4 service nova] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Received event network-vif-deleted-9734c099-ce77-4e2d-ab87-6ae4ab56c56b {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:19:36 user nova-compute[72351]: INFO nova.compute.manager [req-2860abf9-43d2-4d4f-8fa2-9e26c93cd57d req-37922111-b7c2-40f3-9f7d-bde2407b99d4 service nova] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Neutron deleted interface 9734c099-ce77-4e2d-ab87-6ae4ab56c56b; detaching it from the instance and deleting it from the info cache
Aug 09 02:19:36 user nova-compute[72351]: DEBUG nova.network.neutron [req-2860abf9-43d2-4d4f-8fa2-9e26c93cd57d req-37922111-b7c2-40f3-9f7d-bde2407b99d4 service nova] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:36 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Took 0.65 seconds to deallocate network for instance.
Aug 09 02:19:36 user nova-compute[72351]: DEBUG nova.compute.manager [req-2860abf9-43d2-4d4f-8fa2-9e26c93cd57d req-37922111-b7c2-40f3-9f7d-bde2407b99d4 service nova] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Detach interface failed, port_id=9734c099-ce77-4e2d-ab87-6ae4ab56c56b, reason: Instance d0c7002c-8a3a-4bc5-940c-63cf4264668b could not be found. {{(pid=72351) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10866}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:643c3e80-8b8a-40c2-a979-579d65ba653a -p 172.16.0.220:3260 --login" returned: 0 in 0.083s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[41930c77-8b3d-43b8-abc3-562d05016545]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:643c3e80-8b8a-40c2-a979-579d65ba653a, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:643c3e80-8b8a-40c2-a979-579d65ba653a, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f194355-115f-4a7e-bc98-987e987f1cf0 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:643c3e80-8b8a-40c2-a979-579d65ba653a, portal: 172.16.0.220,3260]
Aug 09 02:19:36 user nova-compute[72351]: Login to [iface: default, target: iqn.2010-10.org.openstack:643c3e80-8b8a-40c2-a979-579d65ba653a, portal: 172.16.0.220,3260] successful.
Aug 09 02:19:36 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:643c3e80-8b8a-40c2-a979-579d65ba653a -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:643c3e80-8b8a-40c2-a979-579d65ba653a -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.026s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[3c9b506c-8a35-4ca5-93a1-3a982a7f88e7]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f194355-115f-4a7e-bc98-987e987f1cf0 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-67596b5e-f47e-4ec6-9fe5-8e383162081c tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-67596b5e-f47e-4ec6-9fe5-8e383162081c tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.034s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[9783c7f5-532e-40d8-b49e-84d7267529ce]: (4, ('tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:643c3e80-8b8a-40c2-a979-579d65ba653a (non-flash)\ntcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)\ntcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f194355-115f-4a7e-bc98-987e987f1cf0 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:643c3e80-8b8a-40c2-a979-579d65ba653a (non-flash)
Aug 09 02:19:36 user nova-compute[72351]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:19:36 user nova-compute[72351]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)
Aug 09 02:19:36 user nova-compute[72351]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:19:36 user nova-compute[72351]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)
Aug 09 02:19:36 user nova-compute[72351]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c (non-flash)
Aug 09 02:19:36 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f194355-115f-4a7e-bc98-987e987f1cf0 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] iscsi session list stdout=tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:643c3e80-8b8a-40c2-a979-579d65ba653a (non-flash)
Aug 09 02:19:36 user nova-compute[72351]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:19:36 user nova-compute[72351]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)
Aug 09 02:19:36 user nova-compute[72351]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:19:36 user nova-compute[72351]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)
Aug 09 02:19:36 user nova-compute[72351]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c (non-flash)
Aug 09 02:19:36 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-4f194355-115f-4a7e-bc98-987e987f1cf0 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:643c3e80-8b8a-40c2-a979-579d65ba653a" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.380s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f194355-115f-4a7e-bc98-987e987f1cf0 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-4f194355-115f-4a7e-bc98-987e987f1cf0 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] HCTL ('34', '-', '-', 0) found on session 10 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-4f194355-115f-4a7e-bc98-987e987f1cf0 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Scanning host 34 c: -, t: -, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.024s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[502f029a-dc97-42c1-a476-73ba60fb1276]: (4, ('- - 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-4f194355-115f-4a7e-bc98-987e987f1cf0 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Searching for a device in session 10 and hctl ['34', '*', '*', 0] yield: sdc {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:19:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f194355-115f-4a7e-bc98-987e987f1cf0 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Connected to sdc using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'tzELmHlW', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 02:19:37 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-67596b5e-f47e-4ec6-9fe5-8e383162081c tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:19:37 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-67596b5e-f47e-4ec6-9fe5-8e383162081c tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:19:37 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-67596b5e-f47e-4ec6-9fe5-8e383162081c tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.527s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:37 user nova-compute[72351]: INFO nova.scheduler.client.report [None req-67596b5e-f47e-4ec6-9fe5-8e383162081c tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Deleted allocations for instance d0c7002c-8a3a-4bc5-940c-63cf4264668b
Aug 09 02:19:37 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-67596b5e-f47e-4ec6-9fe5-8e383162081c tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Lock "d0c7002c-8a3a-4bc5-940c-63cf4264668b" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.455s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:37 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:37 user nova-compute[72351]: DEBUG nova.compute.manager [req-cde39972-cb6d-4fdc-88fa-b5f3a837f459 req-1b7fd6df-a4dc-4852-9b50-a885256756c5 service nova] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Received event network-vif-plugged-9734c099-ce77-4e2d-ab87-6ae4ab56c56b {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:19:37 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-cde39972-cb6d-4fdc-88fa-b5f3a837f459 req-1b7fd6df-a4dc-4852-9b50-a885256756c5 service nova] Acquiring lock "d0c7002c-8a3a-4bc5-940c-63cf4264668b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:37 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-cde39972-cb6d-4fdc-88fa-b5f3a837f459 req-1b7fd6df-a4dc-4852-9b50-a885256756c5 service nova] Lock "d0c7002c-8a3a-4bc5-940c-63cf4264668b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:37 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-cde39972-cb6d-4fdc-88fa-b5f3a837f459 req-1b7fd6df-a4dc-4852-9b50-a885256756c5 service nova] Lock "d0c7002c-8a3a-4bc5-940c-63cf4264668b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:37 user nova-compute[72351]: DEBUG nova.compute.manager [req-cde39972-cb6d-4fdc-88fa-b5f3a837f459 req-1b7fd6df-a4dc-4852-9b50-a885256756c5 service nova] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] No waiting events found dispatching network-vif-plugged-9734c099-ce77-4e2d-ab87-6ae4ab56c56b {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:19:37 user nova-compute[72351]: WARNING nova.compute.manager [req-cde39972-cb6d-4fdc-88fa-b5f3a837f459 req-1b7fd6df-a4dc-4852-9b50-a885256756c5 service nova] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Received unexpected event network-vif-plugged-9734c099-ce77-4e2d-ab87-6ae4ab56c56b for instance with vm_state deleted and task_state None.
Aug 09 02:19:37 user nova-compute[72351]: DEBUG nova.compute.manager [req-cde39972-cb6d-4fdc-88fa-b5f3a837f459 req-1b7fd6df-a4dc-4852-9b50-a885256756c5 service nova] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Received event network-vif-plugged-9734c099-ce77-4e2d-ab87-6ae4ab56c56b {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:19:37 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-cde39972-cb6d-4fdc-88fa-b5f3a837f459 req-1b7fd6df-a4dc-4852-9b50-a885256756c5 service nova] Acquiring lock "d0c7002c-8a3a-4bc5-940c-63cf4264668b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:37 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-cde39972-cb6d-4fdc-88fa-b5f3a837f459 req-1b7fd6df-a4dc-4852-9b50-a885256756c5 service nova] Lock "d0c7002c-8a3a-4bc5-940c-63cf4264668b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:37 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-cde39972-cb6d-4fdc-88fa-b5f3a837f459 req-1b7fd6df-a4dc-4852-9b50-a885256756c5 service nova] Lock "d0c7002c-8a3a-4bc5-940c-63cf4264668b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:37 user nova-compute[72351]: DEBUG nova.compute.manager [req-cde39972-cb6d-4fdc-88fa-b5f3a837f459 req-1b7fd6df-a4dc-4852-9b50-a885256756c5 service nova] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] No waiting events found dispatching network-vif-plugged-9734c099-ce77-4e2d-ab87-6ae4ab56c56b {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:19:37 user nova-compute[72351]: WARNING nova.compute.manager [req-cde39972-cb6d-4fdc-88fa-b5f3a837f459 req-1b7fd6df-a4dc-4852-9b50-a885256756c5 service nova] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Received unexpected event network-vif-plugged-9734c099-ce77-4e2d-ab87-6ae4ab56c56b for instance with vm_state deleted and task_state None.
Aug 09 02:19:37 user nova-compute[72351]: DEBUG nova.compute.manager [req-cde39972-cb6d-4fdc-88fa-b5f3a837f459 req-1b7fd6df-a4dc-4852-9b50-a885256756c5 service nova] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Received event network-vif-plugged-9734c099-ce77-4e2d-ab87-6ae4ab56c56b {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:19:37 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-cde39972-cb6d-4fdc-88fa-b5f3a837f459 req-1b7fd6df-a4dc-4852-9b50-a885256756c5 service nova] Acquiring lock "d0c7002c-8a3a-4bc5-940c-63cf4264668b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:37 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-cde39972-cb6d-4fdc-88fa-b5f3a837f459 req-1b7fd6df-a4dc-4852-9b50-a885256756c5 service nova] Lock "d0c7002c-8a3a-4bc5-940c-63cf4264668b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:37 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-cde39972-cb6d-4fdc-88fa-b5f3a837f459 req-1b7fd6df-a4dc-4852-9b50-a885256756c5 service nova] Lock "d0c7002c-8a3a-4bc5-940c-63cf4264668b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:37 user nova-compute[72351]: DEBUG nova.compute.manager [req-cde39972-cb6d-4fdc-88fa-b5f3a837f459 req-1b7fd6df-a4dc-4852-9b50-a885256756c5 service nova] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] No waiting events found dispatching network-vif-plugged-9734c099-ce77-4e2d-ab87-6ae4ab56c56b {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:19:37 user nova-compute[72351]: WARNING nova.compute.manager [req-cde39972-cb6d-4fdc-88fa-b5f3a837f459 req-1b7fd6df-a4dc-4852-9b50-a885256756c5 service nova] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Received unexpected event network-vif-plugged-9734c099-ce77-4e2d-ab87-6ae4ab56c56b for instance with vm_state deleted and task_state None.
Aug 09 02:19:37 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-4f194355-115f-4a7e-bc98-987e987f1cf0 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.449s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:19:37 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f194355-115f-4a7e-bc98-987e987f1cf0 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] <== connect_volume: return (1451ms) {'type': 'block', 'scsi_wwn': '23439393031373364', 'path': '/dev/sdc'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:19:37 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-4f194355-115f-4a7e-bc98-987e987f1cf0 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23439393031373364', 'path': '/dev/sdc'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 02:19:37 user nova-compute[72351]: DEBUG nova.objects.instance [None req-4f194355-115f-4a7e-bc98-987e987f1cf0 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Lazy-loading 'flavor' on Instance uuid 50302d76-f6b9-4fe3-b55a-4a01e483e2e3 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:19:37 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-4f194355-115f-4a7e-bc98-987e987f1cf0 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] attach device xml: <disk type="block" device="disk">
Aug 09 02:19:37 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:19:37 user nova-compute[72351]:   <source dev="/dev/sdc"/>
Aug 09 02:19:37 user nova-compute[72351]:   <target dev="sdc" bus="scsi"/>
Aug 09 02:19:37 user nova-compute[72351]:   <serial>643c3e80-8b8a-40c2-a979-579d65ba653a</serial>
Aug 09 02:19:37 user nova-compute[72351]:   <address type="drive" controller="0" unit="2"/>
Aug 09 02:19:37 user nova-compute[72351]: </disk>
Aug 09 02:19:37 user nova-compute[72351]:  {{(pid=72351) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Aug 09 02:19:38 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-4f194355-115f-4a7e-bc98-987e987f1cf0 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] No BDM found with device name sda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:19:38 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-4f194355-115f-4a7e-bc98-987e987f1cf0 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] No BDM found with device name sdb, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:19:38 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-4f194355-115f-4a7e-bc98-987e987f1cf0 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] No BDM found with device name sdc, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:19:38 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-4f194355-115f-4a7e-bc98-987e987f1cf0 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] No VIF found with MAC fa:16:3e:6e:b8:a5, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:19:38 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-4f194355-115f-4a7e-bc98-987e987f1cf0 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Lock "50302d76-f6b9-4fe3-b55a-4a01e483e2e3" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 20.788s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:39 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5dd81833-196e-438b-a671-f7ae54f5f711 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Acquiring lock "50302d76-f6b9-4fe3-b55a-4a01e483e2e3" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:39 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5dd81833-196e-438b-a671-f7ae54f5f711 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Lock "50302d76-f6b9-4fe3-b55a-4a01e483e2e3" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:39 user nova-compute[72351]: INFO nova.compute.manager [None req-5dd81833-196e-438b-a671-f7ae54f5f711 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Detaching volume 643c3e80-8b8a-40c2-a979-579d65ba653a
Aug 09 02:19:39 user nova-compute[72351]: INFO nova.virt.block_device [None req-5dd81833-196e-438b-a671-f7ae54f5f711 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Attempting to driver detach volume 643c3e80-8b8a-40c2-a979-579d65ba653a from mountpoint /dev/sdc
Aug 09 02:19:39 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-5dd81833-196e-438b-a671-f7ae54f5f711 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Attempting to detach device sdc from instance 50302d76-f6b9-4fe3-b55a-4a01e483e2e3 from the persistent domain config. {{(pid=72351) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2477}}
Aug 09 02:19:39 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-5dd81833-196e-438b-a671-f7ae54f5f711 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 02:19:39 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:19:39 user nova-compute[72351]:   <source dev="/dev/sdc"/>
Aug 09 02:19:39 user nova-compute[72351]:   <target dev="sdc" bus="scsi"/>
Aug 09 02:19:39 user nova-compute[72351]:   <serial>643c3e80-8b8a-40c2-a979-579d65ba653a</serial>
Aug 09 02:19:39 user nova-compute[72351]:   <address type="drive" controller="0" bus="0" target="0" unit="2"/>
Aug 09 02:19:39 user nova-compute[72351]: </disk>
Aug 09 02:19:39 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 02:19:39 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-5dd81833-196e-438b-a671-f7ae54f5f711 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Successfully detached device sdc from instance 50302d76-f6b9-4fe3-b55a-4a01e483e2e3 from the persistent domain config.
Aug 09 02:19:39 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-5dd81833-196e-438b-a671-f7ae54f5f711 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] (1/8): Attempting to detach device sdc with device alias scsi0-0-0-2 from instance 50302d76-f6b9-4fe3-b55a-4a01e483e2e3 from the live domain config. {{(pid=72351) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2513}}
Aug 09 02:19:39 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-5dd81833-196e-438b-a671-f7ae54f5f711 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 02:19:39 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:19:39 user nova-compute[72351]:   <source dev="/dev/sdc"/>
Aug 09 02:19:39 user nova-compute[72351]:   <target dev="sdc" bus="scsi"/>
Aug 09 02:19:39 user nova-compute[72351]:   <serial>643c3e80-8b8a-40c2-a979-579d65ba653a</serial>
Aug 09 02:19:39 user nova-compute[72351]:   <address type="drive" controller="0" bus="0" target="0" unit="2"/>
Aug 09 02:19:39 user nova-compute[72351]: </disk>
Aug 09 02:19:39 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 02:19:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:39 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Received event <DeviceRemovedEvent: 1691547579.673336, 50302d76-f6b9-4fe3-b55a-4a01e483e2e3 => scsi0-0-0-2> from libvirt while the driver is waiting for it; dispatched. {{(pid=72351) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2360}}
Aug 09 02:19:39 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-5dd81833-196e-438b-a671-f7ae54f5f711 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Start waiting for the detach event from libvirt for device sdc with device alias scsi0-0-0-2 for instance 50302d76-f6b9-4fe3-b55a-4a01e483e2e3 {{(pid=72351) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2589}}
Aug 09 02:19:39 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-5dd81833-196e-438b-a671-f7ae54f5f711 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Successfully detached device sdc from instance 50302d76-f6b9-4fe3-b55a-4a01e483e2e3 from the live domain config.
Aug 09 02:19:39 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-5dd81833-196e-438b-a671-f7ae54f5f711 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 02:19:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5dd81833-196e-438b-a671-f7ae54f5f711 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:643c3e80-8b8a-40c2-a979-579d65ba653a'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'tzELmHlW', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc'}, None), 'kwargs': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:19:39 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-5dd81833-196e-438b-a671-f7ae54f5f711 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:19:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-5dd81833-196e-438b-a671-f7ae54f5f711 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:19:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-5dd81833-196e-438b-a671-f7ae54f5f711 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:19:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5dd81833-196e-438b-a671-f7ae54f5f711 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:643c3e80-8b8a-40c2-a979-579d65ba653a', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 02:19:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:39 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[68aa548f-20ff-482b-b3f5-3cdfec640ac0]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:643c3e80-8b8a-40c2-a979-579d65ba653a\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.026s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:39 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d663aafd-cf56-4d55-92e9-858024907a18]: (4, ('tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:643c3e80-8b8a-40c2-a979-579d65ba653a (non-flash)\ntcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)\ntcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5dd81833-196e-438b-a671-f7ae54f5f711 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:643c3e80-8b8a-40c2-a979-579d65ba653a (non-flash)
Aug 09 02:19:39 user nova-compute[72351]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:19:39 user nova-compute[72351]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)
Aug 09 02:19:39 user nova-compute[72351]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:19:39 user nova-compute[72351]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)
Aug 09 02:19:39 user nova-compute[72351]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c (non-flash)
Aug 09 02:19:39 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:19:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5dd81833-196e-438b-a671-f7ae54f5f711 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] iscsi session list stdout=tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:643c3e80-8b8a-40c2-a979-579d65ba653a (non-flash)
Aug 09 02:19:39 user nova-compute[72351]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:19:39 user nova-compute[72351]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)
Aug 09 02:19:39 user nova-compute[72351]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:19:39 user nova-compute[72351]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)
Aug 09 02:19:39 user nova-compute[72351]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c (non-flash)
Aug 09 02:19:39 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:19:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5dd81833-196e-438b-a671-f7ae54f5f711 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fcfdc4ccee0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:643c3e80-8b8a-40c2-a979-579d65ba653a'): ({'sdc'}, set())}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 02:19:39 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-5dd81833-196e-438b-a671-f7ae54f5f711 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Removing single pathed devices sdc {{(pid=72351) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Aug 09 02:19:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:39 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[7fd2f920-6166-4a02-9cac-8103af517395]: (4, ('path checker states:\nup                  6\n\npaths: 0\nbusy: False\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdc {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdc" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:39 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[566123ee-e7a2-4dff-bc14-880a1cb1a35b]: (4, ('ok\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:39 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-5dd81833-196e-438b-a671-f7ae54f5f711 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Flushing IO for device /dev/sdc {{(pid=72351) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Aug 09 02:19:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdc {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdc" returned: 0 in 0.015s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:39 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[9a783099-9651-4cd7-9011-0689e57ffe7b]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:39 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-5dd81833-196e-438b-a671-f7ae54f5f711 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Remove SCSI device /dev/sdc with /sys/block/sdc/device/delete {{(pid=72351) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Aug 09 02:19:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdc/device/delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdc/device/delete" returned: 0 in 0.040s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:39 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[96550177-d358-44c2-9c8a-e0e9c78032b0]: (4, ('1', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:39 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-5dd81833-196e-438b-a671-f7ae54f5f711 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Checking to see if SCSI volumes sdc have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Aug 09 02:19:39 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-5dd81833-196e-438b-a671-f7ae54f5f711 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] SCSI volumes sdc have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Aug 09 02:19:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5dd81833-196e-438b-a671-f7ae54f5f711 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:643c3e80-8b8a-40c2-a979-579d65ba653a')] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 02:19:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:643c3e80-8b8a-40c2-a979-579d65ba653a -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:643c3e80-8b8a-40c2-a979-579d65ba653a -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:39 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d8c72635-8bfe-49bf-b353-199050e6afc5]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5dd81833-196e-438b-a671-f7ae54f5f711 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:643c3e80-8b8a-40c2-a979-579d65ba653a -p 172.16.0.220:3260 --logout {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:643c3e80-8b8a-40c2-a979-579d65ba653a -p 172.16.0.220:3260 --logout" returned: 0 in 0.058s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:39 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[751a7f9d-9f29-42b1-9c78-d12822f78f11]: (4, ('Logging out of session [sid: 10, target: iqn.2010-10.org.openstack:643c3e80-8b8a-40c2-a979-579d65ba653a, portal: 172.16.0.220,3260]\nLogout of [sid: 10, target: iqn.2010-10.org.openstack:643c3e80-8b8a-40c2-a979-579d65ba653a, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5dd81833-196e-438b-a671-f7ae54f5f711 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 10, target: iqn.2010-10.org.openstack:643c3e80-8b8a-40c2-a979-579d65ba653a, portal: 172.16.0.220,3260]
Aug 09 02:19:39 user nova-compute[72351]: Logout of [sid: 10, target: iqn.2010-10.org.openstack:643c3e80-8b8a-40c2-a979-579d65ba653a, portal: 172.16.0.220,3260] successful.
Aug 09 02:19:39 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:643c3e80-8b8a-40c2-a979-579d65ba653a -p 172.16.0.220:3260 --op delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:643c3e80-8b8a-40c2-a979-579d65ba653a -p 172.16.0.220:3260 --op delete" returned: 0 in 0.040s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:39 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[164e393e-ecbf-4e49-b575-4c8002946f82]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5dd81833-196e-438b-a671-f7ae54f5f711 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-5dd81833-196e-438b-a671-f7ae54f5f711 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.306s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:19:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5dd81833-196e-438b-a671-f7ae54f5f711 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] <== disconnect_volume: return (308ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:19:39 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-5dd81833-196e-438b-a671-f7ae54f5f711 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 02:19:40 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:41 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:42 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:45 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:45 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:45 user nova-compute[72351]: DEBUG nova.compute.manager [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] swap_volume: Calling driver volume swap with connection infos: new: {'driver_volume_type': 'iscsi', 'data': {'target_iqns': ['iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'CgQaLjJu', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b', 'attached_at': '', 'detached_at': '', 'volume_id': 'edeae21b-4fa4-464b-b7de-da06cf9e8f58', 'serial': 'edeae21b-4fa4-464b-b7de-da06cf9e8f58'}; old: {'driver_volume_type': 'iscsi', 'data': {'target_iqns': ['iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'sZqwozDV', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdg'}, 'status': 'reserved', 'instance': '11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b', 'attached_at': '', 'detached_at': '', 'volume_id': '0203626e-04e6-4768-b2c2-e7dbaeed4c6c', 'serial'} {{(pid=72351) _swap_volume /opt/stack/nova/nova/compute/manager.py:7702}}
Aug 09 02:19:45 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 02:19:45 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'CgQaLjJu', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:19:45 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:19:45 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:19:45 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:19:45 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:19:45 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:19:45 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:19:45 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 02:19:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58 -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58 -p 172.16.0.220:3260" returned: 21 in 0.027s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:45 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[1168d5b1-91da-4ebe-a797-197efe80a4b2]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:45 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] iscsiadm (): stdout= stderr=iscsiadm: No records found
Aug 09 02:19:45 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58 -p 172.16.0.220:3260 --interface default --op new {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:46 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[5b8eae01-2f67-4d59-a950-6c3399c794fa]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58] added\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58] added
Aug 09 02:19:46 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:46 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[23d01768-d342-4aa3-9908-18e0ca0ca951]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:46 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[54e3b18c-24e0-4cbd-93e5-423d850c0989]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v CgQaLjJu {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v CgQaLjJu" returned: 0 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:46 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[63150296-a21c-40e7-a022-6573036fa784]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'CgQaLjJu'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:46 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[f10b1c29-e550-4951-adce-68a8997d9516]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.028s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:46 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[360b4129-3658-49c3-8282-df0f2d022e92]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)\ntcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:19:46 user nova-compute[72351]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)
Aug 09 02:19:46 user nova-compute[72351]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:19:46 user nova-compute[72351]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)
Aug 09 02:19:46 user nova-compute[72351]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c (non-flash)
Aug 09 02:19:46 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:19:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:19:46 user nova-compute[72351]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)
Aug 09 02:19:46 user nova-compute[72351]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:19:46 user nova-compute[72351]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)
Aug 09 02:19:46 user nova-compute[72351]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c (non-flash)
Aug 09 02:19:46 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:19:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58 -p 172.16.0.220:3260 --login {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58 -p 172.16.0.220:3260 --login" returned: 0 in 0.080s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:46 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[4df0282b-c83a-400c-89b8-d3c5d1b088d6]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58, portal: 172.16.0.220,3260]
Aug 09 02:19:46 user nova-compute[72351]: Login to [iface: default, target: iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58, portal: 172.16.0.220,3260] successful.
Aug 09 02:19:46 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:46 user nova-compute[72351]: DEBUG nova.objects.instance [None req-5dd81833-196e-438b-a671-f7ae54f5f711 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Lazy-loading 'flavor' on Instance uuid 50302d76-f6b9-4fe3-b55a-4a01e483e2e3 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:19:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.030s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:46 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[54fd9ec2-efd3-4e9c-903d-8daa818e8593]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.027s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:46 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[feebe100-6ae3-41ca-a874-3cfe383201b7]: (4, ('tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58 (non-flash)\ntcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)\ntcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] iscsiadm ('-m', 'session'): stdout=tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58 (non-flash)
Aug 09 02:19:46 user nova-compute[72351]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:19:46 user nova-compute[72351]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)
Aug 09 02:19:46 user nova-compute[72351]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:19:46 user nova-compute[72351]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)
Aug 09 02:19:46 user nova-compute[72351]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c (non-flash)
Aug 09 02:19:46 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:19:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] iscsi session list stdout=tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58 (non-flash)
Aug 09 02:19:46 user nova-compute[72351]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:19:46 user nova-compute[72351]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)
Aug 09 02:19:46 user nova-compute[72351]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:19:46 user nova-compute[72351]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)
Aug 09 02:19:46 user nova-compute[72351]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c (non-flash)
Aug 09 02:19:46 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:19:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.336s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:19:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 02:19:46 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] HCTL ('34', '-', '-', 0) found on session 11 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 02:19:46 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] Scanning host 34 c: -, t: -, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 02:19:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5dd81833-196e-438b-a671-f7ae54f5f711 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Lock "50302d76-f6b9-4fe3-b55a-4a01e483e2e3" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 7.122s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:46 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[29c6c6e7-198d-4b7b-9f7f-37d48b363f01]: (4, ('- - 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:46 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] Searching for a device in session 11 and hctl ['34', '*', '*', 0] yield: sdc {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:19:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] Connected to sdc using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'CgQaLjJu', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 02:19:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6a75a4a1-8f25-490b-b386-322ff3f58d51 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Acquiring lock "50302d76-f6b9-4fe3-b55a-4a01e483e2e3" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6a75a4a1-8f25-490b-b386-322ff3f58d51 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Lock "50302d76-f6b9-4fe3-b55a-4a01e483e2e3" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6a75a4a1-8f25-490b-b386-322ff3f58d51 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Acquiring lock "50302d76-f6b9-4fe3-b55a-4a01e483e2e3-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6a75a4a1-8f25-490b-b386-322ff3f58d51 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Lock "50302d76-f6b9-4fe3-b55a-4a01e483e2e3-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6a75a4a1-8f25-490b-b386-322ff3f58d51 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Lock "50302d76-f6b9-4fe3-b55a-4a01e483e2e3-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:47 user nova-compute[72351]: INFO nova.compute.manager [None req-6a75a4a1-8f25-490b-b386-322ff3f58d51 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Terminating instance
Aug 09 02:19:47 user nova-compute[72351]: DEBUG nova.compute.manager [None req-6a75a4a1-8f25-490b-b386-322ff3f58d51 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Start destroying the instance on the hypervisor. {{(pid=72351) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3124}}
Aug 09 02:19:47 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:47 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:47 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:47 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:47 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.382s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:19:47 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] <== connect_volume: return (1384ms) {'type': 'block', 'scsi_wwn': '26332383138336665', 'path': '/dev/sdc'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:19:47 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26332383138336665', 'path': '/dev/sdc'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 02:19:47 user nova-compute[72351]: DEBUG nova.objects.instance [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] Lazy-loading 'flavor' on Instance uuid 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:19:47 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:47 user nova-compute[72351]: DEBUG nova.compute.manager [req-31f602b8-154f-409d-ba73-2de0f45ed832 req-7d4651cf-71c7-430d-80c6-3da4d103ce53 service nova] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Received event network-vif-unplugged-e5ba6e8d-49a5-4761-ab57-21cee6b0fbdd {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:19:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-31f602b8-154f-409d-ba73-2de0f45ed832 req-7d4651cf-71c7-430d-80c6-3da4d103ce53 service nova] Acquiring lock "50302d76-f6b9-4fe3-b55a-4a01e483e2e3-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-31f602b8-154f-409d-ba73-2de0f45ed832 req-7d4651cf-71c7-430d-80c6-3da4d103ce53 service nova] Lock "50302d76-f6b9-4fe3-b55a-4a01e483e2e3-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-31f602b8-154f-409d-ba73-2de0f45ed832 req-7d4651cf-71c7-430d-80c6-3da4d103ce53 service nova] Lock "50302d76-f6b9-4fe3-b55a-4a01e483e2e3-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:47 user nova-compute[72351]: DEBUG nova.compute.manager [req-31f602b8-154f-409d-ba73-2de0f45ed832 req-7d4651cf-71c7-430d-80c6-3da4d103ce53 service nova] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] No waiting events found dispatching network-vif-unplugged-e5ba6e8d-49a5-4761-ab57-21cee6b0fbdd {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:19:47 user nova-compute[72351]: DEBUG nova.compute.manager [req-31f602b8-154f-409d-ba73-2de0f45ed832 req-7d4651cf-71c7-430d-80c6-3da4d103ce53 service nova] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Received event network-vif-unplugged-e5ba6e8d-49a5-4761-ab57-21cee6b0fbdd for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:19:47 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] COPY block job progress, current cursor: 0 final cursor: 1073741824 {{(pid=72351) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Aug 09 02:19:48 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Instance destroyed successfully.
Aug 09 02:19:48 user nova-compute[72351]: DEBUG nova.objects.instance [None req-6a75a4a1-8f25-490b-b386-322ff3f58d51 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Lazy-loading 'resources' on Instance uuid 50302d76-f6b9-4fe3-b55a-4a01e483e2e3 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:19:48 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-6a75a4a1-8f25-490b-b386-322ff3f58d51 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-08-09T02:17:18Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachSCSIVolumeTestJSON-server-1904871607',display_name='tempest-AttachSCSIVolumeTestJSON-server-1904871607',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachscsivolumetestjson-server-1904871607',id=3,image_ref='62f1da90-2927-452b-831f-d34d21ab862a',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJGHMVLR+W7uX3EHxXslcn5rNtsVUdS8noPRzLhTwfCXUymvnqG/yjZ6GlWujXjOHQqzGpQI8gBHB2Bq+KvBrqlYY9bJOEmcP7qmKR9OdnSd6Ve+rANep1lfbliuNjFHjg==',key_name='tempest-keypair-259400434',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:17:36Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='3f04dea5c1d84aaf94feb47b8ea84f0e',ramdisk_id='',reservation_id='r-tbsm9tz2',resources=None,root_device_name='/dev/sda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='62f1da90-2927-452b-831f-d34d21ab862a',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='scsi',image_hw_disk_bus='scsi',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_scsi_model='virtio-scsi',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',owner_project_name='tempest-AttachSCSIVolumeTestJSON-1890923628',owner_user_name='tempest-AttachSCSIVolumeTestJSON-1890923628-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:17:36Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='657c221a2b1840ad8e75165d89d779e8',uuid=50302d76-f6b9-4fe3-b55a-4a01e483e2e3,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "e5ba6e8d-49a5-4761-ab57-21cee6b0fbdd", "address": "fa:16:3e:6e:b8:a5", "network": {"id": "1acf710e-e626-43a0-a32f-389a87f9858e", "bridge": "br-int", "label": "tempest-AttachSCSIVolumeTestJSON-1052335889-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.121", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3f04dea5c1d84aaf94feb47b8ea84f0e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape5ba6e8d-49", "ovs_interfaceid": "e5ba6e8d-49a5-4761-ab57-21cee6b0fbdd", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:19:48 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-6a75a4a1-8f25-490b-b386-322ff3f58d51 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Converting VIF {"id": "e5ba6e8d-49a5-4761-ab57-21cee6b0fbdd", "address": "fa:16:3e:6e:b8:a5", "network": {"id": "1acf710e-e626-43a0-a32f-389a87f9858e", "bridge": "br-int", "label": "tempest-AttachSCSIVolumeTestJSON-1052335889-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.121", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3f04dea5c1d84aaf94feb47b8ea84f0e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape5ba6e8d-49", "ovs_interfaceid": "e5ba6e8d-49a5-4761-ab57-21cee6b0fbdd", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:19:48 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-6a75a4a1-8f25-490b-b386-322ff3f58d51 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:6e:b8:a5,bridge_name='br-int',has_traffic_filtering=True,id=e5ba6e8d-49a5-4761-ab57-21cee6b0fbdd,network=Network(1acf710e-e626-43a0-a32f-389a87f9858e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape5ba6e8d-49') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:19:48 user nova-compute[72351]: DEBUG os_vif [None req-6a75a4a1-8f25-490b-b386-322ff3f58d51 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:6e:b8:a5,bridge_name='br-int',has_traffic_filtering=True,id=e5ba6e8d-49a5-4761-ab57-21cee6b0fbdd,network=Network(1acf710e-e626-43a0-a32f-389a87f9858e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape5ba6e8d-49') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:19:48 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:48 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tape5ba6e8d-49, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:19:48 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:48 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:19:48 user nova-compute[72351]: INFO os_vif [None req-6a75a4a1-8f25-490b-b386-322ff3f58d51 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:6e:b8:a5,bridge_name='br-int',has_traffic_filtering=True,id=e5ba6e8d-49a5-4761-ab57-21cee6b0fbdd,network=Network(1acf710e-e626-43a0-a32f-389a87f9858e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape5ba6e8d-49')
Aug 09 02:19:48 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-6a75a4a1-8f25-490b-b386-322ff3f58d51 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Deleting instance files /opt/stack/data/nova/instances/50302d76-f6b9-4fe3-b55a-4a01e483e2e3_del
Aug 09 02:19:48 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-6a75a4a1-8f25-490b-b386-322ff3f58d51 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Deletion of /opt/stack/data/nova/instances/50302d76-f6b9-4fe3-b55a-4a01e483e2e3_del complete
Aug 09 02:19:48 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] COPY block job progress, current cursor: 165675008 final cursor: 1073741824 {{(pid=72351) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Aug 09 02:19:48 user nova-compute[72351]: INFO nova.compute.manager [None req-6a75a4a1-8f25-490b-b386-322ff3f58d51 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Took 1.43 seconds to destroy the instance on the hypervisor.
Aug 09 02:19:48 user nova-compute[72351]: DEBUG oslo.service.loopingcall [None req-6a75a4a1-8f25-490b-b386-322ff3f58d51 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72351) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Aug 09 02:19:48 user nova-compute[72351]: DEBUG nova.compute.manager [-] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Deallocating network for instance {{(pid=72351) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2263}}
Aug 09 02:19:48 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] deallocate_for_instance() {{(pid=72351) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1795}}
Aug 09 02:19:48 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] COPY block job progress, current cursor: 255852544 final cursor: 1073741824 {{(pid=72351) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Aug 09 02:19:49 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] COPY block job progress, current cursor: 419430400 final cursor: 1073741824 {{(pid=72351) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Aug 09 02:19:49 user nova-compute[72351]: DEBUG nova.compute.manager [req-f948fa00-b3a2-4100-8a68-648a67d634e5 req-2c8175d3-3352-446e-8bcb-1e8d37819983 service nova] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Received event network-vif-plugged-e5ba6e8d-49a5-4761-ab57-21cee6b0fbdd {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:19:49 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-f948fa00-b3a2-4100-8a68-648a67d634e5 req-2c8175d3-3352-446e-8bcb-1e8d37819983 service nova] Acquiring lock "50302d76-f6b9-4fe3-b55a-4a01e483e2e3-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:49 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-f948fa00-b3a2-4100-8a68-648a67d634e5 req-2c8175d3-3352-446e-8bcb-1e8d37819983 service nova] Lock "50302d76-f6b9-4fe3-b55a-4a01e483e2e3-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:49 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-f948fa00-b3a2-4100-8a68-648a67d634e5 req-2c8175d3-3352-446e-8bcb-1e8d37819983 service nova] Lock "50302d76-f6b9-4fe3-b55a-4a01e483e2e3-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:49 user nova-compute[72351]: DEBUG nova.compute.manager [req-f948fa00-b3a2-4100-8a68-648a67d634e5 req-2c8175d3-3352-446e-8bcb-1e8d37819983 service nova] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] No waiting events found dispatching network-vif-plugged-e5ba6e8d-49a5-4761-ab57-21cee6b0fbdd {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:19:49 user nova-compute[72351]: WARNING nova.compute.manager [req-f948fa00-b3a2-4100-8a68-648a67d634e5 req-2c8175d3-3352-446e-8bcb-1e8d37819983 service nova] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Received unexpected event network-vif-plugged-e5ba6e8d-49a5-4761-ab57-21cee6b0fbdd for instance with vm_state active and task_state deleting.
Aug 09 02:19:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:49 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] COPY block job progress, current cursor: 583008256 final cursor: 1073741824 {{(pid=72351) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Aug 09 02:19:50 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:19:50 user nova-compute[72351]: DEBUG nova.compute.manager [req-edfa8b06-bfa0-4ae8-abd9-3bfb0cecbee2 req-a22efc53-16de-4a3f-b410-961a7f30c592 service nova] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Received event network-vif-deleted-e5ba6e8d-49a5-4761-ab57-21cee6b0fbdd {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:19:50 user nova-compute[72351]: INFO nova.compute.manager [req-edfa8b06-bfa0-4ae8-abd9-3bfb0cecbee2 req-a22efc53-16de-4a3f-b410-961a7f30c592 service nova] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Neutron deleted interface e5ba6e8d-49a5-4761-ab57-21cee6b0fbdd; detaching it from the instance and deleting it from the info cache
Aug 09 02:19:50 user nova-compute[72351]: DEBUG nova.network.neutron [req-edfa8b06-bfa0-4ae8-abd9-3bfb0cecbee2 req-a22efc53-16de-4a3f-b410-961a7f30c592 service nova] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:19:50 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Took 1.77 seconds to deallocate network for instance.
Aug 09 02:19:50 user nova-compute[72351]: DEBUG nova.compute.manager [req-edfa8b06-bfa0-4ae8-abd9-3bfb0cecbee2 req-a22efc53-16de-4a3f-b410-961a7f30c592 service nova] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Detach interface failed, port_id=e5ba6e8d-49a5-4761-ab57-21cee6b0fbdd, reason: Instance 50302d76-f6b9-4fe3-b55a-4a01e483e2e3 could not be found. {{(pid=72351) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10866}}
Aug 09 02:19:50 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] COPY block job progress, current cursor: 717225984 final cursor: 1073741824 {{(pid=72351) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Aug 09 02:19:50 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6a75a4a1-8f25-490b-b386-322ff3f58d51 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:50 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6a75a4a1-8f25-490b-b386-322ff3f58d51 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:50 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-6a75a4a1-8f25-490b-b386-322ff3f58d51 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:19:50 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-6a75a4a1-8f25-490b-b386-322ff3f58d51 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:19:50 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6a75a4a1-8f25-490b-b386-322ff3f58d51 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.398s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:50 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691547575.8000455, d0c7002c-8a3a-4bc5-940c-63cf4264668b => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:19:50 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] VM Stopped (Lifecycle Event)
Aug 09 02:19:50 user nova-compute[72351]: INFO nova.scheduler.client.report [None req-6a75a4a1-8f25-490b-b386-322ff3f58d51 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Deleted allocations for instance 50302d76-f6b9-4fe3-b55a-4a01e483e2e3
Aug 09 02:19:50 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] COPY block job progress, current cursor: 876609536 final cursor: 1073741824 {{(pid=72351) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Aug 09 02:19:50 user nova-compute[72351]: DEBUG nova.compute.manager [None req-e081023e-ac2a-4e54-ba72-9a58ecf440a7 None None] [instance: d0c7002c-8a3a-4bc5-940c-63cf4264668b] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:19:50 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:19:50 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6a75a4a1-8f25-490b-b386-322ff3f58d51 tempest-AttachSCSIVolumeTestJSON-1890923628 tempest-AttachSCSIVolumeTestJSON-1890923628-project-member] Lock "50302d76-f6b9-4fe3-b55a-4a01e483e2e3" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 3.922s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:51 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] COPY block job progress, current cursor: 1032847360 final cursor: 1073741824 {{(pid=72351) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Aug 09 02:19:51 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Acquiring lock "52f0d664-7767-4953-bc7a-496b26f22715" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:51 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Lock "52f0d664-7767-4953-bc7a-496b26f22715" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:51 user nova-compute[72351]: DEBUG nova.compute.manager [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] Starting instance... {{(pid=72351) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2406}}
Aug 09 02:19:51 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:51 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:51 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72351) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Aug 09 02:19:51 user nova-compute[72351]: INFO nova.compute.claims [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] Claim successful on node user
Aug 09 02:19:51 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] COPY block job progress, current cursor: 1073741824 final cursor: 1073741824 {{(pid=72351) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Aug 09 02:19:51 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:19:51 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72351) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10461}}
Aug 09 02:19:51 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 02:19:51 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'sZqwozDV', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdg'}, None), 'kwargs': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:19:51 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:19:51 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:19:51 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:19:51 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 02:19:51 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:51 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:51 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[70e0cd0f-aca3-4c09-90fe-2d6a54023c82]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:51 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.028s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[909df148-c0b0-47c5-9f14-4dfd08b8e442]: (4, ('tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58 (non-flash)\ntcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)\ntcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] iscsiadm ('-m', 'session'): stdout=tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58 (non-flash)
Aug 09 02:19:52 user nova-compute[72351]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:19:52 user nova-compute[72351]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)
Aug 09 02:19:52 user nova-compute[72351]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:19:52 user nova-compute[72351]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)
Aug 09 02:19:52 user nova-compute[72351]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c (non-flash)
Aug 09 02:19:52 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] iscsi session list stdout=tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58 (non-flash)
Aug 09 02:19:52 user nova-compute[72351]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:19:52 user nova-compute[72351]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)
Aug 09 02:19:52 user nova-compute[72351]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:19:52 user nova-compute[72351]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)
Aug 09 02:19:52 user nova-compute[72351]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c (non-flash)
Aug 09 02:19:52 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fd00017b490>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c'): ({'sdg'}, set())}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] Removing single pathed devices sdg {{(pid=72351) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[f0fd2063-e370-4d86-8a39-717a3703d746]: (4, ('path checker states:\nup                  6\n\npaths: 0\nbusy: False\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdg {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdg" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[47cd684a-882e-40d5-99bb-b3cfb464bda0]: (4, ('ok\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] Flushing IO for device /dev/sdg {{(pid=72351) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdg {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdg" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ea471a12-3899-4901-9ba1-5592842ab2be]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] Remove SCSI device /dev/sdg with /sys/block/sdg/device/delete {{(pid=72351) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdg/device/delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdg/device/delete" returned: 0 in 0.045s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[5f2fb37c-3225-4a79-a5e2-9bbf13b51e25]: (4, ('1', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] Checking to see if SCSI volumes sdg have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] SCSI volumes sdg have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c')] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[3acbea80-6f8f-4ba1-a809-71611d04767d]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c -p 172.16.0.220:3260 --logout {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.376s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG nova.compute.manager [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] Start building networks asynchronously for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2803}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c -p 172.16.0.220:3260 --logout" returned: 0 in 0.065s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[cb517a9e-59e2-4f05-915d-fc5bfa4df73a]: (4, ('Logging out of session [sid: 8, target: iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c, portal: 172.16.0.220,3260]\nLogout of [sid: 8, target: iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] iscsiadm ('--logout',): stdout=Logging out of session [sid: 8, target: iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c, portal: 172.16.0.220,3260]
Aug 09 02:19:52 user nova-compute[72351]: Logout of [sid: 8, target: iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c, portal: 172.16.0.220,3260] successful.
Aug 09 02:19:52 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c -p 172.16.0.220:3260 --op delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG nova.compute.manager [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] Allocating IP information in the background. {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1956}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG nova.network.neutron [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] allocate_for_instance() {{(pid=72351) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c -p 172.16.0.220:3260 --op delete" returned: 0 in 0.029s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[5d515300-6273-406a-a072-98ac5f94ca95]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.317s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] <== disconnect_volume: return (317ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 02:19:52 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Aug 09 02:19:52 user nova-compute[72351]: DEBUG nova.compute.manager [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] Start building block device mappings for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2838}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG nova.compute.manager [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] swap_volume: Driver volume swap returned, new connection_info is now : {'driver_volume_type': 'iscsi', 'data': {'target_iqns': ['iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'CgQaLjJu', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc'}, 'status': 'reserved', 'instance': '11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b', 'attached_at': '', 'detached_at': '', 'volume_id': 'edeae21b-4fa4-464b-b7de-da06cf9e8f58', 'serial'} {{(pid=72351) _swap_volume /opt/stack/nova/nova/compute/manager.py:7711}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG nova.compute.manager [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] swap_volume: removing Cinder connection for volume 0203626e-04e6-4768-b2c2-e7dbaeed4c6c {{(pid=72351) _swap_volume /opt/stack/nova/nova/compute/manager.py:7756}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG nova.policy [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'ae566c7921844ee0b146d36259fbaf36', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd6461f5cffd540b2b7f6797390561a80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72351) authorize /opt/stack/nova/nova/policy.py:203}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG nova.compute.manager [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] Start spawning the instance on the hypervisor. {{(pid=72351) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2612}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] Creating instance directory {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4717}}
Aug 09 02:19:52 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] Creating image(s)
Aug 09 02:19:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Acquiring lock "/opt/stack/data/nova/instances/52f0d664-7767-4953-bc7a-496b26f22715/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Lock "/opt/stack/data/nova/instances/52f0d664-7767-4953-bc7a-496b26f22715/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Lock "/opt/stack/data/nova/instances/52f0d664-7767-4953-bc7a-496b26f22715/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.151s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Acquiring lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.138s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/52f0d664-7767-4953-bc7a-496b26f22715/disk 1073741824 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/52f0d664-7767-4953-bc7a-496b26f22715/disk 1073741824" returned: 0 in 0.047s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.193s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.145s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Checking if we can resize image /opt/stack/data/nova/instances/52f0d664-7767-4953-bc7a-496b26f22715/disk. size=1073741824 {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/52f0d664-7767-4953-bc7a-496b26f22715/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:52 user nova-compute[72351]: DEBUG nova.network.neutron [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] Successfully created port: 84074c82-d423-45d8-9665-bd92d13b6380 {{(pid=72351) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Aug 09 02:19:53 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:53 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:53 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:53 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Auditing locally available compute resources for user (node: user) {{(pid=72351) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Aug 09 02:19:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/52f0d664-7767-4953-bc7a-496b26f22715/disk --force-share --output=json" returned: 0 in 0.200s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:53 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Cannot resize image /opt/stack/data/nova/instances/52f0d664-7767-4953-bc7a-496b26f22715/disk to a smaller size. {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Aug 09 02:19:53 user nova-compute[72351]: DEBUG nova.objects.instance [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Lazy-loading 'migration_context' on Instance uuid 52f0d664-7767-4953-bc7a-496b26f22715 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:19:53 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] Created local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4851}}
Aug 09 02:19:53 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] Ensure instance console log exists: /opt/stack/data/nova/instances/52f0d664-7767-4953-bc7a-496b26f22715/console.log {{(pid=72351) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4603}}
Aug 09 02:19:53 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:53 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:53 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7c5e633d-a32f-4c5e-bb9d-1e40da964bd0/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7c5e633d-a32f-4c5e-bb9d-1e40da964bd0/disk --force-share --output=json" returned: 0 in 0.154s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7c5e633d-a32f-4c5e-bb9d-1e40da964bd0/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:53 user nova-compute[72351]: DEBUG nova.compute.manager [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] swap_volume: Updating volume 0203626e-04e6-4768-b2c2-e7dbaeed4c6c BDM record with {'connection_info': '{"driver_volume_type": "iscsi", "data": {"target_iqns": ["iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58"], "target_portals": ["172.16.0.220:3260"], "target_luns": [0], "target_discovered": false, "auth_method": "CHAP", "auth_username": "CgQaLjJu", "auth_password": "trlpmTRQExo40i", "target_lun": 0, "qos_specs": null, "access_mode": "rw", "encrypted": false, "cacheable": false, "device_path": "/dev/sdc", "volume_id": "edeae21b-4fa4-464b-b7de-da06cf9e8f58"}, "status": "reserved", "instance": "11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b", "attached_at": "", "detached_at": "", "volume_id": "edeae21b-4fa4-464b-b7de-da06cf9e8f58", "serial": "edeae21b-4fa4-464b-b7de-da06cf9e8f58"}', 'source_type': 'volume', 'destination_type': 'volume', 'snapshot_id': None, 'volume_id': 'edeae21b-4fa4-464b-b7de-da06cf9e8f58', 'no_device': None, 'attachment_id': '7e577d97-2e71-4f67-9c4b-e51bbecbbc89'} {{(pid=72351) _do_swap_volume /opt/stack/nova/nova/compute/manager.py:7904}}
Aug 09 02:19:53 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c4c84910-2e08-48ec-8f33-2b37baf1d833 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] Lock "11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b" "released" by "nova.compute.manager.ComputeManager.swap_volume.<locals>._do_locked_swap_volume" :: held 22.650s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7c5e633d-a32f-4c5e-bb9d-1e40da964bd0/disk --force-share --output=json" returned: 0 in 0.155s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b/disk --force-share --output=json" returned: 0 in 0.133s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:53 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdc (vdb) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:19:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/822e25c4-8dd1-4e21-b0ed-f340d59c6c98/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/822e25c4-8dd1-4e21-b0ed-f340d59c6c98/disk --force-share --output=json" returned: 0 in 0.128s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/822e25c4-8dd1-4e21-b0ed-f340d59c6c98/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG nova.network.neutron [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] Successfully updated port: 84074c82-d423-45d8-9665-bd92d13b6380 {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Acquiring lock "refresh_cache-52f0d664-7767-4953-bc7a-496b26f22715" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Acquired lock "refresh_cache-52f0d664-7767-4953-bc7a-496b26f22715" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG nova.network.neutron [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG nova.compute.manager [req-77f69d7d-3834-43f1-9395-1c259112b930 req-ea3aa5bf-8cb6-40a2-87b4-a2d2b4949569 service nova] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] Received event network-changed-84074c82-d423-45d8-9665-bd92d13b6380 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG nova.compute.manager [req-77f69d7d-3834-43f1-9395-1c259112b930 req-ea3aa5bf-8cb6-40a2-87b4-a2d2b4949569 service nova] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] Refreshing instance network info cache due to event network-changed-84074c82-d423-45d8-9665-bd92d13b6380. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-77f69d7d-3834-43f1-9395-1c259112b930 req-ea3aa5bf-8cb6-40a2-87b4-a2d2b4949569 service nova] Acquiring lock "refresh_cache-52f0d664-7767-4953-bc7a-496b26f22715" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/822e25c4-8dd1-4e21-b0ed-f340d59c6c98/disk --force-share --output=json" returned: 0 in 0.158s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG nova.network.neutron [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2/disk --force-share --output=json" returned: 0 in 0.160s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2/disk --force-share --output=json" returned: 0 in 0.145s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdd (vdb) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdf (vdc) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG nova.network.neutron [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] Updating instance_info_cache with network_info: [{"id": "84074c82-d423-45d8-9665-bd92d13b6380", "address": "fa:16:3e:63:b2:2a", "network": {"id": "885b8ede-6326-4f34-aaaf-159df1b5e4e5", "bridge": "br-int", "label": "tempest-VolumesActionsTest-661898154-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d6461f5cffd540b2b7f6797390561a80", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap84074c82-d4", "ovs_interfaceid": "84074c82-d423-45d8-9665-bd92d13b6380", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/74728ca8-34d4-45e6-a1f2-21c0952ff835/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Releasing lock "refresh_cache-52f0d664-7767-4953-bc7a-496b26f22715" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG nova.compute.manager [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] Instance network_info: |[{"id": "84074c82-d423-45d8-9665-bd92d13b6380", "address": "fa:16:3e:63:b2:2a", "network": {"id": "885b8ede-6326-4f34-aaaf-159df1b5e4e5", "bridge": "br-int", "label": "tempest-VolumesActionsTest-661898154-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d6461f5cffd540b2b7f6797390561a80", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap84074c82-d4", "ovs_interfaceid": "84074c82-d423-45d8-9665-bd92d13b6380", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1971}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-77f69d7d-3834-43f1-9395-1c259112b930 req-ea3aa5bf-8cb6-40a2-87b4-a2d2b4949569 service nova] Acquired lock "refresh_cache-52f0d664-7767-4953-bc7a-496b26f22715" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG nova.network.neutron [req-77f69d7d-3834-43f1-9395-1c259112b930 req-ea3aa5bf-8cb6-40a2-87b4-a2d2b4949569 service nova] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] Refreshing network info cache for port 84074c82-d423-45d8-9665-bd92d13b6380 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] Start _get_guest_xml network_info=[{"id": "84074c82-d423-45d8-9665-bd92d13b6380", "address": "fa:16:3e:63:b2:2a", "network": {"id": "885b8ede-6326-4f34-aaaf-159df1b5e4e5", "bridge": "br-int", "label": "tempest-VolumesActionsTest-661898154-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d6461f5cffd540b2b7f6797390561a80", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap84074c82-d4", "ovs_interfaceid": "84074c82-d423-45d8-9665-bd92d13b6380", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'size': 0, 'device_type': 'disk', 'encryption_secret_uuid': None, 'guest_format': None, 'encryption_options': None, 'disk_bus': 'virtio', 'encrypted': False, 'encryption_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'image_id': '18ccda7f-31d3-4978-ab9d-4ed2f74eff94'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] Acquiring lock "11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b" by "nova.compute.manager.ComputeManager.swap_volume.<locals>._do_locked_swap_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] Lock "11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b" acquired by "nova.compute.manager.ComputeManager.swap_volume.<locals>._do_locked_swap_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:54 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:19:54 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:19:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:19:51Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesActionsTest-instance-1745726771',display_name='tempest-VolumesActionsTest-instance-1745726771',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-volumesactionstest-instance-1745726771',id=12,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='d6461f5cffd540b2b7f6797390561a80',ramdisk_id='',reservation_id='r-g9lvwc12',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesActionsTest-1803517492',owner_user_name='tempest-VolumesActionsTest-1803517492-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:19:52Z,user_data=None,user_id='ae566c7921844ee0b146d36259fbaf36',uuid=52f0d664-7767-4953-bc7a-496b26f22715,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "84074c82-d423-45d8-9665-bd92d13b6380", "address": "fa:16:3e:63:b2:2a", "network": {"id": "885b8ede-6326-4f34-aaaf-159df1b5e4e5", "bridge": "br-int", "label": "tempest-VolumesActionsTest-661898154-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d6461f5cffd540b2b7f6797390561a80", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap84074c82-d4", "ovs_interfaceid": "84074c82-d423-45d8-9665-bd92d13b6380", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Converting VIF {"id": "84074c82-d423-45d8-9665-bd92d13b6380", "address": "fa:16:3e:63:b2:2a", "network": {"id": "885b8ede-6326-4f34-aaaf-159df1b5e4e5", "bridge": "br-int", "label": "tempest-VolumesActionsTest-661898154-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d6461f5cffd540b2b7f6797390561a80", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap84074c82-d4", "ovs_interfaceid": "84074c82-d423-45d8-9665-bd92d13b6380", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:63:b2:2a,bridge_name='br-int',has_traffic_filtering=True,id=84074c82-d423-45d8-9665-bd92d13b6380,network=Network(885b8ede-6326-4f34-aaaf-159df1b5e4e5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap84074c82-d4') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG nova.objects.instance [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Lazy-loading 'pci_devices' on Instance uuid 52f0d664-7767-4953-bc7a-496b26f22715 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG os_brick.utils [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:19:54 user nova-compute[72351]:   <uuid>52f0d664-7767-4953-bc7a-496b26f22715</uuid>
Aug 09 02:19:54 user nova-compute[72351]:   <name>instance-0000000c</name>
Aug 09 02:19:54 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:19:54 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:19:54 user nova-compute[72351]:   <metadata>
Aug 09 02:19:54 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:19:54 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:19:54 user nova-compute[72351]:       <nova:name>tempest-VolumesActionsTest-instance-1745726771</nova:name>
Aug 09 02:19:54 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:19:54</nova:creationTime>
Aug 09 02:19:54 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:19:54 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:19:54 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:19:54 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:19:54 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:19:54 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:19:54 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:19:54 user nova-compute[72351]:       <nova:owner>
Aug 09 02:19:54 user nova-compute[72351]:         <nova:user uuid="ae566c7921844ee0b146d36259fbaf36">tempest-VolumesActionsTest-1803517492-project-member</nova:user>
Aug 09 02:19:54 user nova-compute[72351]:         <nova:project uuid="d6461f5cffd540b2b7f6797390561a80">tempest-VolumesActionsTest-1803517492</nova:project>
Aug 09 02:19:54 user nova-compute[72351]:       </nova:owner>
Aug 09 02:19:54 user nova-compute[72351]:       <nova:root type="image" uuid="18ccda7f-31d3-4978-ab9d-4ed2f74eff94"/>
Aug 09 02:19:54 user nova-compute[72351]:       <nova:ports>
Aug 09 02:19:54 user nova-compute[72351]:         <nova:port uuid="84074c82-d423-45d8-9665-bd92d13b6380">
Aug 09 02:19:54 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.7" ipVersion="4"/>
Aug 09 02:19:54 user nova-compute[72351]:         </nova:port>
Aug 09 02:19:54 user nova-compute[72351]:       </nova:ports>
Aug 09 02:19:54 user nova-compute[72351]:     </nova:instance>
Aug 09 02:19:54 user nova-compute[72351]:   </metadata>
Aug 09 02:19:54 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:19:54 user nova-compute[72351]:     <system>
Aug 09 02:19:54 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:19:54 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:19:54 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:19:54 user nova-compute[72351]:       <entry name="serial">52f0d664-7767-4953-bc7a-496b26f22715</entry>
Aug 09 02:19:54 user nova-compute[72351]:       <entry name="uuid">52f0d664-7767-4953-bc7a-496b26f22715</entry>
Aug 09 02:19:54 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:19:54 user nova-compute[72351]:     </system>
Aug 09 02:19:54 user nova-compute[72351]:   </sysinfo>
Aug 09 02:19:54 user nova-compute[72351]:   <os>
Aug 09 02:19:54 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:19:54 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:19:54 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:19:54 user nova-compute[72351]:   </os>
Aug 09 02:19:54 user nova-compute[72351]:   <features>
Aug 09 02:19:54 user nova-compute[72351]:     <acpi/>
Aug 09 02:19:54 user nova-compute[72351]:     <apic/>
Aug 09 02:19:54 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:19:54 user nova-compute[72351]:   </features>
Aug 09 02:19:54 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:19:54 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:19:54 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:19:54 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:19:54 user nova-compute[72351]:   </clock>
Aug 09 02:19:54 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:19:54 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:19:54 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:19:54 user nova-compute[72351]:   </cpu>
Aug 09 02:19:54 user nova-compute[72351]:   <devices>
Aug 09 02:19:54 user nova-compute[72351]:     <disk type="file" device="disk">
Aug 09 02:19:54 user nova-compute[72351]:       <driver name="qemu" type="qcow2" cache="none"/>
Aug 09 02:19:54 user nova-compute[72351]:       <source file="/opt/stack/data/nova/instances/52f0d664-7767-4953-bc7a-496b26f22715/disk"/>
Aug 09 02:19:54 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:19:54 user nova-compute[72351]:     </disk>
Aug 09 02:19:54 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:19:54 user nova-compute[72351]:       <mac address="fa:16:3e:63:b2:2a"/>
Aug 09 02:19:54 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:19:54 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:19:54 user nova-compute[72351]:       <target dev="tap84074c82-d4"/>
Aug 09 02:19:54 user nova-compute[72351]:     </interface>
Aug 09 02:19:54 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:19:54 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/52f0d664-7767-4953-bc7a-496b26f22715/console.log" append="off"/>
Aug 09 02:19:54 user nova-compute[72351]:     </serial>
Aug 09 02:19:54 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:19:54 user nova-compute[72351]:     <video>
Aug 09 02:19:54 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:19:54 user nova-compute[72351]:     </video>
Aug 09 02:19:54 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:19:54 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:19:54 user nova-compute[72351]:     </rng>
Aug 09 02:19:54 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:19:54 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:19:54 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:19:54 user nova-compute[72351]:     </memballoon>
Aug 09 02:19:54 user nova-compute[72351]:   </devices>
Aug 09 02:19:54 user nova-compute[72351]: </domain>
Aug 09 02:19:54 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:19:51Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesActionsTest-instance-1745726771',display_name='tempest-VolumesActionsTest-instance-1745726771',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-volumesactionstest-instance-1745726771',id=12,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='d6461f5cffd540b2b7f6797390561a80',ramdisk_id='',reservation_id='r-g9lvwc12',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesActionsTest-1803517492',owner_user_name='tempest-VolumesActionsTest-1803517492-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:19:52Z,user_data=None,user_id='ae566c7921844ee0b146d36259fbaf36',uuid=52f0d664-7767-4953-bc7a-496b26f22715,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "84074c82-d423-45d8-9665-bd92d13b6380", "address": "fa:16:3e:63:b2:2a", "network": {"id": "885b8ede-6326-4f34-aaaf-159df1b5e4e5", "bridge": "br-int", "label": "tempest-VolumesActionsTest-661898154-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d6461f5cffd540b2b7f6797390561a80", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap84074c82-d4", "ovs_interfaceid": "84074c82-d423-45d8-9665-bd92d13b6380", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Converting VIF {"id": "84074c82-d423-45d8-9665-bd92d13b6380", "address": "fa:16:3e:63:b2:2a", "network": {"id": "885b8ede-6326-4f34-aaaf-159df1b5e4e5", "bridge": "br-int", "label": "tempest-VolumesActionsTest-661898154-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d6461f5cffd540b2b7f6797390561a80", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap84074c82-d4", "ovs_interfaceid": "84074c82-d423-45d8-9665-bd92d13b6380", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:63:b2:2a,bridge_name='br-int',has_traffic_filtering=True,id=84074c82-d423-45d8-9665-bd92d13b6380,network=Network(885b8ede-6326-4f34-aaaf-159df1b5e4e5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap84074c82-d4') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG os_vif [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:63:b2:2a,bridge_name='br-int',has_traffic_filtering=True,id=84074c82-d423-45d8-9665-bd92d13b6380,network=Network(885b8ede-6326-4f34-aaaf-159df1b5e4e5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap84074c82-d4') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.015s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[c126826c-2ea7-45e9-9e87-2ebe87e03723]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:a52269602ff4\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap84074c82-d4, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap84074c82-d4, col_values=(('external_ids', {'iface-id': '84074c82-d423-45d8-9665-bd92d13b6380', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:63:b2:2a', 'vm-uuid': '52f0d664-7767-4953-bc7a-496b26f22715'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/74728ca8-34d4-45e6-a1f2-21c0952ff835/disk --force-share --output=json" returned: 0 in 0.179s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/74728ca8-34d4-45e6-a1f2-21c0952ff835/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a7b846aa-5247-42cf-8d1f-42e432d931c8]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:54 user nova-compute[72351]: INFO os_vif [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:63:b2:2a,bridge_name='br-int',has_traffic_filtering=True,id=84074c82-d423-45d8-9665-bd92d13b6380,network=Network(885b8ede-6326-4f34-aaaf-159df1b5e4e5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap84074c82-d4')
Aug 09 02:19:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[8ac8001b-0e31-45e1-8cea-aa78e33eb53c]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[92baee48-eb6d-4759-894a-bf75489fc325]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] Running cmd (subprocess): nvme version {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] 'nvme version' failed. Not Retrying. {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.nvmeof [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] nvme not present on system {{(pid=72351) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:19:54 user nova-compute[72351]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Aug 09 02:19:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/74728ca8-34d4-45e6-a1f2-21c0952ff835/disk --force-share --output=json" returned: 0 in 0.139s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[eb54e738-4f71-4645-97aa-451a52727948]: (4, '') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72351) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] LIGHTOS: no hostnqn found. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG os_brick.utils [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] <== get_connector_properties: return (179ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:a52269602ff4', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/97b13ccd-15b4-44bf-b865-71b4da861168/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] No VIF found with MAC fa:16:3e:63:b2:2a, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:19:54 user nova-compute[72351]: INFO nova.compute.manager [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Swapping volume edeae21b-4fa4-464b-b7de-da06cf9e8f58 for 0203626e-04e6-4768-b2c2-e7dbaeed4c6c
Aug 09 02:19:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/97b13ccd-15b4-44bf-b865-71b4da861168/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/97b13ccd-15b4-44bf-b865-71b4da861168/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:19:55 user nova-compute[72351]: DEBUG nova.network.neutron [req-77f69d7d-3834-43f1-9395-1c259112b930 req-ea3aa5bf-8cb6-40a2-87b4-a2d2b4949569 service nova] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] Updated VIF entry in instance network info cache for port 84074c82-d423-45d8-9665-bd92d13b6380. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:19:55 user nova-compute[72351]: DEBUG nova.network.neutron [req-77f69d7d-3834-43f1-9395-1c259112b930 req-ea3aa5bf-8cb6-40a2-87b4-a2d2b4949569 service nova] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] Updating instance_info_cache with network_info: [{"id": "84074c82-d423-45d8-9665-bd92d13b6380", "address": "fa:16:3e:63:b2:2a", "network": {"id": "885b8ede-6326-4f34-aaaf-159df1b5e4e5", "bridge": "br-int", "label": "tempest-VolumesActionsTest-661898154-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d6461f5cffd540b2b7f6797390561a80", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap84074c82-d4", "ovs_interfaceid": "84074c82-d423-45d8-9665-bd92d13b6380", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:19:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/97b13ccd-15b4-44bf-b865-71b4da861168/disk --force-share --output=json" returned: 0 in 0.147s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:19:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-77f69d7d-3834-43f1-9395-1c259112b930 req-ea3aa5bf-8cb6-40a2-87b4-a2d2b4949569 service nova] Releasing lock "refresh_cache-52f0d664-7767-4953-bc7a-496b26f22715" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:19:55 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sde (vda) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:19:58 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547598.2960532, 52f0d664-7767-4953-bc7a-496b26f22715 => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:19:58 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] VM Resumed (Lifecycle Event)
Aug 09 02:19:58 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:19:58 user nova-compute[72351]: DEBUG nova.compute.manager [req-e06d6053-e685-4755-9e80-ebdd2fe04d4b req-81b27b83-dfc2-4a77-91ce-e58dd0261c4e service nova] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] Received event network-vif-plugged-84074c82-d423-45d8-9665-bd92d13b6380 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:19:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-e06d6053-e685-4755-9e80-ebdd2fe04d4b req-81b27b83-dfc2-4a77-91ce-e58dd0261c4e service nova] Acquiring lock "52f0d664-7767-4953-bc7a-496b26f22715-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-e06d6053-e685-4755-9e80-ebdd2fe04d4b req-81b27b83-dfc2-4a77-91ce-e58dd0261c4e service nova] Lock "52f0d664-7767-4953-bc7a-496b26f22715-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-e06d6053-e685-4755-9e80-ebdd2fe04d4b req-81b27b83-dfc2-4a77-91ce-e58dd0261c4e service nova] Lock "52f0d664-7767-4953-bc7a-496b26f22715-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:58 user nova-compute[72351]: DEBUG nova.compute.manager [req-e06d6053-e685-4755-9e80-ebdd2fe04d4b req-81b27b83-dfc2-4a77-91ce-e58dd0261c4e service nova] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] No waiting events found dispatching network-vif-plugged-84074c82-d423-45d8-9665-bd92d13b6380 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:19:58 user nova-compute[72351]: WARNING nova.compute.manager [req-e06d6053-e685-4755-9e80-ebdd2fe04d4b req-81b27b83-dfc2-4a77-91ce-e58dd0261c4e service nova] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] Received unexpected event network-vif-plugged-84074c82-d423-45d8-9665-bd92d13b6380 for instance with vm_state building and task_state spawning.
Aug 09 02:19:58 user nova-compute[72351]: DEBUG nova.compute.manager [req-e798c5e2-b7ca-4f46-b332-326ae830c553 req-c3683b77-f562-4bbb-89cc-b66089d5e959 service nova] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] Received event network-vif-plugged-84074c82-d423-45d8-9665-bd92d13b6380 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:19:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-e798c5e2-b7ca-4f46-b332-326ae830c553 req-c3683b77-f562-4bbb-89cc-b66089d5e959 service nova] Acquiring lock "52f0d664-7767-4953-bc7a-496b26f22715-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-e798c5e2-b7ca-4f46-b332-326ae830c553 req-c3683b77-f562-4bbb-89cc-b66089d5e959 service nova] Lock "52f0d664-7767-4953-bc7a-496b26f22715-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-e798c5e2-b7ca-4f46-b332-326ae830c553 req-c3683b77-f562-4bbb-89cc-b66089d5e959 service nova] Lock "52f0d664-7767-4953-bc7a-496b26f22715-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:58 user nova-compute[72351]: DEBUG nova.compute.manager [req-e798c5e2-b7ca-4f46-b332-326ae830c553 req-c3683b77-f562-4bbb-89cc-b66089d5e959 service nova] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] No waiting events found dispatching network-vif-plugged-84074c82-d423-45d8-9665-bd92d13b6380 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:19:58 user nova-compute[72351]: WARNING nova.compute.manager [req-e798c5e2-b7ca-4f46-b332-326ae830c553 req-c3683b77-f562-4bbb-89cc-b66089d5e959 service nova] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] Received unexpected event network-vif-plugged-84074c82-d423-45d8-9665-bd92d13b6380 for instance with vm_state building and task_state spawning.
Aug 09 02:19:58 user nova-compute[72351]: DEBUG nova.compute.manager [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:19:58 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] Guest created on hypervisor {{(pid=72351) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4411}}
Aug 09 02:19:58 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:19:58 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] Instance spawned successfully.
Aug 09 02:19:58 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:894}}
Aug 09 02:19:58 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:19:58 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:19:58 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Hypervisor/Node resource view: name=user free_ram=7355MB free_disk=25.61135482788086GB free_vcpus=4 pci_devices=[{"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72351) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Aug 09 02:19:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:19:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:19:58 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:19:58 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:19:58 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547598.2978766, 52f0d664-7767-4953-bc7a-496b26f22715 => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:19:58 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] VM Started (Lifecycle Event)
Aug 09 02:19:58 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] Found default for hw_cdrom_bus of ide {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:19:58 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] Found default for hw_disk_bus of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:19:58 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] Found default for hw_input_bus of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:19:58 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] Found default for hw_pointer_model of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:19:58 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] Found default for hw_video_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:19:58 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] Found default for hw_vif_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:19:58 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:19:58 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:19:58 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:19:58 user nova-compute[72351]: INFO nova.compute.manager [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] Took 6.56 seconds to spawn the instance on the hypervisor.
Aug 09 02:19:58 user nova-compute[72351]: DEBUG nova.compute.manager [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:19:59 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 2d78475c-107d-4256-b12e-dbe9c8f3b9b2 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:19:59 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 74728ca8-34d4-45e6-a1f2-21c0952ff835 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:19:59 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:19:59 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:19:59 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 97b13ccd-15b4-44bf-b865-71b4da861168 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:19:59 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 629e177e-afbb-4765-a0b5-7a3f66f72ebf actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:19:59 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance a5ca6400-1125-431e-8a91-057e746d40b5 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:19:59 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 822e25c4-8dd1-4e21-b0ed-f340d59c6c98 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:19:59 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 52f0d664-7767-4953-bc7a-496b26f22715 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:19:59 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Total usable vcpus: 12, total allocated vcpus: 9 {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Aug 09 02:19:59 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Final resource view: name=user phys_ram=16011MB used_ram=5120MB phys_disk=40GB used_disk=7GB total_vcpus=12 used_vcpus=9 pci_stats=[] {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Aug 09 02:19:59 user nova-compute[72351]: INFO nova.compute.manager [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] Took 7.29 seconds to build instance.
Aug 09 02:19:59 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f35c853f-925f-4679-ab39-1abdb6bb4a02 tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Lock "52f0d664-7767-4953-bc7a-496b26f22715" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 7.381s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:59 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:19:59 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:19:59 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Compute_service record updated for user:user {{(pid=72351) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Aug 09 02:19:59 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.543s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:19:59 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:00 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:20:00 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:20:00 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Starting heal instance info cache {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9842}}
Aug 09 02:20:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "refresh_cache-74728ca8-34d4-45e6-a1f2-21c0952ff835" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:20:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquired lock "refresh_cache-74728ca8-34d4-45e6-a1f2-21c0952ff835" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:20:00 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Forcefully refreshing network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1996}}
Aug 09 02:20:01 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Updating instance_info_cache with network_info: [{"id": "3718df01-0e23-424c-8ab1-bf0fd9c7e121", "address": "fa:16:3e:7c:0b:0d", "network": {"id": "21efbfa4-f3a2-4317-89fc-3cc7b60fde97", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1919568535-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.239", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bc5e05fbd7f44f85bc3b1cc169dd03a9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3718df01-0e", "ovs_interfaceid": "3718df01-0e23-424c-8ab1-bf0fd9c7e121", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:20:01 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Releasing lock "refresh_cache-74728ca8-34d4-45e6-a1f2-21c0952ff835" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:20:01 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Updated the network info_cache for instance {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9913}}
Aug 09 02:20:01 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:20:01 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:20:01 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:20:01 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:20:01 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e9fe4002-3316-40be-a63f-78ee56bd7e7d tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Acquiring lock "97b13ccd-15b4-44bf-b865-71b4da861168" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:20:01 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e9fe4002-3316-40be-a63f-78ee56bd7e7d tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "97b13ccd-15b4-44bf-b865-71b4da861168" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:20:01 user nova-compute[72351]: DEBUG nova.objects.instance [None req-e9fe4002-3316-40be-a63f-78ee56bd7e7d tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lazy-loading 'flavor' on Instance uuid 97b13ccd-15b4-44bf-b865-71b4da861168 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:20:01 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e9fe4002-3316-40be-a63f-78ee56bd7e7d tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "97b13ccd-15b4-44bf-b865-71b4da861168" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.069s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:20:01 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e9fe4002-3316-40be-a63f-78ee56bd7e7d tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Acquiring lock "97b13ccd-15b4-44bf-b865-71b4da861168" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:20:01 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e9fe4002-3316-40be-a63f-78ee56bd7e7d tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "97b13ccd-15b4-44bf-b865-71b4da861168" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:20:01 user nova-compute[72351]: INFO nova.compute.manager [None req-e9fe4002-3316-40be-a63f-78ee56bd7e7d tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Attaching volume cda288d2-cdeb-4b8f-abbf-acaa3fab484f to /dev/vdb
Aug 09 02:20:01 user nova-compute[72351]: DEBUG os_brick.utils [None req-e9fe4002-3316-40be-a63f-78ee56bd7e7d tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:20:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.015s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:01 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[235a769b-0407-4a1f-9bef-854ee728a8ef]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:a52269602ff4\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:01 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[775a5d55-5f22-48b6-a66b-b799d6b4ccb3]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:01 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[18a60009-0327-429b-9ed3-ce98c404ddee]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:01 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[6fc76938-82ce-4838-929f-a8c364a2e979]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-e9fe4002-3316-40be-a63f-78ee56bd7e7d tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Running cmd (subprocess): nvme version {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-e9fe4002-3316-40be-a63f-78ee56bd7e7d tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] 'nvme version' failed. Not Retrying. {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:20:01 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.nvmeof [None req-e9fe4002-3316-40be-a63f-78ee56bd7e7d tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] nvme not present on system {{(pid=72351) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Aug 09 02:20:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:20:01 user nova-compute[72351]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Aug 09 02:20:01 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[47c74a83-1be1-443e-a36a-d46e30e92011]: (4, '') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:01 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-e9fe4002-3316-40be-a63f-78ee56bd7e7d tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72351) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Aug 09 02:20:01 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-e9fe4002-3316-40be-a63f-78ee56bd7e7d tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Aug 09 02:20:01 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-e9fe4002-3316-40be-a63f-78ee56bd7e7d tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] LIGHTOS: no hostnqn found. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Aug 09 02:20:01 user nova-compute[72351]: DEBUG os_brick.utils [None req-e9fe4002-3316-40be-a63f-78ee56bd7e7d tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] <== get_connector_properties: return (136ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:a52269602ff4', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:20:01 user nova-compute[72351]: DEBUG nova.virt.block_device [None req-e9fe4002-3316-40be-a63f-78ee56bd7e7d tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Updating existing volume attachment record: 37337a2e-29d1-4999-8d68-3d7a65691f1e {{(pid=72351) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Aug 09 02:20:01 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:02 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:03 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691547588.1932862, 50302d76-f6b9-4fe3-b55a-4a01e483e2e3 => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:20:03 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] VM Stopped (Lifecycle Event)
Aug 09 02:20:03 user nova-compute[72351]: DEBUG nova.compute.manager [None req-9b73241c-3596-4441-9e08-0d7ffb428716 None None] [instance: 50302d76-f6b9-4fe3-b55a-4a01e483e2e3] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:20:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:07 user nova-compute[72351]: DEBUG nova.compute.manager [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] swap_volume: Calling driver volume swap with connection infos: new: {'driver_volume_type': 'iscsi', 'data': {'target_iqns': ['iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'sZqwozDV', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b', 'attached_at': '', 'detached_at': '', 'volume_id': '0203626e-04e6-4768-b2c2-e7dbaeed4c6c', 'serial': '0203626e-04e6-4768-b2c2-e7dbaeed4c6c'}; old: {'driver_volume_type': 'iscsi', 'data': {'target_iqns': ['iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'CgQaLjJu', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc', 'volume_id': 'edeae21b-4fa4-464b-b7de-da06cf9e8f58'}, 'status': 'reserved', 'instance': '11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b', 'attached_at': '', 'detached_at': '', 'volume_id': 'edeae21b-4fa4-464b-b7de-da06cf9e8f58', 'serial'} {{(pid=72351) _swap_volume /opt/stack/nova/nova/compute/manager.py:7702}}
Aug 09 02:20:07 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 02:20:07 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'sZqwozDV', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:20:07 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:20:07 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:20:07 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:20:07 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:20:07 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:20:07 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:20:07 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 02:20:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c -p 172.16.0.220:3260" returned: 21 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:07 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[fe3c2630-a06c-4189-8006-0ab14f2bb7fe]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:07 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] iscsiadm (): stdout= stderr=iscsiadm: No records found
Aug 09 02:20:07 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:20:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c -p 172.16.0.220:3260 --interface default --op new {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:07 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a2d61b81-931f-45c2-9455-f46f72948c27]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c] added\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:07 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c] added
Aug 09 02:20:07 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:20:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:07 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[983ae817-87ae-4e4d-9f70-13bf61d837d6]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:07 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:20:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:07 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[8d7026dc-ae55-485a-baaa-64c0e7c0806e]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:07 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:20:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c -p 172.16.0.220:3260 --op update -n node.session.auth.username -v sZqwozDV {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c -p 172.16.0.220:3260 --op update -n node.session.auth.username -v sZqwozDV" returned: 0 in 0.030s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:08 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[bbd6f30b-429b-4231-be86-d2e53075a87d]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'sZqwozDV'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:20:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:08 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[0ae4ebb1-9081-4d66-a4a6-6381abd69a73]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:20:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:08 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[39c80694-7375-4b8f-a8e4-f001d5835f4b]: (4, ('tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58 (non-flash)\ntcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] iscsiadm ('-m', 'session'): stdout=tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58 (non-flash)
Aug 09 02:20:08 user nova-compute[72351]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:20:08 user nova-compute[72351]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)
Aug 09 02:20:08 user nova-compute[72351]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:20:08 user nova-compute[72351]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)
Aug 09 02:20:08 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:20:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] iscsi session list stdout=tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58 (non-flash)
Aug 09 02:20:08 user nova-compute[72351]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:20:08 user nova-compute[72351]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)
Aug 09 02:20:08 user nova-compute[72351]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:20:08 user nova-compute[72351]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)
Aug 09 02:20:08 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:20:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c -p 172.16.0.220:3260 --login {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c -p 172.16.0.220:3260 --login" returned: 0 in 0.072s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:08 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d1e89bcc-f9e3-4d7b-8f2a-ef4ba414bec8]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c, portal: 172.16.0.220,3260]
Aug 09 02:20:08 user nova-compute[72351]: Login to [iface: default, target: iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c, portal: 172.16.0.220,3260] successful.
Aug 09 02:20:08 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:20:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:08 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[b52c6532-bdbd-4309-9451-bd3e0273047e]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:20:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.035s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:08 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a328c9da-443c-4082-80a6-45d3cbde3469]: (4, ('tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58 (non-flash)\ntcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c (non-flash)\ntcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] iscsiadm ('-m', 'session'): stdout=tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58 (non-flash)
Aug 09 02:20:08 user nova-compute[72351]: tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c (non-flash)
Aug 09 02:20:08 user nova-compute[72351]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:20:08 user nova-compute[72351]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)
Aug 09 02:20:08 user nova-compute[72351]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:20:08 user nova-compute[72351]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)
Aug 09 02:20:08 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:20:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] iscsi session list stdout=tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58 (non-flash)
Aug 09 02:20:08 user nova-compute[72351]: tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c (non-flash)
Aug 09 02:20:08 user nova-compute[72351]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:20:08 user nova-compute[72351]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)
Aug 09 02:20:08 user nova-compute[72351]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:20:08 user nova-compute[72351]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)
Aug 09 02:20:08 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:20:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.308s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:20:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 02:20:08 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] HCTL ('38', '-', '-', 0) found on session 12 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 02:20:08 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] Scanning host 38 c: -, t: -, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 02:20:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host38/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host38/scan" returned: 0 in 0.027s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:08 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a1ddcd43-7372-4521-9de0-cc609c17f1a0]: (4, ('- - 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:08 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] Searching for a device in session 12 and hctl ['38', '*', '*', 0] yield: sdg {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:20:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] Connected to sdg using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'sZqwozDV', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 02:20:09 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.354s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:20:09 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] <== connect_volume: return (1356ms) {'type': 'block', 'scsi_wwn': '23732393866386262', 'path': '/dev/sdg'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:20:09 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23732393866386262', 'path': '/dev/sdg'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 02:20:09 user nova-compute[72351]: DEBUG nova.objects.instance [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] Lazy-loading 'flavor' on Instance uuid 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:20:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:20:09 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] COPY block job progress, current cursor: 0 final cursor: 1073741824 {{(pid=72351) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Aug 09 02:20:10 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] COPY block job progress, current cursor: 156237824 final cursor: 1073741824 {{(pid=72351) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Aug 09 02:20:10 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] COPY block job progress, current cursor: 316669952 final cursor: 1073741824 {{(pid=72351) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Aug 09 02:20:11 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-e9fe4002-3316-40be-a63f-78ee56bd7e7d tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 02:20:11 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e9fe4002-3316-40be-a63f-78ee56bd7e7d tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'cIMBpXdn', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:20:11 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-e9fe4002-3316-40be-a63f-78ee56bd7e7d tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:20:11 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-e9fe4002-3316-40be-a63f-78ee56bd7e7d tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:20:11 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-e9fe4002-3316-40be-a63f-78ee56bd7e7d tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:20:11 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-e9fe4002-3316-40be-a63f-78ee56bd7e7d tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:20:11 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-e9fe4002-3316-40be-a63f-78ee56bd7e7d tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:20:11 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-e9fe4002-3316-40be-a63f-78ee56bd7e7d tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:20:11 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-e9fe4002-3316-40be-a63f-78ee56bd7e7d tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 02:20:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f -p 172.16.0.220:3260" returned: 21 in 0.028s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:11 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d5a5f7ae-d002-4f48-bcd7-0de69e69353d]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:11 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e9fe4002-3316-40be-a63f-78ee56bd7e7d tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Aug 09 02:20:11 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:20:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f -p 172.16.0.220:3260 --interface default --op new {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.033s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:11 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[31fb74cd-689f-4c45-bb7e-a0519844de86]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f] added\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:11 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e9fe4002-3316-40be-a63f-78ee56bd7e7d tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f] added
Aug 09 02:20:11 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:20:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.025s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:11 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[df0be9e0-9275-4c45-b59a-2c0bdbd68eab]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:11 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e9fe4002-3316-40be-a63f-78ee56bd7e7d tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:20:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:11 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[543e8e05-cfbe-4e91-9f85-41b68a73ecf5]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:11 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e9fe4002-3316-40be-a63f-78ee56bd7e7d tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:20:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f -p 172.16.0.220:3260 --op update -n node.session.auth.username -v cIMBpXdn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f -p 172.16.0.220:3260 --op update -n node.session.auth.username -v cIMBpXdn" returned: 0 in 0.024s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:11 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[156f987d-179c-4625-86f4-a2c064f991be]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:11 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e9fe4002-3316-40be-a63f-78ee56bd7e7d tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'cIMBpXdn'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:20:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.027s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:11 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[95a534a8-e8e5-4062-87de-42194380ce8f]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:11 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e9fe4002-3316-40be-a63f-78ee56bd7e7d tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:20:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:11 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] COPY block job progress, current cursor: 459276288 final cursor: 1073741824 {{(pid=72351) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Aug 09 02:20:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.034s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:11 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a59c62e1-fd5f-4809-a351-377b35ed4084]: (4, ('tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58 (non-flash)\ntcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c (non-flash)\ntcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:11 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e9fe4002-3316-40be-a63f-78ee56bd7e7d tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58 (non-flash)
Aug 09 02:20:11 user nova-compute[72351]: tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c (non-flash)
Aug 09 02:20:11 user nova-compute[72351]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:20:11 user nova-compute[72351]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)
Aug 09 02:20:11 user nova-compute[72351]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:20:11 user nova-compute[72351]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)
Aug 09 02:20:11 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:20:11 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e9fe4002-3316-40be-a63f-78ee56bd7e7d tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] iscsi session list stdout=tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58 (non-flash)
Aug 09 02:20:11 user nova-compute[72351]: tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c (non-flash)
Aug 09 02:20:11 user nova-compute[72351]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:20:11 user nova-compute[72351]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)
Aug 09 02:20:11 user nova-compute[72351]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:20:11 user nova-compute[72351]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)
Aug 09 02:20:11 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:20:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f -p 172.16.0.220:3260 --login {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f -p 172.16.0.220:3260 --login" returned: 0 in 0.092s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:11 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d45fa678-7394-4d62-a9d1-28d441c3cc7e]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:11 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e9fe4002-3316-40be-a63f-78ee56bd7e7d tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f, portal: 172.16.0.220,3260]
Aug 09 02:20:11 user nova-compute[72351]: Login to [iface: default, target: iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f, portal: 172.16.0.220,3260] successful.
Aug 09 02:20:11 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:20:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:11 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d78f85c2-2bd3-4853-80bb-45a4e28aa101]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:11 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e9fe4002-3316-40be-a63f-78ee56bd7e7d tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:20:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.041s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:11 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[2c50909d-ff6d-465a-acf8-60e497d3bdba]: (4, ('tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58 (non-flash)\ntcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c (non-flash)\ntcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f (non-flash)\ntcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:11 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e9fe4002-3316-40be-a63f-78ee56bd7e7d tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58 (non-flash)
Aug 09 02:20:11 user nova-compute[72351]: tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c (non-flash)
Aug 09 02:20:11 user nova-compute[72351]: tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f (non-flash)
Aug 09 02:20:11 user nova-compute[72351]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:20:11 user nova-compute[72351]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)
Aug 09 02:20:11 user nova-compute[72351]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:20:11 user nova-compute[72351]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)
Aug 09 02:20:11 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:20:11 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e9fe4002-3316-40be-a63f-78ee56bd7e7d tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] iscsi session list stdout=tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58 (non-flash)
Aug 09 02:20:11 user nova-compute[72351]: tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c (non-flash)
Aug 09 02:20:11 user nova-compute[72351]: tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f (non-flash)
Aug 09 02:20:11 user nova-compute[72351]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:20:11 user nova-compute[72351]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)
Aug 09 02:20:11 user nova-compute[72351]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:20:11 user nova-compute[72351]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)
Aug 09 02:20:11 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:20:11 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-e9fe4002-3316-40be-a63f-78ee56bd7e7d tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.410s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:20:11 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e9fe4002-3316-40be-a63f-78ee56bd7e7d tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 02:20:11 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-e9fe4002-3316-40be-a63f-78ee56bd7e7d tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] HCTL ('39', '-', '-', 0) found on session 13 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 02:20:11 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-e9fe4002-3316-40be-a63f-78ee56bd7e7d tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Scanning host 39 c: -, t: -, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 02:20:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host39/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host39/scan" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:11 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[396f628d-7ca8-4deb-ae58-27a95f19ab6e]: (4, ('- - 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:11 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-e9fe4002-3316-40be-a63f-78ee56bd7e7d tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Searching for a device in session 13 and hctl ['39', '*', '*', 0] yield: None {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:20:11 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] COPY block job progress, current cursor: 622854144 final cursor: 1073741824 {{(pid=72351) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Aug 09 02:20:12 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] COPY block job progress, current cursor: 785383424 final cursor: 1073741824 {{(pid=72351) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Aug 09 02:20:12 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:12 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-e9fe4002-3316-40be-a63f-78ee56bd7e7d tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Searching for a device in session 13 and hctl ['39', '*', '*', 0] yield: sdh {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:20:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e9fe4002-3316-40be-a63f-78ee56bd7e7d tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Connected to sdh using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'cIMBpXdn', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 02:20:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-e9fe4002-3316-40be-a63f-78ee56bd7e7d tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.445s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:20:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e9fe4002-3316-40be-a63f-78ee56bd7e7d tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] <== connect_volume: return (1446ms) {'type': 'block', 'scsi_wwn': '23130366333646234', 'path': '/dev/sdh'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:20:12 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-e9fe4002-3316-40be-a63f-78ee56bd7e7d tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23130366333646234', 'path': '/dev/sdh'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 02:20:12 user nova-compute[72351]: DEBUG nova.objects.instance [None req-e9fe4002-3316-40be-a63f-78ee56bd7e7d tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lazy-loading 'flavor' on Instance uuid 97b13ccd-15b4-44bf-b865-71b4da861168 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:20:12 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-e9fe4002-3316-40be-a63f-78ee56bd7e7d tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] attach device xml: <disk type="block" device="disk">
Aug 09 02:20:12 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:20:12 user nova-compute[72351]:   <source dev="/dev/sdh"/>
Aug 09 02:20:12 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:20:12 user nova-compute[72351]:   <serial>cda288d2-cdeb-4b8f-abbf-acaa3fab484f</serial>
Aug 09 02:20:12 user nova-compute[72351]: </disk>
Aug 09 02:20:12 user nova-compute[72351]:  {{(pid=72351) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Aug 09 02:20:12 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] COPY block job progress, current cursor: 916455424 final cursor: 1073741824 {{(pid=72351) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Aug 09 02:20:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-e9fe4002-3316-40be-a63f-78ee56bd7e7d tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:20:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-e9fe4002-3316-40be-a63f-78ee56bd7e7d tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:20:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-e9fe4002-3316-40be-a63f-78ee56bd7e7d tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] No VIF found with MAC fa:16:3e:d4:4d:0a, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:20:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e9fe4002-3316-40be-a63f-78ee56bd7e7d tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "97b13ccd-15b4-44bf-b865-71b4da861168" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 11.808s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:20:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] COPY block job progress, current cursor: 1073741824 final cursor: 1073741824 {{(pid=72351) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Aug 09 02:20:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 02:20:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'CgQaLjJu', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc', 'volume_id': 'edeae21b-4fa4-464b-b7de-da06cf9e8f58'}, None), 'kwargs': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:20:13 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:20:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:20:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:20:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 02:20:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:13 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[52708cb7-55c3-45fa-9c2e-9174a8ed9eb1]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.029s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:13 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[fcd05246-8e1d-40c5-a442-2a2d0b80e9ea]: (4, ('tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58 (non-flash)\ntcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c (non-flash)\ntcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f (non-flash)\ntcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] iscsiadm ('-m', 'session'): stdout=tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58 (non-flash)
Aug 09 02:20:13 user nova-compute[72351]: tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c (non-flash)
Aug 09 02:20:13 user nova-compute[72351]: tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f (non-flash)
Aug 09 02:20:13 user nova-compute[72351]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:20:13 user nova-compute[72351]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)
Aug 09 02:20:13 user nova-compute[72351]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:20:13 user nova-compute[72351]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)
Aug 09 02:20:13 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:20:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] iscsi session list stdout=tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58 (non-flash)
Aug 09 02:20:13 user nova-compute[72351]: tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c (non-flash)
Aug 09 02:20:13 user nova-compute[72351]: tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f (non-flash)
Aug 09 02:20:13 user nova-compute[72351]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:20:13 user nova-compute[72351]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)
Aug 09 02:20:13 user nova-compute[72351]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:20:13 user nova-compute[72351]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)
Aug 09 02:20:13 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:20:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fcffc1b77f0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58'): ({'sdc'}, set())}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 02:20:13 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] Removing single pathed devices sdc {{(pid=72351) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Aug 09 02:20:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:13 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ba226ce0-b2e7-4010-8dcf-9dd4d70c0b0f]: (4, ('path checker states:\nup                  6\npending             1\n\npaths: 0\nbusy: False\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdc {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdc" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:13 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[30a02d4c-8aae-4dcf-89c3-0d64585bae75]: (4, ('ok\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:13 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] Flushing IO for device /dev/sdc {{(pid=72351) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Aug 09 02:20:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdc {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdc" returned: 0 in 0.013s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:13 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[15ca0720-2f73-402f-a4ce-cde0ecb81b90]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:13 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] Remove SCSI device /dev/sdc with /sys/block/sdc/device/delete {{(pid=72351) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Aug 09 02:20:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdc/device/delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdc/device/delete" returned: 0 in 0.048s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:13 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[0509f010-0a41-4d70-80fc-bda2990faa34]: (4, ('1', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:13 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] Checking to see if SCSI volumes sdc have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Aug 09 02:20:13 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] SCSI volumes sdc have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Aug 09 02:20:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58')] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 02:20:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:13 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[02e17175-c19f-4984-b944-1f88e6eb30bb]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:20:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58 -p 172.16.0.220:3260 --logout {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58 -p 172.16.0.220:3260 --logout" returned: 0 in 0.069s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:13 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[0369cdca-fb70-4839-9f1c-7c44a20e8d5e]: (4, ('Logging out of session [sid: 11, target: iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58, portal: 172.16.0.220,3260]\nLogout of [sid: 11, target: iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] iscsiadm ('--logout',): stdout=Logging out of session [sid: 11, target: iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58, portal: 172.16.0.220,3260]
Aug 09 02:20:13 user nova-compute[72351]: Logout of [sid: 11, target: iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58, portal: 172.16.0.220,3260] successful.
Aug 09 02:20:13 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:20:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58 -p 172.16.0.220:3260 --op delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:edeae21b-4fa4-464b-b7de-da06cf9e8f58 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.031s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:13 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[9b3fb99a-5659-4914-96bc-7ad38d3d838c]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:20:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.316s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:20:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] <== disconnect_volume: return (316ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:20:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 02:20:13 user nova-compute[72351]: DEBUG nova.compute.manager [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] swap_volume: Driver volume swap returned, new connection_info is now : {'driver_volume_type': 'iscsi', 'data': {'target_iqns': ['iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'sZqwozDV', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdg'}, 'status': 'reserved', 'instance': '11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b', 'attached_at': '', 'detached_at': '', 'volume_id': '0203626e-04e6-4768-b2c2-e7dbaeed4c6c', 'serial'} {{(pid=72351) _swap_volume /opt/stack/nova/nova/compute/manager.py:7711}}
Aug 09 02:20:13 user nova-compute[72351]: DEBUG nova.compute.manager [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] swap_volume: removing Cinder connection for volume edeae21b-4fa4-464b-b7de-da06cf9e8f58 {{(pid=72351) _swap_volume /opt/stack/nova/nova/compute/manager.py:7756}}
Aug 09 02:20:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:14 user nova-compute[72351]: DEBUG nova.compute.manager [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] swap_volume: Updating volume edeae21b-4fa4-464b-b7de-da06cf9e8f58 BDM record with {'connection_info': '{"driver_volume_type": "iscsi", "data": {"target_iqns": ["iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c"], "target_portals": ["172.16.0.220:3260"], "target_luns": [0], "target_discovered": false, "auth_method": "CHAP", "auth_username": "sZqwozDV", "auth_password": "gJ8hYdaiuNLTIn", "target_lun": 0, "qos_specs": null, "access_mode": "rw", "encrypted": false, "cacheable": false, "device_path": "/dev/sdg", "volume_id": "0203626e-04e6-4768-b2c2-e7dbaeed4c6c"}, "status": "reserved", "instance": "11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b", "attached_at": "", "detached_at": "", "volume_id": "0203626e-04e6-4768-b2c2-e7dbaeed4c6c", "serial": "0203626e-04e6-4768-b2c2-e7dbaeed4c6c"}', 'source_type': 'volume', 'destination_type': 'volume', 'snapshot_id': None, 'volume_id': '0203626e-04e6-4768-b2c2-e7dbaeed4c6c', 'no_device': None, 'attachment_id': '4b1f843d-e7f2-4288-9a0f-89121ad54316'} {{(pid=72351) _do_swap_volume /opt/stack/nova/nova/compute/manager.py:7904}}
Aug 09 02:20:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5ef8f6a5-ca6c-49f8-8fc3-d55f188db027 tempest-TestVolumeSwap-2111681709 tempest-TestVolumeSwap-2111681709-project-admin] Lock "11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b" "released" by "nova.compute.manager.ComputeManager.swap_volume.<locals>._do_locked_swap_volume" :: held 20.128s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:20:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-d7642f0e-fd44-4b28-9769-a4b70af24cef tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Acquiring lock "11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:20:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-d7642f0e-fd44-4b28-9769-a4b70af24cef tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Lock "11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:20:15 user nova-compute[72351]: INFO nova.compute.manager [None req-d7642f0e-fd44-4b28-9769-a4b70af24cef tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Detaching volume 0203626e-04e6-4768-b2c2-e7dbaeed4c6c
Aug 09 02:20:15 user nova-compute[72351]: INFO nova.virt.block_device [None req-d7642f0e-fd44-4b28-9769-a4b70af24cef tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Attempting to driver detach volume 0203626e-04e6-4768-b2c2-e7dbaeed4c6c from mountpoint /dev/vdb
Aug 09 02:20:15 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-d7642f0e-fd44-4b28-9769-a4b70af24cef tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Attempting to detach device vdb from instance 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b from the persistent domain config. {{(pid=72351) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2477}}
Aug 09 02:20:15 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-d7642f0e-fd44-4b28-9769-a4b70af24cef tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 02:20:15 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:20:15 user nova-compute[72351]:   <source dev="/dev/sdg"/>
Aug 09 02:20:15 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:20:15 user nova-compute[72351]:   <serial>0203626e-04e6-4768-b2c2-e7dbaeed4c6c</serial>
Aug 09 02:20:15 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Aug 09 02:20:15 user nova-compute[72351]: </disk>
Aug 09 02:20:15 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 02:20:15 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-d7642f0e-fd44-4b28-9769-a4b70af24cef tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Successfully detached device vdb from instance 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b from the persistent domain config.
Aug 09 02:20:15 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-d7642f0e-fd44-4b28-9769-a4b70af24cef tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b from the live domain config. {{(pid=72351) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2513}}
Aug 09 02:20:15 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-d7642f0e-fd44-4b28-9769-a4b70af24cef tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 02:20:15 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:20:15 user nova-compute[72351]:   <source dev="/dev/sdg"/>
Aug 09 02:20:15 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:20:15 user nova-compute[72351]:   <serial>0203626e-04e6-4768-b2c2-e7dbaeed4c6c</serial>
Aug 09 02:20:15 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Aug 09 02:20:15 user nova-compute[72351]: </disk>
Aug 09 02:20:15 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 02:20:16 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Received event <DeviceRemovedEvent: 1691547616.200996, 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=72351) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2360}}
Aug 09 02:20:16 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-d7642f0e-fd44-4b28-9769-a4b70af24cef tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b {{(pid=72351) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2589}}
Aug 09 02:20:16 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-d7642f0e-fd44-4b28-9769-a4b70af24cef tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Successfully detached device vdb from instance 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b from the live domain config.
Aug 09 02:20:16 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-d7642f0e-fd44-4b28-9769-a4b70af24cef tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 02:20:16 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7642f0e-fd44-4b28-9769-a4b70af24cef tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'sZqwozDV', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdg', 'volume_id': '0203626e-04e6-4768-b2c2-e7dbaeed4c6c'}, None), 'kwargs': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:20:16 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-d7642f0e-fd44-4b28-9769-a4b70af24cef tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:20:16 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-d7642f0e-fd44-4b28-9769-a4b70af24cef tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:20:16 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-d7642f0e-fd44-4b28-9769-a4b70af24cef tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:20:16 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7642f0e-fd44-4b28-9769-a4b70af24cef tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 02:20:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:16 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a3cf11f4-ce43-43a3-acab-a6b80566de51]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.035s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:16 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[1891df0b-f959-4cbf-99e3-c3da5368b0f5]: (4, ('tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c (non-flash)\ntcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f (non-flash)\ntcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:16 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7642f0e-fd44-4b28-9769-a4b70af24cef tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c (non-flash)
Aug 09 02:20:16 user nova-compute[72351]: tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f (non-flash)
Aug 09 02:20:16 user nova-compute[72351]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:20:16 user nova-compute[72351]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)
Aug 09 02:20:16 user nova-compute[72351]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:20:16 user nova-compute[72351]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)
Aug 09 02:20:16 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:20:16 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7642f0e-fd44-4b28-9769-a4b70af24cef tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] iscsi session list stdout=tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c (non-flash)
Aug 09 02:20:16 user nova-compute[72351]: tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f (non-flash)
Aug 09 02:20:16 user nova-compute[72351]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:20:16 user nova-compute[72351]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)
Aug 09 02:20:16 user nova-compute[72351]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:20:16 user nova-compute[72351]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)
Aug 09 02:20:16 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:20:16 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7642f0e-fd44-4b28-9769-a4b70af24cef tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fcfdc509f30>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c'): ({'sdg'}, set())}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 02:20:16 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-d7642f0e-fd44-4b28-9769-a4b70af24cef tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Removing single pathed devices sdg {{(pid=72351) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Aug 09 02:20:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:16 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a0412fd5-3bc2-4909-8ab5-7215374808b8]: (4, ('path checker states:\nup                  5\npending             1\n\npaths: 0\nbusy: False\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdg {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdg" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:16 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d0ceb6d1-0532-4fcf-ad86-e245cb587d7a]: (4, ('ok\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:16 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-d7642f0e-fd44-4b28-9769-a4b70af24cef tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Flushing IO for device /dev/sdg {{(pid=72351) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Aug 09 02:20:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdg {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdg" returned: 0 in 0.013s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:16 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[9e461f39-7a5d-48be-9cfa-ceca5928da34]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:16 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-d7642f0e-fd44-4b28-9769-a4b70af24cef tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Remove SCSI device /dev/sdg with /sys/block/sdg/device/delete {{(pid=72351) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Aug 09 02:20:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdg/device/delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdg/device/delete" returned: 0 in 0.035s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:16 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ff95fd95-9554-4130-8715-a7968ed1067b]: (4, ('1', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:16 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-d7642f0e-fd44-4b28-9769-a4b70af24cef tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Checking to see if SCSI volumes sdg have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Aug 09 02:20:16 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-d7642f0e-fd44-4b28-9769-a4b70af24cef tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] SCSI volumes sdg have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Aug 09 02:20:16 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7642f0e-fd44-4b28-9769-a4b70af24cef tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c')] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 02:20:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:16 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[5d9470ef-0720-4921-9adc-b56d1724c23b]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:16 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7642f0e-fd44-4b28-9769-a4b70af24cef tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:20:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c -p 172.16.0.220:3260 --logout {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c -p 172.16.0.220:3260 --logout" returned: 0 in 0.063s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:16 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[7b0dfab0-1b39-487f-b29f-55946831887a]: (4, ('Logging out of session [sid: 12, target: iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c, portal: 172.16.0.220,3260]\nLogout of [sid: 12, target: iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:16 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7642f0e-fd44-4b28-9769-a4b70af24cef tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 12, target: iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c, portal: 172.16.0.220,3260]
Aug 09 02:20:16 user nova-compute[72351]: Logout of [sid: 12, target: iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c, portal: 172.16.0.220,3260] successful.
Aug 09 02:20:16 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:20:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c -p 172.16.0.220:3260 --op delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0203626e-04e6-4768-b2c2-e7dbaeed4c6c -p 172.16.0.220:3260 --op delete" returned: 0 in 0.031s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:16 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[7fae4f70-a22c-4358-81f6-b02fe80c17b8]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:16 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7642f0e-fd44-4b28-9769-a4b70af24cef tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:20:16 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-d7642f0e-fd44-4b28-9769-a4b70af24cef tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.292s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:20:16 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7642f0e-fd44-4b28-9769-a4b70af24cef tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] <== disconnect_volume: return (293ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:20:16 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-d7642f0e-fd44-4b28-9769-a4b70af24cef tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 02:20:17 user nova-compute[72351]: DEBUG nova.compute.manager [req-0cd7f353-f5b6-4c4e-a7da-c8eb849289a3 req-c2407a25-eb8f-4aa6-b354-023a64da3a9b service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received event network-changed-e021d67e-09ab-4779-b639-dff1ba39798a {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:20:17 user nova-compute[72351]: DEBUG nova.compute.manager [req-0cd7f353-f5b6-4c4e-a7da-c8eb849289a3 req-c2407a25-eb8f-4aa6-b354-023a64da3a9b service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Refreshing instance network info cache due to event network-changed-e021d67e-09ab-4779-b639-dff1ba39798a. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:20:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-0cd7f353-f5b6-4c4e-a7da-c8eb849289a3 req-c2407a25-eb8f-4aa6-b354-023a64da3a9b service nova] Acquiring lock "refresh_cache-629e177e-afbb-4765-a0b5-7a3f66f72ebf" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:20:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-0cd7f353-f5b6-4c4e-a7da-c8eb849289a3 req-c2407a25-eb8f-4aa6-b354-023a64da3a9b service nova] Acquired lock "refresh_cache-629e177e-afbb-4765-a0b5-7a3f66f72ebf" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:20:17 user nova-compute[72351]: DEBUG nova.network.neutron [req-0cd7f353-f5b6-4c4e-a7da-c8eb849289a3 req-c2407a25-eb8f-4aa6-b354-023a64da3a9b service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Refreshing network info cache for port e021d67e-09ab-4779-b639-dff1ba39798a {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:20:17 user nova-compute[72351]: DEBUG nova.objects.instance [None req-d7642f0e-fd44-4b28-9769-a4b70af24cef tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Lazy-loading 'flavor' on Instance uuid 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:20:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-d7642f0e-fd44-4b28-9769-a4b70af24cef tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Lock "11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.665s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:20:17 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:18 user nova-compute[72351]: DEBUG nova.network.neutron [req-0cd7f353-f5b6-4c4e-a7da-c8eb849289a3 req-c2407a25-eb8f-4aa6-b354-023a64da3a9b service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Updated VIF entry in instance network info cache for port e021d67e-09ab-4779-b639-dff1ba39798a. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:20:18 user nova-compute[72351]: DEBUG nova.network.neutron [req-0cd7f353-f5b6-4c4e-a7da-c8eb849289a3 req-c2407a25-eb8f-4aa6-b354-023a64da3a9b service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Updating instance_info_cache with network_info: [{"id": "e021d67e-09ab-4779-b639-dff1ba39798a", "address": "fa:16:3e:60:29:2e", "network": {"id": "9d2c7895-3144-4087-8a69-40739a321574", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest-566420926-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.196", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape021d67e-09", "ovs_interfaceid": "e021d67e-09ab-4779-b639-dff1ba39798a", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "f5ecb594-7ac3-496b-9bb0-9f908b91c8fb", "address": "fa:16:3e:30:31:a4", "network": {"id": "4e0c5236-b30d-4213-b1b4-26c0f1a636eb", "bridge": "br-int", "label": "tempest-device-tagging-net1-1640959147", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.122", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf5ecb594-7a", "ovs_interfaceid": "f5ecb594-7ac3-496b-9bb0-9f908b91c8fb", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "2e9c8a9c-2d81-4b84-b481-9a6b513e730c", "address": "fa:16:3e:39:9b:9d", "network": {"id": "4e0c5236-b30d-4213-b1b4-26c0f1a636eb", "bridge": "br-int", "label": "tempest-device-tagging-net1-1640959147", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.75", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2e9c8a9c-2d", "ovs_interfaceid": "2e9c8a9c-2d81-4b84-b481-9a6b513e730c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "5d0bb297-60b1-4e66-8f43-3bf1c32ae4ef", "address": "fa:16:3e:9b:17:d2", "network": {"id": "4e0c5236-b30d-4213-b1b4-26c0f1a636eb", "bridge": "br-int", "label": "tempest-device-tagging-net1-1640959147", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.192", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5d0bb297-60", "ovs_interfaceid": "5d0bb297-60b1-4e66-8f43-3bf1c32ae4ef", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "c2df74bf-22ce-48b1-9ab8-3a684cef7266", "address": "fa:16:3e:3f:be:14", "network": {"id": "4e0c5236-b30d-4213-b1b4-26c0f1a636eb", "bridge": "br-int", "label": "tempest-device-tagging-net1-1640959147", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.128", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc2df74bf-22", "ovs_interfaceid": "c2df74bf-22ce-48b1-9ab8-3a684cef7266", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "d1b9314b-1800-41fb-a34b-0186f8e746da", "address": "fa:16:3e:37:27:3b", "network": {"id": "b4ebdbb0-e697-4a42-93f5-946522a3dbce", "bridge": "br-int", "label": "tempest-device-tagging-net2-356335140", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd1b9314b-18", "ovs_interfaceid": "d1b9314b-1800-41fb-a34b-0186f8e746da", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "c7c195c0-af76-445e-8a18-8169f0cdf5fc", "address": "fa:16:3e:f7:2d:31", "network": {"id": "b4ebdbb0-e697-4a42-93f5-946522a3dbce", "bridge": "br-int", "label": "tempest-device-tagging-net2-356335140", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc7c195c0-af", "ovs_interfaceid": "c7c195c0-af76-445e-8a18-8169f0cdf5fc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:20:18 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-0cd7f353-f5b6-4c4e-a7da-c8eb849289a3 req-c2407a25-eb8f-4aa6-b354-023a64da3a9b service nova] Releasing lock "refresh_cache-629e177e-afbb-4765-a0b5-7a3f66f72ebf" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:20:19 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:19 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Acquiring lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:20:19 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:20:19 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Acquiring lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:20:19 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:20:19 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:20:19 user nova-compute[72351]: INFO nova.compute.manager [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Terminating instance
Aug 09 02:20:19 user nova-compute[72351]: DEBUG nova.compute.manager [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Start destroying the instance on the hypervisor. {{(pid=72351) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3124}}
Aug 09 02:20:19 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:19 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:19 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:19 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:19 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:20 user nova-compute[72351]: DEBUG nova.compute.manager [req-ea879167-01a4-4eb4-892f-b60baeae7705 req-ef5e66eb-7159-4de6-9411-36f3a95c2a4b service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received event network-vif-unplugged-f5ecb594-7ac3-496b-9bb0-9f908b91c8fb {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:20:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ea879167-01a4-4eb4-892f-b60baeae7705 req-ef5e66eb-7159-4de6-9411-36f3a95c2a4b service nova] Acquiring lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:20:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ea879167-01a4-4eb4-892f-b60baeae7705 req-ef5e66eb-7159-4de6-9411-36f3a95c2a4b service nova] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:20:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ea879167-01a4-4eb4-892f-b60baeae7705 req-ef5e66eb-7159-4de6-9411-36f3a95c2a4b service nova] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:20:20 user nova-compute[72351]: DEBUG nova.compute.manager [req-ea879167-01a4-4eb4-892f-b60baeae7705 req-ef5e66eb-7159-4de6-9411-36f3a95c2a4b service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] No waiting events found dispatching network-vif-unplugged-f5ecb594-7ac3-496b-9bb0-9f908b91c8fb {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:20:20 user nova-compute[72351]: DEBUG nova.compute.manager [req-ea879167-01a4-4eb4-892f-b60baeae7705 req-ef5e66eb-7159-4de6-9411-36f3a95c2a4b service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received event network-vif-unplugged-f5ecb594-7ac3-496b-9bb0-9f908b91c8fb for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:20:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:20 user nova-compute[72351]: DEBUG nova.compute.manager [req-eb88cf00-cc46-4f0d-9609-3dc972f53e61 req-0b6ef0c1-2373-47e8-947a-c58df6a574e5 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received event network-vif-unplugged-c2df74bf-22ce-48b1-9ab8-3a684cef7266 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:20:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-eb88cf00-cc46-4f0d-9609-3dc972f53e61 req-0b6ef0c1-2373-47e8-947a-c58df6a574e5 service nova] Acquiring lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:20:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-eb88cf00-cc46-4f0d-9609-3dc972f53e61 req-0b6ef0c1-2373-47e8-947a-c58df6a574e5 service nova] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:20:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-eb88cf00-cc46-4f0d-9609-3dc972f53e61 req-0b6ef0c1-2373-47e8-947a-c58df6a574e5 service nova] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:20:20 user nova-compute[72351]: DEBUG nova.compute.manager [req-eb88cf00-cc46-4f0d-9609-3dc972f53e61 req-0b6ef0c1-2373-47e8-947a-c58df6a574e5 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] No waiting events found dispatching network-vif-unplugged-c2df74bf-22ce-48b1-9ab8-3a684cef7266 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:20:20 user nova-compute[72351]: DEBUG nova.compute.manager [req-eb88cf00-cc46-4f0d-9609-3dc972f53e61 req-0b6ef0c1-2373-47e8-947a-c58df6a574e5 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received event network-vif-unplugged-c2df74bf-22ce-48b1-9ab8-3a684cef7266 for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:20:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:20 user nova-compute[72351]: DEBUG nova.compute.manager [req-e3c23258-a7ba-4414-bdb2-033e363e23f8 req-fedd37c1-3323-4940-ab32-6e78d9b8b1c3 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received event network-vif-unplugged-e021d67e-09ab-4779-b639-dff1ba39798a {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:20:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-e3c23258-a7ba-4414-bdb2-033e363e23f8 req-fedd37c1-3323-4940-ab32-6e78d9b8b1c3 service nova] Acquiring lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:20:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-e3c23258-a7ba-4414-bdb2-033e363e23f8 req-fedd37c1-3323-4940-ab32-6e78d9b8b1c3 service nova] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:20:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-e3c23258-a7ba-4414-bdb2-033e363e23f8 req-fedd37c1-3323-4940-ab32-6e78d9b8b1c3 service nova] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:20:20 user nova-compute[72351]: DEBUG nova.compute.manager [req-e3c23258-a7ba-4414-bdb2-033e363e23f8 req-fedd37c1-3323-4940-ab32-6e78d9b8b1c3 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] No waiting events found dispatching network-vif-unplugged-e021d67e-09ab-4779-b639-dff1ba39798a {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:20:20 user nova-compute[72351]: DEBUG nova.compute.manager [req-e3c23258-a7ba-4414-bdb2-033e363e23f8 req-fedd37c1-3323-4940-ab32-6e78d9b8b1c3 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received event network-vif-unplugged-e021d67e-09ab-4779-b639-dff1ba39798a for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:20:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:21 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Instance destroyed successfully.
Aug 09 02:20:21 user nova-compute[72351]: DEBUG nova.objects.instance [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Lazy-loading 'resources' on Instance uuid 629e177e-afbb-4765-a0b5-7a3f66f72ebf {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:20:21 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-08-09T02:18:24Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-180184049',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-180184049',id=9,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNlyDeWP+c+bzipwXZAMO2Z2+8MHps3hDZ0m7Lqwq/td4x0E+p/3ch4cYsc0Ot8OOMGBR+QCJGss0jSgkeH83ijl4d9uevYtC5oVznEoGx5YCooCmb7aiHDwnwmo8gkNlA==',key_name='tempest-keypair-1051869033',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:19:14Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='6908d784cc1a41dbbb4bee47e70c259b',ramdisk_id='',reservation_id='r-rl5gtt0e',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest-1303879303',owner_user_name='tempest-TaggedBootDevicesTest-1303879303-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:19:15Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='c74c3939709643c8915c7d1678935e9a',uuid=629e177e-afbb-4765-a0b5-7a3f66f72ebf,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "e021d67e-09ab-4779-b639-dff1ba39798a", "address": "fa:16:3e:60:29:2e", "network": {"id": "9d2c7895-3144-4087-8a69-40739a321574", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest-566420926-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.196", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape021d67e-09", "ovs_interfaceid": "e021d67e-09ab-4779-b639-dff1ba39798a", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:20:21 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Converting VIF {"id": "e021d67e-09ab-4779-b639-dff1ba39798a", "address": "fa:16:3e:60:29:2e", "network": {"id": "9d2c7895-3144-4087-8a69-40739a321574", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest-566420926-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.196", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape021d67e-09", "ovs_interfaceid": "e021d67e-09ab-4779-b639-dff1ba39798a", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:20:21 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:60:29:2e,bridge_name='br-int',has_traffic_filtering=True,id=e021d67e-09ab-4779-b639-dff1ba39798a,network=Network(9d2c7895-3144-4087-8a69-40739a321574),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape021d67e-09') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:20:21 user nova-compute[72351]: DEBUG os_vif [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:60:29:2e,bridge_name='br-int',has_traffic_filtering=True,id=e021d67e-09ab-4779-b639-dff1ba39798a,network=Network(9d2c7895-3144-4087-8a69-40739a321574),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape021d67e-09') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:20:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tape021d67e-09, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:20:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:20:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:21 user nova-compute[72351]: INFO os_vif [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:60:29:2e,bridge_name='br-int',has_traffic_filtering=True,id=e021d67e-09ab-4779-b639-dff1ba39798a,network=Network(9d2c7895-3144-4087-8a69-40739a321574),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape021d67e-09')
Aug 09 02:20:21 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-08-09T02:18:24Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-180184049',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-180184049',id=9,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNlyDeWP+c+bzipwXZAMO2Z2+8MHps3hDZ0m7Lqwq/td4x0E+p/3ch4cYsc0Ot8OOMGBR+QCJGss0jSgkeH83ijl4d9uevYtC5oVznEoGx5YCooCmb7aiHDwnwmo8gkNlA==',key_name='tempest-keypair-1051869033',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:19:14Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='6908d784cc1a41dbbb4bee47e70c259b',ramdisk_id='',reservation_id='r-rl5gtt0e',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest-1303879303',owner_user_name='tempest-TaggedBootDevicesTest-1303879303-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:19:15Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='c74c3939709643c8915c7d1678935e9a',uuid=629e177e-afbb-4765-a0b5-7a3f66f72ebf,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "f5ecb594-7ac3-496b-9bb0-9f908b91c8fb", "address": "fa:16:3e:30:31:a4", "network": {"id": "4e0c5236-b30d-4213-b1b4-26c0f1a636eb", "bridge": "br-int", "label": "tempest-device-tagging-net1-1640959147", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.122", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf5ecb594-7a", "ovs_interfaceid": "f5ecb594-7ac3-496b-9bb0-9f908b91c8fb", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:20:21 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Converting VIF {"id": "f5ecb594-7ac3-496b-9bb0-9f908b91c8fb", "address": "fa:16:3e:30:31:a4", "network": {"id": "4e0c5236-b30d-4213-b1b4-26c0f1a636eb", "bridge": "br-int", "label": "tempest-device-tagging-net1-1640959147", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.122", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf5ecb594-7a", "ovs_interfaceid": "f5ecb594-7ac3-496b-9bb0-9f908b91c8fb", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:20:21 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:30:31:a4,bridge_name='br-int',has_traffic_filtering=True,id=f5ecb594-7ac3-496b-9bb0-9f908b91c8fb,network=Network(4e0c5236-b30d-4213-b1b4-26c0f1a636eb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tapf5ecb594-7a') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:20:21 user nova-compute[72351]: DEBUG os_vif [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:30:31:a4,bridge_name='br-int',has_traffic_filtering=True,id=f5ecb594-7ac3-496b-9bb0-9f908b91c8fb,network=Network(4e0c5236-b30d-4213-b1b4-26c0f1a636eb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tapf5ecb594-7a') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:20:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapf5ecb594-7a, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:20:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:20:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:21 user nova-compute[72351]: INFO os_vif [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:30:31:a4,bridge_name='br-int',has_traffic_filtering=True,id=f5ecb594-7ac3-496b-9bb0-9f908b91c8fb,network=Network(4e0c5236-b30d-4213-b1b4-26c0f1a636eb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tapf5ecb594-7a')
Aug 09 02:20:21 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-08-09T02:18:24Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-180184049',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-180184049',id=9,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNlyDeWP+c+bzipwXZAMO2Z2+8MHps3hDZ0m7Lqwq/td4x0E+p/3ch4cYsc0Ot8OOMGBR+QCJGss0jSgkeH83ijl4d9uevYtC5oVznEoGx5YCooCmb7aiHDwnwmo8gkNlA==',key_name='tempest-keypair-1051869033',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:19:14Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='6908d784cc1a41dbbb4bee47e70c259b',ramdisk_id='',reservation_id='r-rl5gtt0e',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest-1303879303',owner_user_name='tempest-TaggedBootDevicesTest-1303879303-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:19:15Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='c74c3939709643c8915c7d1678935e9a',uuid=629e177e-afbb-4765-a0b5-7a3f66f72ebf,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "2e9c8a9c-2d81-4b84-b481-9a6b513e730c", "address": "fa:16:3e:39:9b:9d", "network": {"id": "4e0c5236-b30d-4213-b1b4-26c0f1a636eb", "bridge": "br-int", "label": "tempest-device-tagging-net1-1640959147", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.75", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2e9c8a9c-2d", "ovs_interfaceid": "2e9c8a9c-2d81-4b84-b481-9a6b513e730c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:20:21 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Converting VIF {"id": "2e9c8a9c-2d81-4b84-b481-9a6b513e730c", "address": "fa:16:3e:39:9b:9d", "network": {"id": "4e0c5236-b30d-4213-b1b4-26c0f1a636eb", "bridge": "br-int", "label": "tempest-device-tagging-net1-1640959147", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.75", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2e9c8a9c-2d", "ovs_interfaceid": "2e9c8a9c-2d81-4b84-b481-9a6b513e730c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:20:21 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:39:9b:9d,bridge_name='br-int',has_traffic_filtering=True,id=2e9c8a9c-2d81-4b84-b481-9a6b513e730c,network=Network(4e0c5236-b30d-4213-b1b4-26c0f1a636eb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap2e9c8a9c-2d') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:20:21 user nova-compute[72351]: DEBUG os_vif [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:39:9b:9d,bridge_name='br-int',has_traffic_filtering=True,id=2e9c8a9c-2d81-4b84-b481-9a6b513e730c,network=Network(4e0c5236-b30d-4213-b1b4-26c0f1a636eb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap2e9c8a9c-2d') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:20:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap2e9c8a9c-2d, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:20:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:20:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:21 user nova-compute[72351]: INFO os_vif [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:39:9b:9d,bridge_name='br-int',has_traffic_filtering=True,id=2e9c8a9c-2d81-4b84-b481-9a6b513e730c,network=Network(4e0c5236-b30d-4213-b1b4-26c0f1a636eb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap2e9c8a9c-2d')
Aug 09 02:20:21 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-08-09T02:18:24Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-180184049',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-180184049',id=9,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNlyDeWP+c+bzipwXZAMO2Z2+8MHps3hDZ0m7Lqwq/td4x0E+p/3ch4cYsc0Ot8OOMGBR+QCJGss0jSgkeH83ijl4d9uevYtC5oVznEoGx5YCooCmb7aiHDwnwmo8gkNlA==',key_name='tempest-keypair-1051869033',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:19:14Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='6908d784cc1a41dbbb4bee47e70c259b',ramdisk_id='',reservation_id='r-rl5gtt0e',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest-1303879303',owner_user_name='tempest-TaggedBootDevicesTest-1303879303-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:19:15Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='c74c3939709643c8915c7d1678935e9a',uuid=629e177e-afbb-4765-a0b5-7a3f66f72ebf,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "5d0bb297-60b1-4e66-8f43-3bf1c32ae4ef", "address": "fa:16:3e:9b:17:d2", "network": {"id": "4e0c5236-b30d-4213-b1b4-26c0f1a636eb", "bridge": "br-int", "label": "tempest-device-tagging-net1-1640959147", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.192", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5d0bb297-60", "ovs_interfaceid": "5d0bb297-60b1-4e66-8f43-3bf1c32ae4ef", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:20:21 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Converting VIF {"id": "5d0bb297-60b1-4e66-8f43-3bf1c32ae4ef", "address": "fa:16:3e:9b:17:d2", "network": {"id": "4e0c5236-b30d-4213-b1b4-26c0f1a636eb", "bridge": "br-int", "label": "tempest-device-tagging-net1-1640959147", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.192", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5d0bb297-60", "ovs_interfaceid": "5d0bb297-60b1-4e66-8f43-3bf1c32ae4ef", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:20:21 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:9b:17:d2,bridge_name='br-int',has_traffic_filtering=True,id=5d0bb297-60b1-4e66-8f43-3bf1c32ae4ef,network=Network(4e0c5236-b30d-4213-b1b4-26c0f1a636eb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap5d0bb297-60') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:20:21 user nova-compute[72351]: DEBUG os_vif [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:9b:17:d2,bridge_name='br-int',has_traffic_filtering=True,id=5d0bb297-60b1-4e66-8f43-3bf1c32ae4ef,network=Network(4e0c5236-b30d-4213-b1b4-26c0f1a636eb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap5d0bb297-60') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:20:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap5d0bb297-60, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:20:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:20:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:21 user nova-compute[72351]: INFO os_vif [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:9b:17:d2,bridge_name='br-int',has_traffic_filtering=True,id=5d0bb297-60b1-4e66-8f43-3bf1c32ae4ef,network=Network(4e0c5236-b30d-4213-b1b4-26c0f1a636eb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap5d0bb297-60')
Aug 09 02:20:21 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-08-09T02:18:24Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-180184049',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-180184049',id=9,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNlyDeWP+c+bzipwXZAMO2Z2+8MHps3hDZ0m7Lqwq/td4x0E+p/3ch4cYsc0Ot8OOMGBR+QCJGss0jSgkeH83ijl4d9uevYtC5oVznEoGx5YCooCmb7aiHDwnwmo8gkNlA==',key_name='tempest-keypair-1051869033',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:19:14Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='6908d784cc1a41dbbb4bee47e70c259b',ramdisk_id='',reservation_id='r-rl5gtt0e',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest-1303879303',owner_user_name='tempest-TaggedBootDevicesTest-1303879303-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:19:15Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='c74c3939709643c8915c7d1678935e9a',uuid=629e177e-afbb-4765-a0b5-7a3f66f72ebf,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "c2df74bf-22ce-48b1-9ab8-3a684cef7266", "address": "fa:16:3e:3f:be:14", "network": {"id": "4e0c5236-b30d-4213-b1b4-26c0f1a636eb", "bridge": "br-int", "label": "tempest-device-tagging-net1-1640959147", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.128", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc2df74bf-22", "ovs_interfaceid": "c2df74bf-22ce-48b1-9ab8-3a684cef7266", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:20:21 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Converting VIF {"id": "c2df74bf-22ce-48b1-9ab8-3a684cef7266", "address": "fa:16:3e:3f:be:14", "network": {"id": "4e0c5236-b30d-4213-b1b4-26c0f1a636eb", "bridge": "br-int", "label": "tempest-device-tagging-net1-1640959147", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.128", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc2df74bf-22", "ovs_interfaceid": "c2df74bf-22ce-48b1-9ab8-3a684cef7266", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:20:21 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:3f:be:14,bridge_name='br-int',has_traffic_filtering=True,id=c2df74bf-22ce-48b1-9ab8-3a684cef7266,network=Network(4e0c5236-b30d-4213-b1b4-26c0f1a636eb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc2df74bf-22') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:20:21 user nova-compute[72351]: DEBUG os_vif [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:3f:be:14,bridge_name='br-int',has_traffic_filtering=True,id=c2df74bf-22ce-48b1-9ab8-3a684cef7266,network=Network(4e0c5236-b30d-4213-b1b4-26c0f1a636eb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc2df74bf-22') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:20:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapc2df74bf-22, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:20:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:22 user nova-compute[72351]: INFO os_vif [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:3f:be:14,bridge_name='br-int',has_traffic_filtering=True,id=c2df74bf-22ce-48b1-9ab8-3a684cef7266,network=Network(4e0c5236-b30d-4213-b1b4-26c0f1a636eb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc2df74bf-22')
Aug 09 02:20:22 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-08-09T02:18:24Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-180184049',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-180184049',id=9,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNlyDeWP+c+bzipwXZAMO2Z2+8MHps3hDZ0m7Lqwq/td4x0E+p/3ch4cYsc0Ot8OOMGBR+QCJGss0jSgkeH83ijl4d9uevYtC5oVznEoGx5YCooCmb7aiHDwnwmo8gkNlA==',key_name='tempest-keypair-1051869033',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:19:14Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='6908d784cc1a41dbbb4bee47e70c259b',ramdisk_id='',reservation_id='r-rl5gtt0e',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest-1303879303',owner_user_name='tempest-TaggedBootDevicesTest-1303879303-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:19:15Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='c74c3939709643c8915c7d1678935e9a',uuid=629e177e-afbb-4765-a0b5-7a3f66f72ebf,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "d1b9314b-1800-41fb-a34b-0186f8e746da", "address": "fa:16:3e:37:27:3b", "network": {"id": "b4ebdbb0-e697-4a42-93f5-946522a3dbce", "bridge": "br-int", "label": "tempest-device-tagging-net2-356335140", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd1b9314b-18", "ovs_interfaceid": "d1b9314b-1800-41fb-a34b-0186f8e746da", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Converting VIF {"id": "d1b9314b-1800-41fb-a34b-0186f8e746da", "address": "fa:16:3e:37:27:3b", "network": {"id": "b4ebdbb0-e697-4a42-93f5-946522a3dbce", "bridge": "br-int", "label": "tempest-device-tagging-net2-356335140", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd1b9314b-18", "ovs_interfaceid": "d1b9314b-1800-41fb-a34b-0186f8e746da", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:37:27:3b,bridge_name='br-int',has_traffic_filtering=True,id=d1b9314b-1800-41fb-a34b-0186f8e746da,network=Network(b4ebdbb0-e697-4a42-93f5-946522a3dbce),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd1b9314b-18') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG os_vif [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:37:27:3b,bridge_name='br-int',has_traffic_filtering=True,id=d1b9314b-1800-41fb-a34b-0186f8e746da,network=Network(b4ebdbb0-e697-4a42-93f5-946522a3dbce),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd1b9314b-18') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapd1b9314b-18, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:22 user nova-compute[72351]: INFO os_vif [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:37:27:3b,bridge_name='br-int',has_traffic_filtering=True,id=d1b9314b-1800-41fb-a34b-0186f8e746da,network=Network(b4ebdbb0-e697-4a42-93f5-946522a3dbce),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd1b9314b-18')
Aug 09 02:20:22 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-08-09T02:18:24Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-180184049',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-180184049',id=9,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNlyDeWP+c+bzipwXZAMO2Z2+8MHps3hDZ0m7Lqwq/td4x0E+p/3ch4cYsc0Ot8OOMGBR+QCJGss0jSgkeH83ijl4d9uevYtC5oVznEoGx5YCooCmb7aiHDwnwmo8gkNlA==',key_name='tempest-keypair-1051869033',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:19:14Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='6908d784cc1a41dbbb4bee47e70c259b',ramdisk_id='',reservation_id='r-rl5gtt0e',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest-1303879303',owner_user_name='tempest-TaggedBootDevicesTest-1303879303-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:19:15Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='c74c3939709643c8915c7d1678935e9a',uuid=629e177e-afbb-4765-a0b5-7a3f66f72ebf,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "c7c195c0-af76-445e-8a18-8169f0cdf5fc", "address": "fa:16:3e:f7:2d:31", "network": {"id": "b4ebdbb0-e697-4a42-93f5-946522a3dbce", "bridge": "br-int", "label": "tempest-device-tagging-net2-356335140", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc7c195c0-af", "ovs_interfaceid": "c7c195c0-af76-445e-8a18-8169f0cdf5fc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Converting VIF {"id": "c7c195c0-af76-445e-8a18-8169f0cdf5fc", "address": "fa:16:3e:f7:2d:31", "network": {"id": "b4ebdbb0-e697-4a42-93f5-946522a3dbce", "bridge": "br-int", "label": "tempest-device-tagging-net2-356335140", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc7c195c0-af", "ovs_interfaceid": "c7c195c0-af76-445e-8a18-8169f0cdf5fc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:f7:2d:31,bridge_name='br-int',has_traffic_filtering=True,id=c7c195c0-af76-445e-8a18-8169f0cdf5fc,network=Network(b4ebdbb0-e697-4a42-93f5-946522a3dbce),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc7c195c0-af') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG os_vif [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:f7:2d:31,bridge_name='br-int',has_traffic_filtering=True,id=c7c195c0-af76-445e-8a18-8169f0cdf5fc,network=Network(b4ebdbb0-e697-4a42-93f5-946522a3dbce),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc7c195c0-af') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapc7c195c0-af, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:20:22 user nova-compute[72351]: INFO os_vif [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:f7:2d:31,bridge_name='br-int',has_traffic_filtering=True,id=c7c195c0-af76-445e-8a18-8169f0cdf5fc,network=Network(b4ebdbb0-e697-4a42-93f5-946522a3dbce),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc7c195c0-af')
Aug 09 02:20:22 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'nLZUTgbV', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdf'}, None), 'kwargs': True}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:20:22 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:20:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a4d1945d-92fd-48af-aeac-01e2874e77d8]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.027s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[454fab43-375f-4019-93fa-047b6c713abd]: (4, ('tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f (non-flash)\ntcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f (non-flash)
Aug 09 02:20:22 user nova-compute[72351]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:20:22 user nova-compute[72351]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)
Aug 09 02:20:22 user nova-compute[72351]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:20:22 user nova-compute[72351]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)
Aug 09 02:20:22 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] iscsi session list stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f (non-flash)
Aug 09 02:20:22 user nova-compute[72351]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:20:22 user nova-compute[72351]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)
Aug 09 02:20:22 user nova-compute[72351]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:20:22 user nova-compute[72351]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c (non-flash)
Aug 09 02:20:22 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fcfdc191750>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c'): ({'sdf'}, set())}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Removing single pathed devices sdf {{(pid=72351) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[af0e6ffc-c357-4f2e-a41a-9df653cbef16]: (4, ('path checker states:\nup                  4\npending             1\n\npaths: 0\nbusy: False\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdf {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdf" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[90b85be5-67ae-4ad5-ba75-c4a42f0488c5]: (4, ('ok\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Flushing IO for device /dev/sdf {{(pid=72351) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdf {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdf" returned: 0 in 0.013s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[647acce3-d8e8-4879-b5ad-609369b34814]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Remove SCSI device /dev/sdf with /sys/block/sdf/device/delete {{(pid=72351) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdf/device/delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdf/device/delete" returned: 0 in 0.043s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[359a9610-2061-4a59-96cd-70f0cc755f0f]: (4, ('1', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Checking to see if SCSI volumes sdf have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] SCSI volumes sdf have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c')] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d2524cd7-cd1e-4059-afd7-2a1cac97120c]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c -p 172.16.0.220:3260 --logout {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c -p 172.16.0.220:3260 --logout" returned: 0 in 0.064s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[dc88b29f-e6d3-42df-b1cf-0b50137093a9]: (4, ('Logging out of session [sid: 6, target: iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c, portal: 172.16.0.220,3260]\nLogout of [sid: 6, target: iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 6, target: iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c, portal: 172.16.0.220,3260]
Aug 09 02:20:22 user nova-compute[72351]: Logout of [sid: 6, target: iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c, portal: 172.16.0.220,3260] successful.
Aug 09 02:20:22 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c -p 172.16.0.220:3260 --op delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a3218523-2c3f-4787-b0ea-f589160d940c -p 172.16.0.220:3260 --op delete" returned: 0 in 0.031s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a5176294-8c6e-41cb-bc42-d5fd374074c0]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.312s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] <== disconnect_volume: return (314ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'FftGaAqL', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb'}, None), 'kwargs': True}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:20:22 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:20:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[9ecb495d-e168-45b6-af36-1a7cfdd47cfc]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG nova.compute.manager [req-ad343ad0-1f2b-4d28-8fe9-e79d229b9bfb req-71019311-0ddd-4148-b097-c71ad7f1ee7c service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received event network-vif-plugged-f5ecb594-7ac3-496b-9bb0-9f908b91c8fb {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ad343ad0-1f2b-4d28-8fe9-e79d229b9bfb req-71019311-0ddd-4148-b097-c71ad7f1ee7c service nova] Acquiring lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ad343ad0-1f2b-4d28-8fe9-e79d229b9bfb req-71019311-0ddd-4148-b097-c71ad7f1ee7c service nova] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ad343ad0-1f2b-4d28-8fe9-e79d229b9bfb req-71019311-0ddd-4148-b097-c71ad7f1ee7c service nova] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG nova.compute.manager [req-ad343ad0-1f2b-4d28-8fe9-e79d229b9bfb req-71019311-0ddd-4148-b097-c71ad7f1ee7c service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] No waiting events found dispatching network-vif-plugged-f5ecb594-7ac3-496b-9bb0-9f908b91c8fb {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:20:22 user nova-compute[72351]: WARNING nova.compute.manager [req-ad343ad0-1f2b-4d28-8fe9-e79d229b9bfb req-71019311-0ddd-4148-b097-c71ad7f1ee7c service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received unexpected event network-vif-plugged-f5ecb594-7ac3-496b-9bb0-9f908b91c8fb for instance with vm_state active and task_state deleting.
Aug 09 02:20:22 user nova-compute[72351]: DEBUG nova.compute.manager [req-ad343ad0-1f2b-4d28-8fe9-e79d229b9bfb req-71019311-0ddd-4148-b097-c71ad7f1ee7c service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received event network-vif-unplugged-d1b9314b-1800-41fb-a34b-0186f8e746da {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ad343ad0-1f2b-4d28-8fe9-e79d229b9bfb req-71019311-0ddd-4148-b097-c71ad7f1ee7c service nova] Acquiring lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ad343ad0-1f2b-4d28-8fe9-e79d229b9bfb req-71019311-0ddd-4148-b097-c71ad7f1ee7c service nova] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ad343ad0-1f2b-4d28-8fe9-e79d229b9bfb req-71019311-0ddd-4148-b097-c71ad7f1ee7c service nova] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG nova.compute.manager [req-ad343ad0-1f2b-4d28-8fe9-e79d229b9bfb req-71019311-0ddd-4148-b097-c71ad7f1ee7c service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] No waiting events found dispatching network-vif-unplugged-d1b9314b-1800-41fb-a34b-0186f8e746da {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG nova.compute.manager [req-ad343ad0-1f2b-4d28-8fe9-e79d229b9bfb req-71019311-0ddd-4148-b097-c71ad7f1ee7c service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received event network-vif-unplugged-d1b9314b-1800-41fb-a34b-0186f8e746da for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG nova.compute.manager [req-ad343ad0-1f2b-4d28-8fe9-e79d229b9bfb req-71019311-0ddd-4148-b097-c71ad7f1ee7c service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received event network-vif-plugged-d1b9314b-1800-41fb-a34b-0186f8e746da {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ad343ad0-1f2b-4d28-8fe9-e79d229b9bfb req-71019311-0ddd-4148-b097-c71ad7f1ee7c service nova] Acquiring lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ad343ad0-1f2b-4d28-8fe9-e79d229b9bfb req-71019311-0ddd-4148-b097-c71ad7f1ee7c service nova] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ad343ad0-1f2b-4d28-8fe9-e79d229b9bfb req-71019311-0ddd-4148-b097-c71ad7f1ee7c service nova] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG nova.compute.manager [req-ad343ad0-1f2b-4d28-8fe9-e79d229b9bfb req-71019311-0ddd-4148-b097-c71ad7f1ee7c service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] No waiting events found dispatching network-vif-plugged-d1b9314b-1800-41fb-a34b-0186f8e746da {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:20:22 user nova-compute[72351]: WARNING nova.compute.manager [req-ad343ad0-1f2b-4d28-8fe9-e79d229b9bfb req-71019311-0ddd-4148-b097-c71ad7f1ee7c service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received unexpected event network-vif-plugged-d1b9314b-1800-41fb-a34b-0186f8e746da for instance with vm_state active and task_state deleting.
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.028s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[68c18c24-fb17-4a05-b213-f58642f3efca]: (4, ('tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f (non-flash)\ntcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f (non-flash)
Aug 09 02:20:22 user nova-compute[72351]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:20:22 user nova-compute[72351]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)
Aug 09 02:20:22 user nova-compute[72351]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:20:22 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] iscsi session list stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f (non-flash)
Aug 09 02:20:22 user nova-compute[72351]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 (non-flash)
Aug 09 02:20:22 user nova-compute[72351]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)
Aug 09 02:20:22 user nova-compute[72351]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:20:22 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fcfdc192f80>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97'): ({'sdb'}, set())}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Removing single pathed devices sdb {{(pid=72351) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a983ddc3-6457-49b6-84a9-86acbd8b308d]: (4, ('path checker states:\nup                  3\npending             1\n\npaths: 0\nbusy: False\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[2b6755a1-d7f3-4708-bd2b-a7037c2b4d6b]: (4, ('ok\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Flushing IO for device /dev/sdb {{(pid=72351) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.013s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[9e2981a5-af9d-414b-92ee-a0c14aea992a]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=72351) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.051s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[5b037749-8250-4b58-98f5-a5942d4c674f]: (4, ('1', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] SCSI volumes sdb have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97')] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[65fa4990-2b49-40a0-b9a9-eccd42bb61b4]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 -p 172.16.0.220:3260 --logout {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 -p 172.16.0.220:3260 --logout" returned: 0 in 0.060s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[15182ced-a51f-47fd-ba7b-851a6edd32c8]: (4, ('Logging out of session [sid: 2, target: iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97, portal: 172.16.0.220,3260]\nLogout of [sid: 2, target: iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 2, target: iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97, portal: 172.16.0.220,3260]
Aug 09 02:20:22 user nova-compute[72351]: Logout of [sid: 2, target: iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97, portal: 172.16.0.220,3260] successful.
Aug 09 02:20:22 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 -p 172.16.0.220:3260 --op delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a8ce86ce-79f7-4138-a49b-c45c9db38c97 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.027s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ee86892f-f120-4c18-aa79-e1348131fc40]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.307s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] <== disconnect_volume: return (309ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'pynsiAWj', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdd'}, None), 'kwargs': True}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:20:22 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:20:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.003s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG nova.compute.manager [req-f4026e87-fe0c-4572-95b9-7d0156888a54 req-c8d6bdcf-e475-4c70-8784-a320ad89df0c service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received event network-vif-plugged-c2df74bf-22ce-48b1-9ab8-3a684cef7266 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-f4026e87-fe0c-4572-95b9-7d0156888a54 req-c8d6bdcf-e475-4c70-8784-a320ad89df0c service nova] Acquiring lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-f4026e87-fe0c-4572-95b9-7d0156888a54 req-c8d6bdcf-e475-4c70-8784-a320ad89df0c service nova] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-f4026e87-fe0c-4572-95b9-7d0156888a54 req-c8d6bdcf-e475-4c70-8784-a320ad89df0c service nova] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG nova.compute.manager [req-f4026e87-fe0c-4572-95b9-7d0156888a54 req-c8d6bdcf-e475-4c70-8784-a320ad89df0c service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] No waiting events found dispatching network-vif-plugged-c2df74bf-22ce-48b1-9ab8-3a684cef7266 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:20:22 user nova-compute[72351]: WARNING nova.compute.manager [req-f4026e87-fe0c-4572-95b9-7d0156888a54 req-c8d6bdcf-e475-4c70-8784-a320ad89df0c service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received unexpected event network-vif-plugged-c2df74bf-22ce-48b1-9ab8-3a684cef7266 for instance with vm_state active and task_state deleting.
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[46eb6ea9-db88-4fb3-8622-94de680bd212]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG nova.compute.manager [req-9a167c57-6389-4ece-9558-f70d8189c4fa req-705e43eb-5764-4f05-9ebb-0f64eb04d95a service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received event network-vif-plugged-e021d67e-09ab-4779-b639-dff1ba39798a {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-9a167c57-6389-4ece-9558-f70d8189c4fa req-705e43eb-5764-4f05-9ebb-0f64eb04d95a service nova] Acquiring lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-9a167c57-6389-4ece-9558-f70d8189c4fa req-705e43eb-5764-4f05-9ebb-0f64eb04d95a service nova] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-9a167c57-6389-4ece-9558-f70d8189c4fa req-705e43eb-5764-4f05-9ebb-0f64eb04d95a service nova] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG nova.compute.manager [req-9a167c57-6389-4ece-9558-f70d8189c4fa req-705e43eb-5764-4f05-9ebb-0f64eb04d95a service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] No waiting events found dispatching network-vif-plugged-e021d67e-09ab-4779-b639-dff1ba39798a {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:20:22 user nova-compute[72351]: WARNING nova.compute.manager [req-9a167c57-6389-4ece-9558-f70d8189c4fa req-705e43eb-5764-4f05-9ebb-0f64eb04d95a service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received unexpected event network-vif-plugged-e021d67e-09ab-4779-b639-dff1ba39798a for instance with vm_state active and task_state deleting.
Aug 09 02:20:22 user nova-compute[72351]: DEBUG nova.compute.manager [req-9a167c57-6389-4ece-9558-f70d8189c4fa req-705e43eb-5764-4f05-9ebb-0f64eb04d95a service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received event network-vif-unplugged-2e9c8a9c-2d81-4b84-b481-9a6b513e730c {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-9a167c57-6389-4ece-9558-f70d8189c4fa req-705e43eb-5764-4f05-9ebb-0f64eb04d95a service nova] Acquiring lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-9a167c57-6389-4ece-9558-f70d8189c4fa req-705e43eb-5764-4f05-9ebb-0f64eb04d95a service nova] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-9a167c57-6389-4ece-9558-f70d8189c4fa req-705e43eb-5764-4f05-9ebb-0f64eb04d95a service nova] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG nova.compute.manager [req-9a167c57-6389-4ece-9558-f70d8189c4fa req-705e43eb-5764-4f05-9ebb-0f64eb04d95a service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] No waiting events found dispatching network-vif-unplugged-2e9c8a9c-2d81-4b84-b481-9a6b513e730c {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG nova.compute.manager [req-9a167c57-6389-4ece-9558-f70d8189c4fa req-705e43eb-5764-4f05-9ebb-0f64eb04d95a service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received event network-vif-unplugged-2e9c8a9c-2d81-4b84-b481-9a6b513e730c for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG nova.compute.manager [req-9a167c57-6389-4ece-9558-f70d8189c4fa req-705e43eb-5764-4f05-9ebb-0f64eb04d95a service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received event network-vif-plugged-2e9c8a9c-2d81-4b84-b481-9a6b513e730c {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-9a167c57-6389-4ece-9558-f70d8189c4fa req-705e43eb-5764-4f05-9ebb-0f64eb04d95a service nova] Acquiring lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-9a167c57-6389-4ece-9558-f70d8189c4fa req-705e43eb-5764-4f05-9ebb-0f64eb04d95a service nova] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-9a167c57-6389-4ece-9558-f70d8189c4fa req-705e43eb-5764-4f05-9ebb-0f64eb04d95a service nova] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG nova.compute.manager [req-9a167c57-6389-4ece-9558-f70d8189c4fa req-705e43eb-5764-4f05-9ebb-0f64eb04d95a service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] No waiting events found dispatching network-vif-plugged-2e9c8a9c-2d81-4b84-b481-9a6b513e730c {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:20:22 user nova-compute[72351]: WARNING nova.compute.manager [req-9a167c57-6389-4ece-9558-f70d8189c4fa req-705e43eb-5764-4f05-9ebb-0f64eb04d95a service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received unexpected event network-vif-plugged-2e9c8a9c-2d81-4b84-b481-9a6b513e730c for instance with vm_state active and task_state deleting.
Aug 09 02:20:22 user nova-compute[72351]: DEBUG nova.compute.manager [req-9a167c57-6389-4ece-9558-f70d8189c4fa req-705e43eb-5764-4f05-9ebb-0f64eb04d95a service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received event network-vif-unplugged-5d0bb297-60b1-4e66-8f43-3bf1c32ae4ef {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-9a167c57-6389-4ece-9558-f70d8189c4fa req-705e43eb-5764-4f05-9ebb-0f64eb04d95a service nova] Acquiring lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-9a167c57-6389-4ece-9558-f70d8189c4fa req-705e43eb-5764-4f05-9ebb-0f64eb04d95a service nova] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-9a167c57-6389-4ece-9558-f70d8189c4fa req-705e43eb-5764-4f05-9ebb-0f64eb04d95a service nova] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG nova.compute.manager [req-9a167c57-6389-4ece-9558-f70d8189c4fa req-705e43eb-5764-4f05-9ebb-0f64eb04d95a service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] No waiting events found dispatching network-vif-unplugged-5d0bb297-60b1-4e66-8f43-3bf1c32ae4ef {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG nova.compute.manager [req-9a167c57-6389-4ece-9558-f70d8189c4fa req-705e43eb-5764-4f05-9ebb-0f64eb04d95a service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received event network-vif-unplugged-5d0bb297-60b1-4e66-8f43-3bf1c32ae4ef for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG nova.compute.manager [req-9a167c57-6389-4ece-9558-f70d8189c4fa req-705e43eb-5764-4f05-9ebb-0f64eb04d95a service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received event network-vif-plugged-5d0bb297-60b1-4e66-8f43-3bf1c32ae4ef {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-9a167c57-6389-4ece-9558-f70d8189c4fa req-705e43eb-5764-4f05-9ebb-0f64eb04d95a service nova] Acquiring lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-9a167c57-6389-4ece-9558-f70d8189c4fa req-705e43eb-5764-4f05-9ebb-0f64eb04d95a service nova] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-9a167c57-6389-4ece-9558-f70d8189c4fa req-705e43eb-5764-4f05-9ebb-0f64eb04d95a service nova] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG nova.compute.manager [req-9a167c57-6389-4ece-9558-f70d8189c4fa req-705e43eb-5764-4f05-9ebb-0f64eb04d95a service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] No waiting events found dispatching network-vif-plugged-5d0bb297-60b1-4e66-8f43-3bf1c32ae4ef {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:20:22 user nova-compute[72351]: WARNING nova.compute.manager [req-9a167c57-6389-4ece-9558-f70d8189c4fa req-705e43eb-5764-4f05-9ebb-0f64eb04d95a service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received unexpected event network-vif-plugged-5d0bb297-60b1-4e66-8f43-3bf1c32ae4ef for instance with vm_state active and task_state deleting.
Aug 09 02:20:22 user nova-compute[72351]: DEBUG nova.compute.manager [req-9a167c57-6389-4ece-9558-f70d8189c4fa req-705e43eb-5764-4f05-9ebb-0f64eb04d95a service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received event network-vif-unplugged-c7c195c0-af76-445e-8a18-8169f0cdf5fc {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-9a167c57-6389-4ece-9558-f70d8189c4fa req-705e43eb-5764-4f05-9ebb-0f64eb04d95a service nova] Acquiring lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-9a167c57-6389-4ece-9558-f70d8189c4fa req-705e43eb-5764-4f05-9ebb-0f64eb04d95a service nova] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-9a167c57-6389-4ece-9558-f70d8189c4fa req-705e43eb-5764-4f05-9ebb-0f64eb04d95a service nova] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG nova.compute.manager [req-9a167c57-6389-4ece-9558-f70d8189c4fa req-705e43eb-5764-4f05-9ebb-0f64eb04d95a service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] No waiting events found dispatching network-vif-unplugged-c7c195c0-af76-445e-8a18-8169f0cdf5fc {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG nova.compute.manager [req-9a167c57-6389-4ece-9558-f70d8189c4fa req-705e43eb-5764-4f05-9ebb-0f64eb04d95a service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received event network-vif-unplugged-c7c195c0-af76-445e-8a18-8169f0cdf5fc for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG nova.compute.manager [req-9a167c57-6389-4ece-9558-f70d8189c4fa req-705e43eb-5764-4f05-9ebb-0f64eb04d95a service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received event network-vif-plugged-c7c195c0-af76-445e-8a18-8169f0cdf5fc {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-9a167c57-6389-4ece-9558-f70d8189c4fa req-705e43eb-5764-4f05-9ebb-0f64eb04d95a service nova] Acquiring lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-9a167c57-6389-4ece-9558-f70d8189c4fa req-705e43eb-5764-4f05-9ebb-0f64eb04d95a service nova] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-9a167c57-6389-4ece-9558-f70d8189c4fa req-705e43eb-5764-4f05-9ebb-0f64eb04d95a service nova] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG nova.compute.manager [req-9a167c57-6389-4ece-9558-f70d8189c4fa req-705e43eb-5764-4f05-9ebb-0f64eb04d95a service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] No waiting events found dispatching network-vif-plugged-c7c195c0-af76-445e-8a18-8169f0cdf5fc {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:20:22 user nova-compute[72351]: WARNING nova.compute.manager [req-9a167c57-6389-4ece-9558-f70d8189c4fa req-705e43eb-5764-4f05-9ebb-0f64eb04d95a service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received unexpected event network-vif-plugged-c7c195c0-af76-445e-8a18-8169f0cdf5fc for instance with vm_state active and task_state deleting.
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[9860a834-2eb2-49b3-8507-6322377b5a8f]: (4, ('tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f (non-flash)
Aug 09 02:20:22 user nova-compute[72351]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)
Aug 09 02:20:22 user nova-compute[72351]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:20:22 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] iscsi session list stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f (non-flash)
Aug 09 02:20:22 user nova-compute[72351]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b (non-flash)
Aug 09 02:20:22 user nova-compute[72351]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:20:22 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fcfdc1939a0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b'): ({'sdd'}, set())}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Removing single pathed devices sdd {{(pid=72351) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[7c9ee204-2bd0-44e7-8d3e-07b77b81c983]: (4, ('path checker states:\nup                  2\npending             1\n\npaths: 0\nbusy: False\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdd {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdd" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[f8e68453-f74c-487f-abe2-573326f1bf9c]: (4, ('ok\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Flushing IO for device /dev/sdd {{(pid=72351) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdd {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdd" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[bb0b93e1-e623-437f-af19-4b92588ddfe0]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Remove SCSI device /dev/sdd with /sys/block/sdd/device/delete {{(pid=72351) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdd/device/delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdd/device/delete" returned: 0 in 0.047s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[5f7deca6-2c47-4180-b0f8-e8d33fa7a13e]: (4, ('1', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Checking to see if SCSI volumes sdd have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] SCSI volumes sdd have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b')] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a1b9ecf4-445d-41d8-91c4-04b8a9c0ad5b]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b -p 172.16.0.220:3260 --logout {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b -p 172.16.0.220:3260 --logout" returned: 0 in 0.055s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[86fe4cf1-e63b-480d-90cd-8da085300ea6]: (4, ('Logging out of session [sid: 4, target: iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b, portal: 172.16.0.220,3260]\nLogout of [sid: 4, target: iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 4, target: iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b, portal: 172.16.0.220,3260]
Aug 09 02:20:22 user nova-compute[72351]: Logout of [sid: 4, target: iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b, portal: 172.16.0.220,3260] successful.
Aug 09 02:20:22 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:20:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b -p 172.16.0.220:3260 --op delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d1fa2bd8-1c39-4730-b60c-35dcbf24b18b -p 172.16.0.220:3260 --op delete" returned: 0 in 0.028s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:23 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d2b1fcd7-fff2-416e-8c20-ce9c8d370e03]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:20:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.305s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:20:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] <== disconnect_volume: return (308ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:20:23 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 02:20:23 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Deleting instance files /opt/stack/data/nova/instances/629e177e-afbb-4765-a0b5-7a3f66f72ebf_del
Aug 09 02:20:23 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Deletion of /opt/stack/data/nova/instances/629e177e-afbb-4765-a0b5-7a3f66f72ebf_del complete
Aug 09 02:20:23 user nova-compute[72351]: INFO nova.compute.manager [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Took 3.31 seconds to destroy the instance on the hypervisor.
Aug 09 02:20:23 user nova-compute[72351]: DEBUG oslo.service.loopingcall [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72351) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Aug 09 02:20:23 user nova-compute[72351]: DEBUG nova.compute.manager [-] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Deallocating network for instance {{(pid=72351) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2263}}
Aug 09 02:20:23 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] deallocate_for_instance() {{(pid=72351) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1795}}
Aug 09 02:20:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Acquiring lock "a1d1f738-0edf-4184-aa56-020c07830492" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:20:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Lock "a1d1f738-0edf-4184-aa56-020c07830492" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:20:24 user nova-compute[72351]: DEBUG nova.compute.manager [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: a1d1f738-0edf-4184-aa56-020c07830492] Starting instance... {{(pid=72351) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2406}}
Aug 09 02:20:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:20:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:20:24 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72351) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Aug 09 02:20:24 user nova-compute[72351]: INFO nova.compute.claims [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: a1d1f738-0edf-4184-aa56-020c07830492] Claim successful on node user
Aug 09 02:20:24 user nova-compute[72351]: DEBUG nova.compute.manager [req-458dacbc-0bdf-4224-a6ac-5dd590a962fc req-31c99c5f-ae56-4750-ad0f-ae19f026b591 service nova] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Received event network-changed-74c98543-ce04-43d8-a58c-267e372db698 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:20:24 user nova-compute[72351]: DEBUG nova.compute.manager [req-458dacbc-0bdf-4224-a6ac-5dd590a962fc req-31c99c5f-ae56-4750-ad0f-ae19f026b591 service nova] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Refreshing instance network info cache due to event network-changed-74c98543-ce04-43d8-a58c-267e372db698. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:20:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-458dacbc-0bdf-4224-a6ac-5dd590a962fc req-31c99c5f-ae56-4750-ad0f-ae19f026b591 service nova] Acquiring lock "refresh_cache-97b13ccd-15b4-44bf-b865-71b4da861168" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:20:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-458dacbc-0bdf-4224-a6ac-5dd590a962fc req-31c99c5f-ae56-4750-ad0f-ae19f026b591 service nova] Acquired lock "refresh_cache-97b13ccd-15b4-44bf-b865-71b4da861168" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:20:24 user nova-compute[72351]: DEBUG nova.network.neutron [req-458dacbc-0bdf-4224-a6ac-5dd590a962fc req-31c99c5f-ae56-4750-ad0f-ae19f026b591 service nova] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Refreshing network info cache for port 74c98543-ce04-43d8-a58c-267e372db698 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:20:24 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:20:24 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:20:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.481s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:20:24 user nova-compute[72351]: DEBUG nova.compute.manager [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: a1d1f738-0edf-4184-aa56-020c07830492] Start building networks asynchronously for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2803}}
Aug 09 02:20:24 user nova-compute[72351]: DEBUG nova.compute.manager [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: a1d1f738-0edf-4184-aa56-020c07830492] Allocating IP information in the background. {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1956}}
Aug 09 02:20:24 user nova-compute[72351]: DEBUG nova.network.neutron [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: a1d1f738-0edf-4184-aa56-020c07830492] allocate_for_instance() {{(pid=72351) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Aug 09 02:20:24 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: a1d1f738-0edf-4184-aa56-020c07830492] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Aug 09 02:20:24 user nova-compute[72351]: DEBUG nova.compute.manager [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: a1d1f738-0edf-4184-aa56-020c07830492] Start building block device mappings for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2838}}
Aug 09 02:20:25 user nova-compute[72351]: DEBUG nova.policy [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'fb9f73926e0d42f586e7c0055764d611', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bc5e05fbd7f44f85bc3b1cc169dd03a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72351) authorize /opt/stack/nova/nova/policy.py:203}}
Aug 09 02:20:25 user nova-compute[72351]: DEBUG nova.compute.manager [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: a1d1f738-0edf-4184-aa56-020c07830492] Start spawning the instance on the hypervisor. {{(pid=72351) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2612}}
Aug 09 02:20:25 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: a1d1f738-0edf-4184-aa56-020c07830492] Creating instance directory {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4717}}
Aug 09 02:20:25 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: a1d1f738-0edf-4184-aa56-020c07830492] Creating image(s)
Aug 09 02:20:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Acquiring lock "/opt/stack/data/nova/instances/a1d1f738-0edf-4184-aa56-020c07830492/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:20:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Lock "/opt/stack/data/nova/instances/a1d1f738-0edf-4184-aa56-020c07830492/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:20:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Lock "/opt/stack/data/nova/instances/a1d1f738-0edf-4184-aa56-020c07830492/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:20:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.412s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Acquiring lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:20:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:20:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.159s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/a1d1f738-0edf-4184-aa56-020c07830492/disk 1073741824 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:25 user nova-compute[72351]: DEBUG nova.compute.manager [req-af491498-1977-4b2f-ac67-6f22d1c53862 req-f576899c-4459-4681-ab5f-9d00cb4f40b3 service nova] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Received event network-changed-74c98543-ce04-43d8-a58c-267e372db698 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:20:25 user nova-compute[72351]: DEBUG nova.compute.manager [req-af491498-1977-4b2f-ac67-6f22d1c53862 req-f576899c-4459-4681-ab5f-9d00cb4f40b3 service nova] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Refreshing instance network info cache due to event network-changed-74c98543-ce04-43d8-a58c-267e372db698. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:20:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-af491498-1977-4b2f-ac67-6f22d1c53862 req-f576899c-4459-4681-ab5f-9d00cb4f40b3 service nova] Acquiring lock "refresh_cache-97b13ccd-15b4-44bf-b865-71b4da861168" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:20:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/a1d1f738-0edf-4184-aa56-020c07830492/disk 1073741824" returned: 0 in 0.052s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.216s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:20:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.160s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:25 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Checking if we can resize image /opt/stack/data/nova/instances/a1d1f738-0edf-4184-aa56-020c07830492/disk. size=1073741824 {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Aug 09 02:20:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a1d1f738-0edf-4184-aa56-020c07830492/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:26 user nova-compute[72351]: DEBUG nova.compute.manager [req-aac37e78-fc42-43b8-92b6-26795d56e867 req-fc38a960-ee12-4428-ab5f-b9a784e6ba78 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received event network-vif-deleted-5d0bb297-60b1-4e66-8f43-3bf1c32ae4ef {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:20:26 user nova-compute[72351]: INFO nova.compute.manager [req-aac37e78-fc42-43b8-92b6-26795d56e867 req-fc38a960-ee12-4428-ab5f-b9a784e6ba78 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Neutron deleted interface 5d0bb297-60b1-4e66-8f43-3bf1c32ae4ef; detaching it from the instance and deleting it from the info cache
Aug 09 02:20:26 user nova-compute[72351]: DEBUG nova.network.neutron [req-aac37e78-fc42-43b8-92b6-26795d56e867 req-fc38a960-ee12-4428-ab5f-b9a784e6ba78 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Updating instance_info_cache with network_info: [{"id": "e021d67e-09ab-4779-b639-dff1ba39798a", "address": "fa:16:3e:60:29:2e", "network": {"id": "9d2c7895-3144-4087-8a69-40739a321574", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest-566420926-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.196", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape021d67e-09", "ovs_interfaceid": "e021d67e-09ab-4779-b639-dff1ba39798a", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "f5ecb594-7ac3-496b-9bb0-9f908b91c8fb", "address": "fa:16:3e:30:31:a4", "network": {"id": "4e0c5236-b30d-4213-b1b4-26c0f1a636eb", "bridge": "br-int", "label": "tempest-device-tagging-net1-1640959147", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.122", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf5ecb594-7a", "ovs_interfaceid": "f5ecb594-7ac3-496b-9bb0-9f908b91c8fb", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "2e9c8a9c-2d81-4b84-b481-9a6b513e730c", "address": "fa:16:3e:39:9b:9d", "network": {"id": "4e0c5236-b30d-4213-b1b4-26c0f1a636eb", "bridge": "br-int", "label": "tempest-device-tagging-net1-1640959147", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.75", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2e9c8a9c-2d", "ovs_interfaceid": "2e9c8a9c-2d81-4b84-b481-9a6b513e730c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "c2df74bf-22ce-48b1-9ab8-3a684cef7266", "address": "fa:16:3e:3f:be:14", "network": {"id": "4e0c5236-b30d-4213-b1b4-26c0f1a636eb", "bridge": "br-int", "label": "tempest-device-tagging-net1-1640959147", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.128", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc2df74bf-22", "ovs_interfaceid": "c2df74bf-22ce-48b1-9ab8-3a684cef7266", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "d1b9314b-1800-41fb-a34b-0186f8e746da", "address": "fa:16:3e:37:27:3b", "network": {"id": "b4ebdbb0-e697-4a42-93f5-946522a3dbce", "bridge": "br-int", "label": "tempest-device-tagging-net2-356335140", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd1b9314b-18", "ovs_interfaceid": "d1b9314b-1800-41fb-a34b-0186f8e746da", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "c7c195c0-af76-445e-8a18-8169f0cdf5fc", "address": "fa:16:3e:f7:2d:31", "network": {"id": "b4ebdbb0-e697-4a42-93f5-946522a3dbce", "bridge": "br-int", "label": "tempest-device-tagging-net2-356335140", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc7c195c0-af", "ovs_interfaceid": "c7c195c0-af76-445e-8a18-8169f0cdf5fc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:20:26 user nova-compute[72351]: DEBUG nova.compute.manager [req-aac37e78-fc42-43b8-92b6-26795d56e867 req-fc38a960-ee12-4428-ab5f-b9a784e6ba78 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Detach interface failed, port_id=5d0bb297-60b1-4e66-8f43-3bf1c32ae4ef, reason: Instance 629e177e-afbb-4765-a0b5-7a3f66f72ebf could not be found. {{(pid=72351) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10866}}
Aug 09 02:20:26 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a1d1f738-0edf-4184-aa56-020c07830492/disk --force-share --output=json" returned: 0 in 0.146s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:26 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Cannot resize image /opt/stack/data/nova/instances/a1d1f738-0edf-4184-aa56-020c07830492/disk to a smaller size. {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Aug 09 02:20:26 user nova-compute[72351]: DEBUG nova.objects.instance [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Lazy-loading 'migration_context' on Instance uuid a1d1f738-0edf-4184-aa56-020c07830492 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:20:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: a1d1f738-0edf-4184-aa56-020c07830492] Created local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4851}}
Aug 09 02:20:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: a1d1f738-0edf-4184-aa56-020c07830492] Ensure instance console log exists: /opt/stack/data/nova/instances/a1d1f738-0edf-4184-aa56-020c07830492/console.log {{(pid=72351) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4603}}
Aug 09 02:20:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:20:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:20:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:20:26 user nova-compute[72351]: DEBUG nova.network.neutron [req-458dacbc-0bdf-4224-a6ac-5dd590a962fc req-31c99c5f-ae56-4750-ad0f-ae19f026b591 service nova] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Updated VIF entry in instance network info cache for port 74c98543-ce04-43d8-a58c-267e372db698. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:20:26 user nova-compute[72351]: DEBUG nova.network.neutron [req-458dacbc-0bdf-4224-a6ac-5dd590a962fc req-31c99c5f-ae56-4750-ad0f-ae19f026b591 service nova] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Updating instance_info_cache with network_info: [{"id": "74c98543-ce04-43d8-a58c-267e372db698", "address": "fa:16:3e:d4:4d:0a", "network": {"id": "7024d1f6-a400-4482-9364-78ae3b6adc80", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1603258958-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.62", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c2f67da658d94fabbab82ee6a6171c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap74c98543-ce", "ovs_interfaceid": "74c98543-ce04-43d8-a58c-267e372db698", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:20:26 user nova-compute[72351]: DEBUG nova.network.neutron [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: a1d1f738-0edf-4184-aa56-020c07830492] Successfully created port: 3e179747-c782-4b72-90d0-5b3bb68c3101 {{(pid=72351) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Aug 09 02:20:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-458dacbc-0bdf-4224-a6ac-5dd590a962fc req-31c99c5f-ae56-4750-ad0f-ae19f026b591 service nova] Releasing lock "refresh_cache-97b13ccd-15b4-44bf-b865-71b4da861168" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:20:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-af491498-1977-4b2f-ac67-6f22d1c53862 req-f576899c-4459-4681-ab5f-9d00cb4f40b3 service nova] Acquired lock "refresh_cache-97b13ccd-15b4-44bf-b865-71b4da861168" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:20:26 user nova-compute[72351]: DEBUG nova.network.neutron [req-af491498-1977-4b2f-ac67-6f22d1c53862 req-f576899c-4459-4681-ab5f-9d00cb4f40b3 service nova] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Refreshing network info cache for port 74c98543-ce04-43d8-a58c-267e372db698 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:20:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5003d907-18be-42e6-97ab-15331b7d1b0b tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Acquiring lock "7c5e633d-a32f-4c5e-bb9d-1e40da964bd0" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:20:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5003d907-18be-42e6-97ab-15331b7d1b0b tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Lock "7c5e633d-a32f-4c5e-bb9d-1e40da964bd0" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:20:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5003d907-18be-42e6-97ab-15331b7d1b0b tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Acquiring lock "7c5e633d-a32f-4c5e-bb9d-1e40da964bd0-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:20:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5003d907-18be-42e6-97ab-15331b7d1b0b tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Lock "7c5e633d-a32f-4c5e-bb9d-1e40da964bd0-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:20:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5003d907-18be-42e6-97ab-15331b7d1b0b tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Lock "7c5e633d-a32f-4c5e-bb9d-1e40da964bd0-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:20:26 user nova-compute[72351]: INFO nova.compute.manager [None req-5003d907-18be-42e6-97ab-15331b7d1b0b tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Terminating instance
Aug 09 02:20:26 user nova-compute[72351]: DEBUG nova.compute.manager [None req-5003d907-18be-42e6-97ab-15331b7d1b0b tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Start destroying the instance on the hypervisor. {{(pid=72351) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3124}}
Aug 09 02:20:26 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:26 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:26 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG nova.network.neutron [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: a1d1f738-0edf-4184-aa56-020c07830492] Successfully updated port: 3e179747-c782-4b72-90d0-5b3bb68c3101 {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Acquiring lock "refresh_cache-a1d1f738-0edf-4184-aa56-020c07830492" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Acquired lock "refresh_cache-a1d1f738-0edf-4184-aa56-020c07830492" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG nova.network.neutron [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: a1d1f738-0edf-4184-aa56-020c07830492] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG nova.compute.manager [req-3eeb248f-853e-466d-ab24-6b922d145333 req-f4f930dc-b49d-429f-b60e-a593b2b7f843 service nova] [instance: a1d1f738-0edf-4184-aa56-020c07830492] Received event network-changed-3e179747-c782-4b72-90d0-5b3bb68c3101 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG nova.compute.manager [req-3eeb248f-853e-466d-ab24-6b922d145333 req-f4f930dc-b49d-429f-b60e-a593b2b7f843 service nova] [instance: a1d1f738-0edf-4184-aa56-020c07830492] Refreshing instance network info cache due to event network-changed-3e179747-c782-4b72-90d0-5b3bb68c3101. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-3eeb248f-853e-466d-ab24-6b922d145333 req-f4f930dc-b49d-429f-b60e-a593b2b7f843 service nova] Acquiring lock "refresh_cache-a1d1f738-0edf-4184-aa56-020c07830492" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG nova.network.neutron [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: a1d1f738-0edf-4184-aa56-020c07830492] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:27 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Instance destroyed successfully.
Aug 09 02:20:27 user nova-compute[72351]: DEBUG nova.objects.instance [None req-5003d907-18be-42e6-97ab-15331b7d1b0b tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Lazy-loading 'resources' on Instance uuid 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Acquiring lock "32a28ecb-af4b-4c6e-8f2d-a51857134b34" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Lock "32a28ecb-af4b-4c6e-8f2d-a51857134b34" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-5003d907-18be-42e6-97ab-15331b7d1b0b tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:17:21Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-VolumesAssistedSnapshotsTest-server-340980583',display_name='tempest-VolumesAssistedSnapshotsTest-server-340980583',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-volumesassistedsnapshotstest-server-340980583',id=6,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBAHay+TchlGaaKhLpYV0vC5z6GvCYSIp4aglFVFWN37IvZ4glkz7z31jiUXDM9oqyU3JNQYda4Z7n5Vj2jgZz/+uvC2MLvz6lGYCHuZO0g1H0LqpN/JGwocv3XJU6QV9kA==',key_name='tempest-keypair-1914691164',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:17:36Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='26827058be4b48a3a7aed21aad7c5e77',ramdisk_id='',reservation_id='r-pij98zku',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-VolumesAssistedSnapshotsTest-1116923063',owner_user_name='tempest-VolumesAssistedSnapshotsTest-1116923063-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:17:36Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='52bc0e2d81424ca19722e8af947d399b',uuid=7c5e633d-a32f-4c5e-bb9d-1e40da964bd0,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "7e7b4229-7c23-4a73-9294-4c4fc9f6a01a", "address": "fa:16:3e:d9:0b:0f", "network": {"id": "49cd7d7c-b635-484c-9ef3-268529d7344f", "bridge": "br-int", "label": "tempest-VolumesAssistedSnapshotsTest-24910214-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.168", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "26827058be4b48a3a7aed21aad7c5e77", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7e7b4229-7c", "ovs_interfaceid": "7e7b4229-7c23-4a73-9294-4c4fc9f6a01a", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-5003d907-18be-42e6-97ab-15331b7d1b0b tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Converting VIF {"id": "7e7b4229-7c23-4a73-9294-4c4fc9f6a01a", "address": "fa:16:3e:d9:0b:0f", "network": {"id": "49cd7d7c-b635-484c-9ef3-268529d7344f", "bridge": "br-int", "label": "tempest-VolumesAssistedSnapshotsTest-24910214-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.168", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "26827058be4b48a3a7aed21aad7c5e77", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7e7b4229-7c", "ovs_interfaceid": "7e7b4229-7c23-4a73-9294-4c4fc9f6a01a", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-5003d907-18be-42e6-97ab-15331b7d1b0b tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:d9:0b:0f,bridge_name='br-int',has_traffic_filtering=True,id=7e7b4229-7c23-4a73-9294-4c4fc9f6a01a,network=Network(49cd7d7c-b635-484c-9ef3-268529d7344f),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7e7b4229-7c') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG os_vif [None req-5003d907-18be-42e6-97ab-15331b7d1b0b tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:d9:0b:0f,bridge_name='br-int',has_traffic_filtering=True,id=7e7b4229-7c23-4a73-9294-4c4fc9f6a01a,network=Network(49cd7d7c-b635-484c-9ef3-268529d7344f),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7e7b4229-7c') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap7e7b4229-7c, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:27 user nova-compute[72351]: INFO os_vif [None req-5003d907-18be-42e6-97ab-15331b7d1b0b tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:d9:0b:0f,bridge_name='br-int',has_traffic_filtering=True,id=7e7b4229-7c23-4a73-9294-4c4fc9f6a01a,network=Network(49cd7d7c-b635-484c-9ef3-268529d7344f),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7e7b4229-7c')
Aug 09 02:20:27 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-5003d907-18be-42e6-97ab-15331b7d1b0b tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Deleting instance files /opt/stack/data/nova/instances/7c5e633d-a32f-4c5e-bb9d-1e40da964bd0_del
Aug 09 02:20:27 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-5003d907-18be-42e6-97ab-15331b7d1b0b tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Deletion of /opt/stack/data/nova/instances/7c5e633d-a32f-4c5e-bb9d-1e40da964bd0_del complete
Aug 09 02:20:27 user nova-compute[72351]: DEBUG nova.compute.manager [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] Starting instance... {{(pid=72351) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2406}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG nova.network.neutron [req-af491498-1977-4b2f-ac67-6f22d1c53862 req-f576899c-4459-4681-ab5f-9d00cb4f40b3 service nova] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Updated VIF entry in instance network info cache for port 74c98543-ce04-43d8-a58c-267e372db698. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG nova.network.neutron [req-af491498-1977-4b2f-ac67-6f22d1c53862 req-f576899c-4459-4681-ab5f-9d00cb4f40b3 service nova] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Updating instance_info_cache with network_info: [{"id": "74c98543-ce04-43d8-a58c-267e372db698", "address": "fa:16:3e:d4:4d:0a", "network": {"id": "7024d1f6-a400-4482-9364-78ae3b6adc80", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1603258958-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.62", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c2f67da658d94fabbab82ee6a6171c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap74c98543-ce", "ovs_interfaceid": "74c98543-ce04-43d8-a58c-267e372db698", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-af491498-1977-4b2f-ac67-6f22d1c53862 req-f576899c-4459-4681-ab5f-9d00cb4f40b3 service nova] Releasing lock "refresh_cache-97b13ccd-15b4-44bf-b865-71b4da861168" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:20:27 user nova-compute[72351]: INFO nova.compute.manager [None req-5003d907-18be-42e6-97ab-15331b7d1b0b tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Took 0.93 seconds to destroy the instance on the hypervisor.
Aug 09 02:20:27 user nova-compute[72351]: DEBUG oslo.service.loopingcall [None req-5003d907-18be-42e6-97ab-15331b7d1b0b tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72351) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG nova.compute.manager [-] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Deallocating network for instance {{(pid=72351) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2263}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] deallocate_for_instance() {{(pid=72351) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1795}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72351) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Aug 09 02:20:27 user nova-compute[72351]: INFO nova.compute.claims [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] Claim successful on node user
Aug 09 02:20:27 user nova-compute[72351]: DEBUG nova.network.neutron [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: a1d1f738-0edf-4184-aa56-020c07830492] Updating instance_info_cache with network_info: [{"id": "3e179747-c782-4b72-90d0-5b3bb68c3101", "address": "fa:16:3e:54:6f:e3", "network": {"id": "21efbfa4-f3a2-4317-89fc-3cc7b60fde97", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1919568535-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bc5e05fbd7f44f85bc3b1cc169dd03a9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3e179747-c7", "ovs_interfaceid": "3e179747-c782-4b72-90d0-5b3bb68c3101", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Releasing lock "refresh_cache-a1d1f738-0edf-4184-aa56-020c07830492" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG nova.compute.manager [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: a1d1f738-0edf-4184-aa56-020c07830492] Instance network_info: |[{"id": "3e179747-c782-4b72-90d0-5b3bb68c3101", "address": "fa:16:3e:54:6f:e3", "network": {"id": "21efbfa4-f3a2-4317-89fc-3cc7b60fde97", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1919568535-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bc5e05fbd7f44f85bc3b1cc169dd03a9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3e179747-c7", "ovs_interfaceid": "3e179747-c782-4b72-90d0-5b3bb68c3101", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1971}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-3eeb248f-853e-466d-ab24-6b922d145333 req-f4f930dc-b49d-429f-b60e-a593b2b7f843 service nova] Acquired lock "refresh_cache-a1d1f738-0edf-4184-aa56-020c07830492" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG nova.network.neutron [req-3eeb248f-853e-466d-ab24-6b922d145333 req-f4f930dc-b49d-429f-b60e-a593b2b7f843 service nova] [instance: a1d1f738-0edf-4184-aa56-020c07830492] Refreshing network info cache for port 3e179747-c782-4b72-90d0-5b3bb68c3101 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: a1d1f738-0edf-4184-aa56-020c07830492] Start _get_guest_xml network_info=[{"id": "3e179747-c782-4b72-90d0-5b3bb68c3101", "address": "fa:16:3e:54:6f:e3", "network": {"id": "21efbfa4-f3a2-4317-89fc-3cc7b60fde97", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1919568535-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bc5e05fbd7f44f85bc3b1cc169dd03a9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3e179747-c7", "ovs_interfaceid": "3e179747-c782-4b72-90d0-5b3bb68c3101", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'size': 0, 'device_type': 'disk', 'encryption_secret_uuid': None, 'guest_format': None, 'encryption_options': None, 'disk_bus': 'virtio', 'encrypted': False, 'encryption_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'image_id': '18ccda7f-31d3-4978-ab9d-4ed2f74eff94'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:20:27 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:20:27 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:20:27 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:20:23Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesAdminNegativeTest-server-1398011561',display_name='tempest-VolumesAdminNegativeTest-server-1398011561',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-volumesadminnegativetest-server-1398011561',id=13,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='bc5e05fbd7f44f85bc3b1cc169dd03a9',ramdisk_id='',reservation_id='r-vh2c5s83',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesAdminNegativeTest-35684003',owner_user_name='tempest-VolumesAdminNegativeTest-35684003-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:20:25Z,user_data=None,user_id='fb9f73926e0d42f586e7c0055764d611',uuid=a1d1f738-0edf-4184-aa56-020c07830492,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "3e179747-c782-4b72-90d0-5b3bb68c3101", "address": "fa:16:3e:54:6f:e3", "network": {"id": "21efbfa4-f3a2-4317-89fc-3cc7b60fde97", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1919568535-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bc5e05fbd7f44f85bc3b1cc169dd03a9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3e179747-c7", "ovs_interfaceid": "3e179747-c782-4b72-90d0-5b3bb68c3101", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Converting VIF {"id": "3e179747-c782-4b72-90d0-5b3bb68c3101", "address": "fa:16:3e:54:6f:e3", "network": {"id": "21efbfa4-f3a2-4317-89fc-3cc7b60fde97", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1919568535-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bc5e05fbd7f44f85bc3b1cc169dd03a9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3e179747-c7", "ovs_interfaceid": "3e179747-c782-4b72-90d0-5b3bb68c3101", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:54:6f:e3,bridge_name='br-int',has_traffic_filtering=True,id=3e179747-c782-4b72-90d0-5b3bb68c3101,network=Network(21efbfa4-f3a2-4317-89fc-3cc7b60fde97),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3e179747-c7') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG nova.objects.instance [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Lazy-loading 'pci_devices' on Instance uuid a1d1f738-0edf-4184-aa56-020c07830492 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: a1d1f738-0edf-4184-aa56-020c07830492] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:20:27 user nova-compute[72351]:   <uuid>a1d1f738-0edf-4184-aa56-020c07830492</uuid>
Aug 09 02:20:27 user nova-compute[72351]:   <name>instance-0000000d</name>
Aug 09 02:20:27 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:20:27 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:20:27 user nova-compute[72351]:   <metadata>
Aug 09 02:20:27 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:20:27 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:20:27 user nova-compute[72351]:       <nova:name>tempest-VolumesAdminNegativeTest-server-1398011561</nova:name>
Aug 09 02:20:27 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:20:27</nova:creationTime>
Aug 09 02:20:27 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:20:27 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:20:27 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:20:27 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:20:27 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:20:27 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:20:27 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:20:27 user nova-compute[72351]:       <nova:owner>
Aug 09 02:20:27 user nova-compute[72351]:         <nova:user uuid="fb9f73926e0d42f586e7c0055764d611">tempest-VolumesAdminNegativeTest-35684003-project-member</nova:user>
Aug 09 02:20:27 user nova-compute[72351]:         <nova:project uuid="bc5e05fbd7f44f85bc3b1cc169dd03a9">tempest-VolumesAdminNegativeTest-35684003</nova:project>
Aug 09 02:20:27 user nova-compute[72351]:       </nova:owner>
Aug 09 02:20:27 user nova-compute[72351]:       <nova:root type="image" uuid="18ccda7f-31d3-4978-ab9d-4ed2f74eff94"/>
Aug 09 02:20:27 user nova-compute[72351]:       <nova:ports>
Aug 09 02:20:27 user nova-compute[72351]:         <nova:port uuid="3e179747-c782-4b72-90d0-5b3bb68c3101">
Aug 09 02:20:27 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.3" ipVersion="4"/>
Aug 09 02:20:27 user nova-compute[72351]:         </nova:port>
Aug 09 02:20:27 user nova-compute[72351]:       </nova:ports>
Aug 09 02:20:27 user nova-compute[72351]:     </nova:instance>
Aug 09 02:20:27 user nova-compute[72351]:   </metadata>
Aug 09 02:20:27 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:20:27 user nova-compute[72351]:     <system>
Aug 09 02:20:27 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:20:27 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:20:27 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:20:27 user nova-compute[72351]:       <entry name="serial">a1d1f738-0edf-4184-aa56-020c07830492</entry>
Aug 09 02:20:27 user nova-compute[72351]:       <entry name="uuid">a1d1f738-0edf-4184-aa56-020c07830492</entry>
Aug 09 02:20:27 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:20:27 user nova-compute[72351]:     </system>
Aug 09 02:20:27 user nova-compute[72351]:   </sysinfo>
Aug 09 02:20:27 user nova-compute[72351]:   <os>
Aug 09 02:20:27 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:20:27 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:20:27 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:20:27 user nova-compute[72351]:   </os>
Aug 09 02:20:27 user nova-compute[72351]:   <features>
Aug 09 02:20:27 user nova-compute[72351]:     <acpi/>
Aug 09 02:20:27 user nova-compute[72351]:     <apic/>
Aug 09 02:20:27 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:20:27 user nova-compute[72351]:   </features>
Aug 09 02:20:27 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:20:27 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:20:27 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:20:27 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:20:27 user nova-compute[72351]:   </clock>
Aug 09 02:20:27 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:20:27 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:20:27 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:20:27 user nova-compute[72351]:   </cpu>
Aug 09 02:20:27 user nova-compute[72351]:   <devices>
Aug 09 02:20:27 user nova-compute[72351]:     <disk type="file" device="disk">
Aug 09 02:20:27 user nova-compute[72351]:       <driver name="qemu" type="qcow2" cache="none"/>
Aug 09 02:20:27 user nova-compute[72351]:       <source file="/opt/stack/data/nova/instances/a1d1f738-0edf-4184-aa56-020c07830492/disk"/>
Aug 09 02:20:27 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:20:27 user nova-compute[72351]:     </disk>
Aug 09 02:20:27 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:20:27 user nova-compute[72351]:       <mac address="fa:16:3e:54:6f:e3"/>
Aug 09 02:20:27 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:20:27 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:20:27 user nova-compute[72351]:       <target dev="tap3e179747-c7"/>
Aug 09 02:20:27 user nova-compute[72351]:     </interface>
Aug 09 02:20:27 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:20:27 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/a1d1f738-0edf-4184-aa56-020c07830492/console.log" append="off"/>
Aug 09 02:20:27 user nova-compute[72351]:     </serial>
Aug 09 02:20:27 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:20:27 user nova-compute[72351]:     <video>
Aug 09 02:20:27 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:20:27 user nova-compute[72351]:     </video>
Aug 09 02:20:27 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:20:27 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:20:27 user nova-compute[72351]:     </rng>
Aug 09 02:20:27 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:20:27 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:20:27 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:20:27 user nova-compute[72351]:     </memballoon>
Aug 09 02:20:27 user nova-compute[72351]:   </devices>
Aug 09 02:20:27 user nova-compute[72351]: </domain>
Aug 09 02:20:27 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:20:23Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesAdminNegativeTest-server-1398011561',display_name='tempest-VolumesAdminNegativeTest-server-1398011561',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-volumesadminnegativetest-server-1398011561',id=13,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='bc5e05fbd7f44f85bc3b1cc169dd03a9',ramdisk_id='',reservation_id='r-vh2c5s83',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesAdminNegativeTest-35684003',owner_user_name='tempest-VolumesAdminNegativeTest-35684003-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:20:25Z,user_data=None,user_id='fb9f73926e0d42f586e7c0055764d611',uuid=a1d1f738-0edf-4184-aa56-020c07830492,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "3e179747-c782-4b72-90d0-5b3bb68c3101", "address": "fa:16:3e:54:6f:e3", "network": {"id": "21efbfa4-f3a2-4317-89fc-3cc7b60fde97", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1919568535-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bc5e05fbd7f44f85bc3b1cc169dd03a9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3e179747-c7", "ovs_interfaceid": "3e179747-c782-4b72-90d0-5b3bb68c3101", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Converting VIF {"id": "3e179747-c782-4b72-90d0-5b3bb68c3101", "address": "fa:16:3e:54:6f:e3", "network": {"id": "21efbfa4-f3a2-4317-89fc-3cc7b60fde97", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1919568535-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bc5e05fbd7f44f85bc3b1cc169dd03a9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3e179747-c7", "ovs_interfaceid": "3e179747-c782-4b72-90d0-5b3bb68c3101", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:54:6f:e3,bridge_name='br-int',has_traffic_filtering=True,id=3e179747-c782-4b72-90d0-5b3bb68c3101,network=Network(21efbfa4-f3a2-4317-89fc-3cc7b60fde97),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3e179747-c7') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG os_vif [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:54:6f:e3,bridge_name='br-int',has_traffic_filtering=True,id=3e179747-c782-4b72-90d0-5b3bb68c3101,network=Network(21efbfa4-f3a2-4317-89fc-3cc7b60fde97),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3e179747-c7') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap3e179747-c7, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap3e179747-c7, col_values=(('external_ids', {'iface-id': '3e179747-c782-4b72-90d0-5b3bb68c3101', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:54:6f:e3', 'vm-uuid': 'a1d1f738-0edf-4184-aa56-020c07830492'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:27 user nova-compute[72351]: INFO os_vif [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:54:6f:e3,bridge_name='br-int',has_traffic_filtering=True,id=3e179747-c782-4b72-90d0-5b3bb68c3101,network=Network(21efbfa4-f3a2-4317-89fc-3cc7b60fde97),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3e179747-c7')
Aug 09 02:20:27 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:20:27 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] No VIF found with MAC fa:16:3e:54:6f:e3, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:20:28 user nova-compute[72351]: DEBUG nova.compute.manager [req-5ac19888-a180-40c2-a71b-c268a507bf13 req-cac2edc1-6022-49ad-b57c-37a82c941d00 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received event network-vif-deleted-d1b9314b-1800-41fb-a34b-0186f8e746da {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:20:28 user nova-compute[72351]: INFO nova.compute.manager [req-5ac19888-a180-40c2-a71b-c268a507bf13 req-cac2edc1-6022-49ad-b57c-37a82c941d00 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Neutron deleted interface d1b9314b-1800-41fb-a34b-0186f8e746da; detaching it from the instance and deleting it from the info cache
Aug 09 02:20:28 user nova-compute[72351]: DEBUG nova.network.neutron [req-5ac19888-a180-40c2-a71b-c268a507bf13 req-cac2edc1-6022-49ad-b57c-37a82c941d00 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Updating instance_info_cache with network_info: [{"id": "e021d67e-09ab-4779-b639-dff1ba39798a", "address": "fa:16:3e:60:29:2e", "network": {"id": "9d2c7895-3144-4087-8a69-40739a321574", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest-566420926-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.196", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape021d67e-09", "ovs_interfaceid": "e021d67e-09ab-4779-b639-dff1ba39798a", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "f5ecb594-7ac3-496b-9bb0-9f908b91c8fb", "address": "fa:16:3e:30:31:a4", "network": {"id": "4e0c5236-b30d-4213-b1b4-26c0f1a636eb", "bridge": "br-int", "label": "tempest-device-tagging-net1-1640959147", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.122", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf5ecb594-7a", "ovs_interfaceid": "f5ecb594-7ac3-496b-9bb0-9f908b91c8fb", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "2e9c8a9c-2d81-4b84-b481-9a6b513e730c", "address": "fa:16:3e:39:9b:9d", "network": {"id": "4e0c5236-b30d-4213-b1b4-26c0f1a636eb", "bridge": "br-int", "label": "tempest-device-tagging-net1-1640959147", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.75", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2e9c8a9c-2d", "ovs_interfaceid": "2e9c8a9c-2d81-4b84-b481-9a6b513e730c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "c2df74bf-22ce-48b1-9ab8-3a684cef7266", "address": "fa:16:3e:3f:be:14", "network": {"id": "4e0c5236-b30d-4213-b1b4-26c0f1a636eb", "bridge": "br-int", "label": "tempest-device-tagging-net1-1640959147", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.128", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc2df74bf-22", "ovs_interfaceid": "c2df74bf-22ce-48b1-9ab8-3a684cef7266", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "c7c195c0-af76-445e-8a18-8169f0cdf5fc", "address": "fa:16:3e:f7:2d:31", "network": {"id": "b4ebdbb0-e697-4a42-93f5-946522a3dbce", "bridge": "br-int", "label": "tempest-device-tagging-net2-356335140", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc7c195c0-af", "ovs_interfaceid": "c7c195c0-af76-445e-8a18-8169f0cdf5fc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:20:28 user nova-compute[72351]: DEBUG nova.compute.manager [req-5ac19888-a180-40c2-a71b-c268a507bf13 req-cac2edc1-6022-49ad-b57c-37a82c941d00 service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Detach interface failed, port_id=d1b9314b-1800-41fb-a34b-0186f8e746da, reason: Instance 629e177e-afbb-4765-a0b5-7a3f66f72ebf could not be found. {{(pid=72351) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10866}}
Aug 09 02:20:28 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:20:28 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:20:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.539s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:20:28 user nova-compute[72351]: DEBUG nova.compute.manager [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] Start building networks asynchronously for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2803}}
Aug 09 02:20:28 user nova-compute[72351]: DEBUG nova.network.neutron [req-3eeb248f-853e-466d-ab24-6b922d145333 req-f4f930dc-b49d-429f-b60e-a593b2b7f843 service nova] [instance: a1d1f738-0edf-4184-aa56-020c07830492] Updated VIF entry in instance network info cache for port 3e179747-c782-4b72-90d0-5b3bb68c3101. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:20:28 user nova-compute[72351]: DEBUG nova.network.neutron [req-3eeb248f-853e-466d-ab24-6b922d145333 req-f4f930dc-b49d-429f-b60e-a593b2b7f843 service nova] [instance: a1d1f738-0edf-4184-aa56-020c07830492] Updating instance_info_cache with network_info: [{"id": "3e179747-c782-4b72-90d0-5b3bb68c3101", "address": "fa:16:3e:54:6f:e3", "network": {"id": "21efbfa4-f3a2-4317-89fc-3cc7b60fde97", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1919568535-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bc5e05fbd7f44f85bc3b1cc169dd03a9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3e179747-c7", "ovs_interfaceid": "3e179747-c782-4b72-90d0-5b3bb68c3101", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:20:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-3eeb248f-853e-466d-ab24-6b922d145333 req-f4f930dc-b49d-429f-b60e-a593b2b7f843 service nova] Releasing lock "refresh_cache-a1d1f738-0edf-4184-aa56-020c07830492" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:20:28 user nova-compute[72351]: DEBUG nova.compute.manager [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] Allocating IP information in the background. {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1956}}
Aug 09 02:20:28 user nova-compute[72351]: DEBUG nova.network.neutron [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] allocate_for_instance() {{(pid=72351) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Aug 09 02:20:28 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Aug 09 02:20:28 user nova-compute[72351]: DEBUG nova.compute.manager [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] Start building block device mappings for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2838}}
Aug 09 02:20:28 user nova-compute[72351]: DEBUG nova.policy [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '00a45ab3cb2d43028418af1ae209da51', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5209610aaaa34f7a8ffb7757dbfac85d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72351) authorize /opt/stack/nova/nova/policy.py:203}}
Aug 09 02:20:28 user nova-compute[72351]: DEBUG nova.compute.manager [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] Start spawning the instance on the hypervisor. {{(pid=72351) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2612}}
Aug 09 02:20:28 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] Creating instance directory {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4717}}
Aug 09 02:20:28 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] Creating image(s)
Aug 09 02:20:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Acquiring lock "/opt/stack/data/nova/instances/32a28ecb-af4b-4c6e-8f2d-a51857134b34/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:20:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Lock "/opt/stack/data/nova/instances/32a28ecb-af4b-4c6e-8f2d-a51857134b34/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:20:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Lock "/opt/stack/data/nova/instances/32a28ecb-af4b-4c6e-8f2d-a51857134b34/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:20:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.131s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Acquiring lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:20:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:20:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.166s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/32a28ecb-af4b-4c6e-8f2d-a51857134b34/disk 1073741824 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/32a28ecb-af4b-4c6e-8f2d-a51857134b34/disk 1073741824" returned: 0 in 0.048s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.222s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:20:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:29 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.145s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:29 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Checking if we can resize image /opt/stack/data/nova/instances/32a28ecb-af4b-4c6e-8f2d-a51857134b34/disk. size=1073741824 {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Aug 09 02:20:29 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/32a28ecb-af4b-4c6e-8f2d-a51857134b34/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:29 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/32a28ecb-af4b-4c6e-8f2d-a51857134b34/disk --force-share --output=json" returned: 0 in 0.149s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:29 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Cannot resize image /opt/stack/data/nova/instances/32a28ecb-af4b-4c6e-8f2d-a51857134b34/disk to a smaller size. {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Aug 09 02:20:29 user nova-compute[72351]: DEBUG nova.objects.instance [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Lazy-loading 'migration_context' on Instance uuid 32a28ecb-af4b-4c6e-8f2d-a51857134b34 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:20:29 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] Created local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4851}}
Aug 09 02:20:29 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] Ensure instance console log exists: /opt/stack/data/nova/instances/32a28ecb-af4b-4c6e-8f2d-a51857134b34/console.log {{(pid=72351) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4603}}
Aug 09 02:20:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:20:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:20:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:20:29 user nova-compute[72351]: DEBUG nova.compute.manager [req-c18f18f8-5996-425d-89dd-ae47e251aee6 req-76a67447-cc64-48d6-bf90-19fd83b7fc33 service nova] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Received event network-vif-unplugged-7e7b4229-7c23-4a73-9294-4c4fc9f6a01a {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:20:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c18f18f8-5996-425d-89dd-ae47e251aee6 req-76a67447-cc64-48d6-bf90-19fd83b7fc33 service nova] Acquiring lock "7c5e633d-a32f-4c5e-bb9d-1e40da964bd0-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:20:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c18f18f8-5996-425d-89dd-ae47e251aee6 req-76a67447-cc64-48d6-bf90-19fd83b7fc33 service nova] Lock "7c5e633d-a32f-4c5e-bb9d-1e40da964bd0-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:20:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c18f18f8-5996-425d-89dd-ae47e251aee6 req-76a67447-cc64-48d6-bf90-19fd83b7fc33 service nova] Lock "7c5e633d-a32f-4c5e-bb9d-1e40da964bd0-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:20:29 user nova-compute[72351]: DEBUG nova.compute.manager [req-c18f18f8-5996-425d-89dd-ae47e251aee6 req-76a67447-cc64-48d6-bf90-19fd83b7fc33 service nova] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] No waiting events found dispatching network-vif-unplugged-7e7b4229-7c23-4a73-9294-4c4fc9f6a01a {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:20:29 user nova-compute[72351]: DEBUG nova.compute.manager [req-c18f18f8-5996-425d-89dd-ae47e251aee6 req-76a67447-cc64-48d6-bf90-19fd83b7fc33 service nova] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Received event network-vif-unplugged-7e7b4229-7c23-4a73-9294-4c4fc9f6a01a for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:20:29 user nova-compute[72351]: DEBUG nova.compute.manager [req-c18f18f8-5996-425d-89dd-ae47e251aee6 req-76a67447-cc64-48d6-bf90-19fd83b7fc33 service nova] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Received event network-vif-plugged-7e7b4229-7c23-4a73-9294-4c4fc9f6a01a {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:20:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c18f18f8-5996-425d-89dd-ae47e251aee6 req-76a67447-cc64-48d6-bf90-19fd83b7fc33 service nova] Acquiring lock "7c5e633d-a32f-4c5e-bb9d-1e40da964bd0-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:20:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c18f18f8-5996-425d-89dd-ae47e251aee6 req-76a67447-cc64-48d6-bf90-19fd83b7fc33 service nova] Lock "7c5e633d-a32f-4c5e-bb9d-1e40da964bd0-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:20:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c18f18f8-5996-425d-89dd-ae47e251aee6 req-76a67447-cc64-48d6-bf90-19fd83b7fc33 service nova] Lock "7c5e633d-a32f-4c5e-bb9d-1e40da964bd0-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:20:29 user nova-compute[72351]: DEBUG nova.compute.manager [req-c18f18f8-5996-425d-89dd-ae47e251aee6 req-76a67447-cc64-48d6-bf90-19fd83b7fc33 service nova] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] No waiting events found dispatching network-vif-plugged-7e7b4229-7c23-4a73-9294-4c4fc9f6a01a {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:20:29 user nova-compute[72351]: WARNING nova.compute.manager [req-c18f18f8-5996-425d-89dd-ae47e251aee6 req-76a67447-cc64-48d6-bf90-19fd83b7fc33 service nova] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Received unexpected event network-vif-plugged-7e7b4229-7c23-4a73-9294-4c4fc9f6a01a for instance with vm_state active and task_state deleting.
Aug 09 02:20:29 user nova-compute[72351]: DEBUG nova.network.neutron [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] Successfully created port: 4bd2f461-68b0-4882-98d6-2bb99a324210 {{(pid=72351) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Aug 09 02:20:29 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:20:29 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Took 1.70 seconds to deallocate network for instance.
Aug 09 02:20:29 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5003d907-18be-42e6-97ab-15331b7d1b0b tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:20:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5003d907-18be-42e6-97ab-15331b7d1b0b tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:20:29 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:29 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:29 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-5003d907-18be-42e6-97ab-15331b7d1b0b tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:20:30 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-5003d907-18be-42e6-97ab-15331b7d1b0b tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:20:30 user nova-compute[72351]: DEBUG nova.compute.manager [req-f81f94d8-b201-4679-b4a3-984d47a866f7 req-22910b84-e0f8-456c-b55d-c31af97dc74e service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received event network-vif-deleted-e021d67e-09ab-4779-b639-dff1ba39798a {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:20:30 user nova-compute[72351]: INFO nova.compute.manager [req-f81f94d8-b201-4679-b4a3-984d47a866f7 req-22910b84-e0f8-456c-b55d-c31af97dc74e service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Neutron deleted interface e021d67e-09ab-4779-b639-dff1ba39798a; detaching it from the instance and deleting it from the info cache
Aug 09 02:20:30 user nova-compute[72351]: DEBUG nova.network.neutron [req-f81f94d8-b201-4679-b4a3-984d47a866f7 req-22910b84-e0f8-456c-b55d-c31af97dc74e service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Updating instance_info_cache with network_info: [{"id": "f5ecb594-7ac3-496b-9bb0-9f908b91c8fb", "address": "fa:16:3e:30:31:a4", "network": {"id": "4e0c5236-b30d-4213-b1b4-26c0f1a636eb", "bridge": "br-int", "label": "tempest-device-tagging-net1-1640959147", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.122", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf5ecb594-7a", "ovs_interfaceid": "f5ecb594-7ac3-496b-9bb0-9f908b91c8fb", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "2e9c8a9c-2d81-4b84-b481-9a6b513e730c", "address": "fa:16:3e:39:9b:9d", "network": {"id": "4e0c5236-b30d-4213-b1b4-26c0f1a636eb", "bridge": "br-int", "label": "tempest-device-tagging-net1-1640959147", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.75", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2e9c8a9c-2d", "ovs_interfaceid": "2e9c8a9c-2d81-4b84-b481-9a6b513e730c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "c2df74bf-22ce-48b1-9ab8-3a684cef7266", "address": "fa:16:3e:3f:be:14", "network": {"id": "4e0c5236-b30d-4213-b1b4-26c0f1a636eb", "bridge": "br-int", "label": "tempest-device-tagging-net1-1640959147", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.128", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc2df74bf-22", "ovs_interfaceid": "c2df74bf-22ce-48b1-9ab8-3a684cef7266", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "c7c195c0-af76-445e-8a18-8169f0cdf5fc", "address": "fa:16:3e:f7:2d:31", "network": {"id": "b4ebdbb0-e697-4a42-93f5-946522a3dbce", "bridge": "br-int", "label": "tempest-device-tagging-net2-356335140", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc7c195c0-af", "ovs_interfaceid": "c7c195c0-af76-445e-8a18-8169f0cdf5fc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:20:30 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5003d907-18be-42e6-97ab-15331b7d1b0b tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.523s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:20:30 user nova-compute[72351]: DEBUG nova.compute.manager [req-f81f94d8-b201-4679-b4a3-984d47a866f7 req-22910b84-e0f8-456c-b55d-c31af97dc74e service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Detach interface failed, port_id=e021d67e-09ab-4779-b639-dff1ba39798a, reason: Instance 629e177e-afbb-4765-a0b5-7a3f66f72ebf could not be found. {{(pid=72351) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10866}}
Aug 09 02:20:30 user nova-compute[72351]: DEBUG nova.compute.manager [req-f81f94d8-b201-4679-b4a3-984d47a866f7 req-22910b84-e0f8-456c-b55d-c31af97dc74e service nova] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Received event network-vif-deleted-7e7b4229-7c23-4a73-9294-4c4fc9f6a01a {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:20:30 user nova-compute[72351]: DEBUG nova.compute.manager [req-f81f94d8-b201-4679-b4a3-984d47a866f7 req-22910b84-e0f8-456c-b55d-c31af97dc74e service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received event network-vif-deleted-c7c195c0-af76-445e-8a18-8169f0cdf5fc {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:20:30 user nova-compute[72351]: INFO nova.compute.manager [req-f81f94d8-b201-4679-b4a3-984d47a866f7 req-22910b84-e0f8-456c-b55d-c31af97dc74e service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Neutron deleted interface c7c195c0-af76-445e-8a18-8169f0cdf5fc; detaching it from the instance and deleting it from the info cache
Aug 09 02:20:30 user nova-compute[72351]: DEBUG nova.network.neutron [req-f81f94d8-b201-4679-b4a3-984d47a866f7 req-22910b84-e0f8-456c-b55d-c31af97dc74e service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Updating instance_info_cache with network_info: [{"id": "f5ecb594-7ac3-496b-9bb0-9f908b91c8fb", "address": "fa:16:3e:30:31:a4", "network": {"id": "4e0c5236-b30d-4213-b1b4-26c0f1a636eb", "bridge": "br-int", "label": "tempest-device-tagging-net1-1640959147", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.122", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf5ecb594-7a", "ovs_interfaceid": "f5ecb594-7ac3-496b-9bb0-9f908b91c8fb", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "2e9c8a9c-2d81-4b84-b481-9a6b513e730c", "address": "fa:16:3e:39:9b:9d", "network": {"id": "4e0c5236-b30d-4213-b1b4-26c0f1a636eb", "bridge": "br-int", "label": "tempest-device-tagging-net1-1640959147", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.75", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2e9c8a9c-2d", "ovs_interfaceid": "2e9c8a9c-2d81-4b84-b481-9a6b513e730c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "c2df74bf-22ce-48b1-9ab8-3a684cef7266", "address": "fa:16:3e:3f:be:14", "network": {"id": "4e0c5236-b30d-4213-b1b4-26c0f1a636eb", "bridge": "br-int", "label": "tempest-device-tagging-net1-1640959147", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.128", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "6908d784cc1a41dbbb4bee47e70c259b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc2df74bf-22", "ovs_interfaceid": "c2df74bf-22ce-48b1-9ab8-3a684cef7266", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:20:30 user nova-compute[72351]: INFO nova.scheduler.client.report [None req-5003d907-18be-42e6-97ab-15331b7d1b0b tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Deleted allocations for instance 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0
Aug 09 02:20:30 user nova-compute[72351]: DEBUG nova.compute.manager [req-f81f94d8-b201-4679-b4a3-984d47a866f7 req-22910b84-e0f8-456c-b55d-c31af97dc74e service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Detach interface failed, port_id=c7c195c0-af76-445e-8a18-8169f0cdf5fc, reason: Instance 629e177e-afbb-4765-a0b5-7a3f66f72ebf could not be found. {{(pid=72351) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10866}}
Aug 09 02:20:30 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:20:30 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Took 7.18 seconds to deallocate network for instance.
Aug 09 02:20:30 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5003d907-18be-42e6-97ab-15331b7d1b0b tempest-VolumesAssistedSnapshotsTest-1116923063 tempest-VolumesAssistedSnapshotsTest-1116923063-project-member] Lock "7c5e633d-a32f-4c5e-bb9d-1e40da964bd0" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 3.518s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:20:30 user nova-compute[72351]: DEBUG nova.network.neutron [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] Successfully updated port: 4bd2f461-68b0-4882-98d6-2bb99a324210 {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:20:30 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Acquiring lock "refresh_cache-32a28ecb-af4b-4c6e-8f2d-a51857134b34" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:20:30 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Acquired lock "refresh_cache-32a28ecb-af4b-4c6e-8f2d-a51857134b34" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:20:30 user nova-compute[72351]: DEBUG nova.network.neutron [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:20:31 user nova-compute[72351]: DEBUG nova.network.neutron [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:20:31 user nova-compute[72351]: DEBUG nova.compute.manager [req-e9af33ff-bcc1-4dc0-8f17-fcb6d83e2c3a req-4b987086-690f-49df-82f5-5ea52b411acd service nova] [instance: a1d1f738-0edf-4184-aa56-020c07830492] Received event network-vif-plugged-3e179747-c782-4b72-90d0-5b3bb68c3101 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:20:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-e9af33ff-bcc1-4dc0-8f17-fcb6d83e2c3a req-4b987086-690f-49df-82f5-5ea52b411acd service nova] Acquiring lock "a1d1f738-0edf-4184-aa56-020c07830492-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:20:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-e9af33ff-bcc1-4dc0-8f17-fcb6d83e2c3a req-4b987086-690f-49df-82f5-5ea52b411acd service nova] Lock "a1d1f738-0edf-4184-aa56-020c07830492-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:20:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-e9af33ff-bcc1-4dc0-8f17-fcb6d83e2c3a req-4b987086-690f-49df-82f5-5ea52b411acd service nova] Lock "a1d1f738-0edf-4184-aa56-020c07830492-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:20:31 user nova-compute[72351]: DEBUG nova.compute.manager [req-e9af33ff-bcc1-4dc0-8f17-fcb6d83e2c3a req-4b987086-690f-49df-82f5-5ea52b411acd service nova] [instance: a1d1f738-0edf-4184-aa56-020c07830492] No waiting events found dispatching network-vif-plugged-3e179747-c782-4b72-90d0-5b3bb68c3101 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:20:31 user nova-compute[72351]: WARNING nova.compute.manager [req-e9af33ff-bcc1-4dc0-8f17-fcb6d83e2c3a req-4b987086-690f-49df-82f5-5ea52b411acd service nova] [instance: a1d1f738-0edf-4184-aa56-020c07830492] Received unexpected event network-vif-plugged-3e179747-c782-4b72-90d0-5b3bb68c3101 for instance with vm_state building and task_state spawning.
Aug 09 02:20:31 user nova-compute[72351]: DEBUG nova.compute.manager [req-e9af33ff-bcc1-4dc0-8f17-fcb6d83e2c3a req-4b987086-690f-49df-82f5-5ea52b411acd service nova] [instance: a1d1f738-0edf-4184-aa56-020c07830492] Received event network-vif-plugged-3e179747-c782-4b72-90d0-5b3bb68c3101 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:20:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-e9af33ff-bcc1-4dc0-8f17-fcb6d83e2c3a req-4b987086-690f-49df-82f5-5ea52b411acd service nova] Acquiring lock "a1d1f738-0edf-4184-aa56-020c07830492-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:20:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-e9af33ff-bcc1-4dc0-8f17-fcb6d83e2c3a req-4b987086-690f-49df-82f5-5ea52b411acd service nova] Lock "a1d1f738-0edf-4184-aa56-020c07830492-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:20:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-e9af33ff-bcc1-4dc0-8f17-fcb6d83e2c3a req-4b987086-690f-49df-82f5-5ea52b411acd service nova] Lock "a1d1f738-0edf-4184-aa56-020c07830492-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:20:31 user nova-compute[72351]: DEBUG nova.compute.manager [req-e9af33ff-bcc1-4dc0-8f17-fcb6d83e2c3a req-4b987086-690f-49df-82f5-5ea52b411acd service nova] [instance: a1d1f738-0edf-4184-aa56-020c07830492] No waiting events found dispatching network-vif-plugged-3e179747-c782-4b72-90d0-5b3bb68c3101 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:20:31 user nova-compute[72351]: WARNING nova.compute.manager [req-e9af33ff-bcc1-4dc0-8f17-fcb6d83e2c3a req-4b987086-690f-49df-82f5-5ea52b411acd service nova] [instance: a1d1f738-0edf-4184-aa56-020c07830492] Received unexpected event network-vif-plugged-3e179747-c782-4b72-90d0-5b3bb68c3101 for instance with vm_state building and task_state spawning.
Aug 09 02:20:31 user nova-compute[72351]: DEBUG nova.compute.manager [req-e9af33ff-bcc1-4dc0-8f17-fcb6d83e2c3a req-4b987086-690f-49df-82f5-5ea52b411acd service nova] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] Received event network-changed-4bd2f461-68b0-4882-98d6-2bb99a324210 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:20:31 user nova-compute[72351]: DEBUG nova.compute.manager [req-e9af33ff-bcc1-4dc0-8f17-fcb6d83e2c3a req-4b987086-690f-49df-82f5-5ea52b411acd service nova] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] Refreshing instance network info cache due to event network-changed-4bd2f461-68b0-4882-98d6-2bb99a324210. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:20:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-e9af33ff-bcc1-4dc0-8f17-fcb6d83e2c3a req-4b987086-690f-49df-82f5-5ea52b411acd service nova] Acquiring lock "refresh_cache-32a28ecb-af4b-4c6e-8f2d-a51857134b34" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:20:31 user nova-compute[72351]: DEBUG nova.network.neutron [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] Updating instance_info_cache with network_info: [{"id": "4bd2f461-68b0-4882-98d6-2bb99a324210", "address": "fa:16:3e:15:97:f8", "network": {"id": "fc48221d-cf2e-4871-9741-c102904e10a6", "bridge": "br-int", "label": "tempest-VolumesActionsTest-857807030-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5209610aaaa34f7a8ffb7757dbfac85d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4bd2f461-68", "ovs_interfaceid": "4bd2f461-68b0-4882-98d6-2bb99a324210", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:20:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Releasing lock "refresh_cache-32a28ecb-af4b-4c6e-8f2d-a51857134b34" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:20:31 user nova-compute[72351]: DEBUG nova.compute.manager [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] Instance network_info: |[{"id": "4bd2f461-68b0-4882-98d6-2bb99a324210", "address": "fa:16:3e:15:97:f8", "network": {"id": "fc48221d-cf2e-4871-9741-c102904e10a6", "bridge": "br-int", "label": "tempest-VolumesActionsTest-857807030-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5209610aaaa34f7a8ffb7757dbfac85d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4bd2f461-68", "ovs_interfaceid": "4bd2f461-68b0-4882-98d6-2bb99a324210", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1971}}
Aug 09 02:20:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-e9af33ff-bcc1-4dc0-8f17-fcb6d83e2c3a req-4b987086-690f-49df-82f5-5ea52b411acd service nova] Acquired lock "refresh_cache-32a28ecb-af4b-4c6e-8f2d-a51857134b34" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:20:31 user nova-compute[72351]: DEBUG nova.network.neutron [req-e9af33ff-bcc1-4dc0-8f17-fcb6d83e2c3a req-4b987086-690f-49df-82f5-5ea52b411acd service nova] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] Refreshing network info cache for port 4bd2f461-68b0-4882-98d6-2bb99a324210 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:20:31 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] Start _get_guest_xml network_info=[{"id": "4bd2f461-68b0-4882-98d6-2bb99a324210", "address": "fa:16:3e:15:97:f8", "network": {"id": "fc48221d-cf2e-4871-9741-c102904e10a6", "bridge": "br-int", "label": "tempest-VolumesActionsTest-857807030-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5209610aaaa34f7a8ffb7757dbfac85d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4bd2f461-68", "ovs_interfaceid": "4bd2f461-68b0-4882-98d6-2bb99a324210", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'size': 0, 'device_type': 'disk', 'encryption_secret_uuid': None, 'guest_format': None, 'encryption_options': None, 'disk_bus': 'virtio', 'encrypted': False, 'encryption_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'image_id': '18ccda7f-31d3-4978-ab9d-4ed2f74eff94'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:20:31 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:20:31 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:20:31 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:20:31 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:20:31 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:20:31 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:20:31 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:20:31 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:20:31 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:20:31 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:20:31 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:20:31 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:20:31 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:20:31 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:20:31 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:20:31 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:20:31 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:20:31 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:20:31 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:20:27Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesActionsTest-instance-434546642',display_name='tempest-VolumesActionsTest-instance-434546642',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-volumesactionstest-instance-434546642',id=14,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='5209610aaaa34f7a8ffb7757dbfac85d',ramdisk_id='',reservation_id='r-4al0zprq',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesActionsTest-1343359891',owner_user_name='tempest-VolumesActionsTest-1343359891-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:20:28Z,user_data=None,user_id='00a45ab3cb2d43028418af1ae209da51',uuid=32a28ecb-af4b-4c6e-8f2d-a51857134b34,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "4bd2f461-68b0-4882-98d6-2bb99a324210", "address": "fa:16:3e:15:97:f8", "network": {"id": "fc48221d-cf2e-4871-9741-c102904e10a6", "bridge": "br-int", "label": "tempest-VolumesActionsTest-857807030-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5209610aaaa34f7a8ffb7757dbfac85d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4bd2f461-68", "ovs_interfaceid": "4bd2f461-68b0-4882-98d6-2bb99a324210", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:20:31 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Converting VIF {"id": "4bd2f461-68b0-4882-98d6-2bb99a324210", "address": "fa:16:3e:15:97:f8", "network": {"id": "fc48221d-cf2e-4871-9741-c102904e10a6", "bridge": "br-int", "label": "tempest-VolumesActionsTest-857807030-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5209610aaaa34f7a8ffb7757dbfac85d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4bd2f461-68", "ovs_interfaceid": "4bd2f461-68b0-4882-98d6-2bb99a324210", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:20:31 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:15:97:f8,bridge_name='br-int',has_traffic_filtering=True,id=4bd2f461-68b0-4882-98d6-2bb99a324210,network=Network(fc48221d-cf2e-4871-9741-c102904e10a6),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap4bd2f461-68') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:20:31 user nova-compute[72351]: DEBUG nova.objects.instance [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Lazy-loading 'pci_devices' on Instance uuid 32a28ecb-af4b-4c6e-8f2d-a51857134b34 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:20:31 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:20:31 user nova-compute[72351]:   <uuid>32a28ecb-af4b-4c6e-8f2d-a51857134b34</uuid>
Aug 09 02:20:31 user nova-compute[72351]:   <name>instance-0000000e</name>
Aug 09 02:20:31 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:20:31 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:20:31 user nova-compute[72351]:   <metadata>
Aug 09 02:20:31 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:20:31 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:20:31 user nova-compute[72351]:       <nova:name>tempest-VolumesActionsTest-instance-434546642</nova:name>
Aug 09 02:20:31 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:20:31</nova:creationTime>
Aug 09 02:20:31 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:20:31 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:20:31 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:20:31 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:20:31 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:20:31 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:20:31 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:20:31 user nova-compute[72351]:       <nova:owner>
Aug 09 02:20:31 user nova-compute[72351]:         <nova:user uuid="00a45ab3cb2d43028418af1ae209da51">tempest-VolumesActionsTest-1343359891-project-member</nova:user>
Aug 09 02:20:31 user nova-compute[72351]:         <nova:project uuid="5209610aaaa34f7a8ffb7757dbfac85d">tempest-VolumesActionsTest-1343359891</nova:project>
Aug 09 02:20:31 user nova-compute[72351]:       </nova:owner>
Aug 09 02:20:31 user nova-compute[72351]:       <nova:root type="image" uuid="18ccda7f-31d3-4978-ab9d-4ed2f74eff94"/>
Aug 09 02:20:31 user nova-compute[72351]:       <nova:ports>
Aug 09 02:20:31 user nova-compute[72351]:         <nova:port uuid="4bd2f461-68b0-4882-98d6-2bb99a324210">
Aug 09 02:20:31 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.8" ipVersion="4"/>
Aug 09 02:20:31 user nova-compute[72351]:         </nova:port>
Aug 09 02:20:31 user nova-compute[72351]:       </nova:ports>
Aug 09 02:20:31 user nova-compute[72351]:     </nova:instance>
Aug 09 02:20:31 user nova-compute[72351]:   </metadata>
Aug 09 02:20:31 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:20:31 user nova-compute[72351]:     <system>
Aug 09 02:20:31 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:20:31 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:20:31 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:20:31 user nova-compute[72351]:       <entry name="serial">32a28ecb-af4b-4c6e-8f2d-a51857134b34</entry>
Aug 09 02:20:31 user nova-compute[72351]:       <entry name="uuid">32a28ecb-af4b-4c6e-8f2d-a51857134b34</entry>
Aug 09 02:20:31 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:20:31 user nova-compute[72351]:     </system>
Aug 09 02:20:31 user nova-compute[72351]:   </sysinfo>
Aug 09 02:20:31 user nova-compute[72351]:   <os>
Aug 09 02:20:31 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:20:31 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:20:31 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:20:31 user nova-compute[72351]:   </os>
Aug 09 02:20:31 user nova-compute[72351]:   <features>
Aug 09 02:20:31 user nova-compute[72351]:     <acpi/>
Aug 09 02:20:31 user nova-compute[72351]:     <apic/>
Aug 09 02:20:31 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:20:31 user nova-compute[72351]:   </features>
Aug 09 02:20:31 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:20:31 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:20:31 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:20:31 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:20:31 user nova-compute[72351]:   </clock>
Aug 09 02:20:31 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:20:31 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:20:31 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:20:31 user nova-compute[72351]:   </cpu>
Aug 09 02:20:31 user nova-compute[72351]:   <devices>
Aug 09 02:20:31 user nova-compute[72351]:     <disk type="file" device="disk">
Aug 09 02:20:31 user nova-compute[72351]:       <driver name="qemu" type="qcow2" cache="none"/>
Aug 09 02:20:31 user nova-compute[72351]:       <source file="/opt/stack/data/nova/instances/32a28ecb-af4b-4c6e-8f2d-a51857134b34/disk"/>
Aug 09 02:20:31 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:20:31 user nova-compute[72351]:     </disk>
Aug 09 02:20:31 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:20:31 user nova-compute[72351]:       <mac address="fa:16:3e:15:97:f8"/>
Aug 09 02:20:31 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:20:31 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:20:31 user nova-compute[72351]:       <target dev="tap4bd2f461-68"/>
Aug 09 02:20:31 user nova-compute[72351]:     </interface>
Aug 09 02:20:31 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:20:31 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/32a28ecb-af4b-4c6e-8f2d-a51857134b34/console.log" append="off"/>
Aug 09 02:20:31 user nova-compute[72351]:     </serial>
Aug 09 02:20:31 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:20:31 user nova-compute[72351]:     <video>
Aug 09 02:20:31 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:20:31 user nova-compute[72351]:     </video>
Aug 09 02:20:31 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:20:31 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:20:31 user nova-compute[72351]:     </rng>
Aug 09 02:20:31 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:20:31 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:20:31 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:20:31 user nova-compute[72351]:     </memballoon>
Aug 09 02:20:31 user nova-compute[72351]:   </devices>
Aug 09 02:20:31 user nova-compute[72351]: </domain>
Aug 09 02:20:31 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:20:31 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:20:27Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesActionsTest-instance-434546642',display_name='tempest-VolumesActionsTest-instance-434546642',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-volumesactionstest-instance-434546642',id=14,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='5209610aaaa34f7a8ffb7757dbfac85d',ramdisk_id='',reservation_id='r-4al0zprq',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesActionsTest-1343359891',owner_user_name='tempest-VolumesActionsTest-1343359891-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:20:28Z,user_data=None,user_id='00a45ab3cb2d43028418af1ae209da51',uuid=32a28ecb-af4b-4c6e-8f2d-a51857134b34,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "4bd2f461-68b0-4882-98d6-2bb99a324210", "address": "fa:16:3e:15:97:f8", "network": {"id": "fc48221d-cf2e-4871-9741-c102904e10a6", "bridge": "br-int", "label": "tempest-VolumesActionsTest-857807030-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5209610aaaa34f7a8ffb7757dbfac85d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4bd2f461-68", "ovs_interfaceid": "4bd2f461-68b0-4882-98d6-2bb99a324210", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:20:31 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Converting VIF {"id": "4bd2f461-68b0-4882-98d6-2bb99a324210", "address": "fa:16:3e:15:97:f8", "network": {"id": "fc48221d-cf2e-4871-9741-c102904e10a6", "bridge": "br-int", "label": "tempest-VolumesActionsTest-857807030-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5209610aaaa34f7a8ffb7757dbfac85d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4bd2f461-68", "ovs_interfaceid": "4bd2f461-68b0-4882-98d6-2bb99a324210", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:20:31 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:15:97:f8,bridge_name='br-int',has_traffic_filtering=True,id=4bd2f461-68b0-4882-98d6-2bb99a324210,network=Network(fc48221d-cf2e-4871-9741-c102904e10a6),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap4bd2f461-68') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:20:31 user nova-compute[72351]: DEBUG os_vif [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:15:97:f8,bridge_name='br-int',has_traffic_filtering=True,id=4bd2f461-68b0-4882-98d6-2bb99a324210,network=Network(fc48221d-cf2e-4871-9741-c102904e10a6),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap4bd2f461-68') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:20:31 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:31 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:20:31 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:20:31 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:31 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap4bd2f461-68, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:20:31 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap4bd2f461-68, col_values=(('external_ids', {'iface-id': '4bd2f461-68b0-4882-98d6-2bb99a324210', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:15:97:f8', 'vm-uuid': '32a28ecb-af4b-4c6e-8f2d-a51857134b34'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:20:31 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:31 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:31 user nova-compute[72351]: INFO os_vif [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:15:97:f8,bridge_name='br-int',has_traffic_filtering=True,id=4bd2f461-68b0-4882-98d6-2bb99a324210,network=Network(fc48221d-cf2e-4871-9741-c102904e10a6),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap4bd2f461-68')
Aug 09 02:20:31 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:20:31 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] No VIF found with MAC fa:16:3e:15:97:f8, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:20:31 user nova-compute[72351]: DEBUG nova.network.neutron [req-e9af33ff-bcc1-4dc0-8f17-fcb6d83e2c3a req-4b987086-690f-49df-82f5-5ea52b411acd service nova] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] Updated VIF entry in instance network info cache for port 4bd2f461-68b0-4882-98d6-2bb99a324210. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:20:31 user nova-compute[72351]: DEBUG nova.network.neutron [req-e9af33ff-bcc1-4dc0-8f17-fcb6d83e2c3a req-4b987086-690f-49df-82f5-5ea52b411acd service nova] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] Updating instance_info_cache with network_info: [{"id": "4bd2f461-68b0-4882-98d6-2bb99a324210", "address": "fa:16:3e:15:97:f8", "network": {"id": "fc48221d-cf2e-4871-9741-c102904e10a6", "bridge": "br-int", "label": "tempest-VolumesActionsTest-857807030-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5209610aaaa34f7a8ffb7757dbfac85d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4bd2f461-68", "ovs_interfaceid": "4bd2f461-68b0-4882-98d6-2bb99a324210", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:20:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-e9af33ff-bcc1-4dc0-8f17-fcb6d83e2c3a req-4b987086-690f-49df-82f5-5ea52b411acd service nova] Releasing lock "refresh_cache-32a28ecb-af4b-4c6e-8f2d-a51857134b34" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:20:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Acquiring lock "97b13ccd-15b4-44bf-b865-71b4da861168" by "nova.compute.manager.ComputeManager.reboot_instance.<locals>.do_reboot_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:20:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "97b13ccd-15b4-44bf-b865-71b4da861168" acquired by "nova.compute.manager.ComputeManager.reboot_instance.<locals>.do_reboot_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:20:31 user nova-compute[72351]: INFO nova.compute.manager [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Rebooting instance
Aug 09 02:20:32 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Acquiring lock "refresh_cache-97b13ccd-15b4-44bf-b865-71b4da861168" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:20:32 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Acquired lock "refresh_cache-97b13ccd-15b4-44bf-b865-71b4da861168" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:20:32 user nova-compute[72351]: DEBUG nova.network.neutron [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:20:32 user nova-compute[72351]: DEBUG nova.compute.manager [req-d5091f32-eb16-4b1a-bcb9-9b69ff4fdbaf req-f025b046-c6e5-47b8-8987-a91346e81dda service nova] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Received event network-vif-deleted-c2df74bf-22ce-48b1-9ab8-3a684cef7266 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:20:32 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547632.180351, a1d1f738-0edf-4184-aa56-020c07830492 => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:20:32 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: a1d1f738-0edf-4184-aa56-020c07830492] VM Resumed (Lifecycle Event)
Aug 09 02:20:32 user nova-compute[72351]: DEBUG nova.compute.manager [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: a1d1f738-0edf-4184-aa56-020c07830492] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:20:32 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: a1d1f738-0edf-4184-aa56-020c07830492] Guest created on hypervisor {{(pid=72351) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4411}}
Aug 09 02:20:32 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: a1d1f738-0edf-4184-aa56-020c07830492] Instance spawned successfully.
Aug 09 02:20:32 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: a1d1f738-0edf-4184-aa56-020c07830492] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:894}}
Aug 09 02:20:32 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: a1d1f738-0edf-4184-aa56-020c07830492] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:20:32 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: a1d1f738-0edf-4184-aa56-020c07830492] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:20:32 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: a1d1f738-0edf-4184-aa56-020c07830492] Found default for hw_cdrom_bus of ide {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:20:32 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: a1d1f738-0edf-4184-aa56-020c07830492] Found default for hw_disk_bus of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:20:32 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: a1d1f738-0edf-4184-aa56-020c07830492] Found default for hw_input_bus of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:20:32 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: a1d1f738-0edf-4184-aa56-020c07830492] Found default for hw_pointer_model of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:20:32 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: a1d1f738-0edf-4184-aa56-020c07830492] Found default for hw_video_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:20:32 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: a1d1f738-0edf-4184-aa56-020c07830492] Found default for hw_vif_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:20:32 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: a1d1f738-0edf-4184-aa56-020c07830492] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:20:32 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547632.1824439, a1d1f738-0edf-4184-aa56-020c07830492 => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:20:32 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: a1d1f738-0edf-4184-aa56-020c07830492] VM Started (Lifecycle Event)
Aug 09 02:20:32 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: a1d1f738-0edf-4184-aa56-020c07830492] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:20:32 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: a1d1f738-0edf-4184-aa56-020c07830492] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:20:32 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: a1d1f738-0edf-4184-aa56-020c07830492] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:20:32 user nova-compute[72351]: INFO nova.compute.manager [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: a1d1f738-0edf-4184-aa56-020c07830492] Took 7.20 seconds to spawn the instance on the hypervisor.
Aug 09 02:20:32 user nova-compute[72351]: DEBUG nova.compute.manager [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: a1d1f738-0edf-4184-aa56-020c07830492] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:20:32 user nova-compute[72351]: INFO nova.compute.manager [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: a1d1f738-0edf-4184-aa56-020c07830492] Took 8.30 seconds to build instance.
Aug 09 02:20:32 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-450d6472-acb8-40ac-85d1-1729f82f99cb tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Lock "a1d1f738-0edf-4184-aa56-020c07830492" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 8.391s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:20:32 user nova-compute[72351]: DEBUG nova.network.neutron [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Updating instance_info_cache with network_info: [{"id": "74c98543-ce04-43d8-a58c-267e372db698", "address": "fa:16:3e:d4:4d:0a", "network": {"id": "7024d1f6-a400-4482-9364-78ae3b6adc80", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1603258958-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.62", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c2f67da658d94fabbab82ee6a6171c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap74c98543-ce", "ovs_interfaceid": "74c98543-ce04-43d8-a58c-267e372db698", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:20:32 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Releasing lock "refresh_cache-97b13ccd-15b4-44bf-b865-71b4da861168" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:20:32 user nova-compute[72351]: DEBUG nova.compute.manager [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:20:32 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:32 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:32 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:32 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:32 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:32 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:32 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:32 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:33 user nova-compute[72351]: INFO nova.compute.manager [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Took 2.92 seconds to detach 3 volumes for instance.
Aug 09 02:20:33 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:20:33 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:20:33 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:20:33 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:20:33 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Instance destroyed successfully.
Aug 09 02:20:33 user nova-compute[72351]: DEBUG nova.objects.instance [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lazy-loading 'resources' on Instance uuid 97b13ccd-15b4-44bf-b865-71b4da861168 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:20:33 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:18:04Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestMinimumBasicScenario-server-1233822315',display_name='tempest-TestMinimumBasicScenario-server-1233822315',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testminimumbasicscenario-server-1233822315',id=8,image_ref='9eadc1b0-9cb8-4bc1-97d6-af86e590c548',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOk8lLSpSCaCDIKlT4lhWRaHWy+oi2s0IJNKXHyKayH1X4sCjRLnVKeip/ShH5Av4ZZCopMp+S7MD8OAGw8MyvDsE398yCbkjnI/7XCIasyAoPOQXBUUZ52swhczfVlPLg==',key_name='tempest-TestMinimumBasicScenario-1974156570',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:18:12Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='c2f67da658d94fabbab82ee6a6171c4b',ramdisk_id='',reservation_id='r-rhqfwdu0',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='9eadc1b0-9cb8-4bc1-97d6-af86e590c548',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',owner_project_name='tempest-TestMinimumBasicScenario-1380132717',owner_user_name='tempest-TestMinimumBasicScenario-1380132717-project-member'},tags=<?>,task_state='reboot_started_hard',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:20:33Z,user_data=None,user_id='bec5b5704e9b4e3480f428d5c037c20d',uuid=97b13ccd-15b4-44bf-b865-71b4da861168,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "74c98543-ce04-43d8-a58c-267e372db698", "address": "fa:16:3e:d4:4d:0a", "network": {"id": "7024d1f6-a400-4482-9364-78ae3b6adc80", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1603258958-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.62", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c2f67da658d94fabbab82ee6a6171c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap74c98543-ce", "ovs_interfaceid": "74c98543-ce04-43d8-a58c-267e372db698", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:20:33 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Converting VIF {"id": "74c98543-ce04-43d8-a58c-267e372db698", "address": "fa:16:3e:d4:4d:0a", "network": {"id": "7024d1f6-a400-4482-9364-78ae3b6adc80", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1603258958-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.62", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c2f67da658d94fabbab82ee6a6171c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap74c98543-ce", "ovs_interfaceid": "74c98543-ce04-43d8-a58c-267e372db698", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:20:33 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:d4:4d:0a,bridge_name='br-int',has_traffic_filtering=True,id=74c98543-ce04-43d8-a58c-267e372db698,network=Network(7024d1f6-a400-4482-9364-78ae3b6adc80),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap74c98543-ce') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:20:33 user nova-compute[72351]: DEBUG os_vif [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:d4:4d:0a,bridge_name='br-int',has_traffic_filtering=True,id=74c98543-ce04-43d8-a58c-267e372db698,network=Network(7024d1f6-a400-4482-9364-78ae3b6adc80),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap74c98543-ce') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:20:33 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.413s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:20:33 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:33 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap74c98543-ce, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:20:33 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:33 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:20:33 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:33 user nova-compute[72351]: INFO os_vif [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:d4:4d:0a,bridge_name='br-int',has_traffic_filtering=True,id=74c98543-ce04-43d8-a58c-267e372db698,network=Network(7024d1f6-a400-4482-9364-78ae3b6adc80),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap74c98543-ce')
Aug 09 02:20:33 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 02:20:33 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'cIMBpXdn', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdh'}, None), 'kwargs': True}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:20:33 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:20:33 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:20:33 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.003s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:20:33 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 02:20:33 user nova-compute[72351]: INFO nova.scheduler.client.report [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Deleted allocations for instance 629e177e-afbb-4765-a0b5-7a3f66f72ebf
Aug 09 02:20:33 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:33 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:33 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[5d4dd25c-e7d7-4a15-977e-b2305ef25be8]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:33 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:33 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:33 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:33 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[b0d94216-18b7-43c3-ae28-01d8e095fe60]: (4, ('tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:33 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f (non-flash)
Aug 09 02:20:33 user nova-compute[72351]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:20:33 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:20:33 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] iscsi session list stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f (non-flash)
Aug 09 02:20:33 user nova-compute[72351]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:20:33 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:20:33 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fcfdc1a4430>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f'): ({'sdh'}, set())}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 02:20:33 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Removing single pathed devices sdh {{(pid=72351) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Aug 09 02:20:33 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:33 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:33 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:33 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[46267479-cefa-4c6c-9b69-173f1b13f347]: (4, ('path checker states:\nup                  1\npending             1\n\npaths: 0\nbusy: False\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:33 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-99c475a0-b11d-4dea-b27b-195efeaa9c92 tempest-TaggedBootDevicesTest-1303879303 tempest-TaggedBootDevicesTest-1303879303-project-member] Lock "629e177e-afbb-4765-a0b5-7a3f66f72ebf" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 14.101s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:20:33 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdh {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:33 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdh" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:33 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[462c72e8-46fe-4b53-92f8-20f76badc3fd]: (4, ('ok\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:33 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Flushing IO for device /dev/sdh {{(pid=72351) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Aug 09 02:20:33 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdh {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:33 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdh" returned: 0 in 0.014s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:33 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e21927ef-b5a1-4b75-8fcc-0b95744a01c7]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:33 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Remove SCSI device /dev/sdh with /sys/block/sdh/device/delete {{(pid=72351) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Aug 09 02:20:33 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdh/device/delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:33 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdh/device/delete" returned: 0 in 0.046s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:33 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[1c75c18f-e262-427f-a959-28c95640fe71]: (4, ('1', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:33 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Checking to see if SCSI volumes sdh have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Aug 09 02:20:33 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] SCSI volumes sdh have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Aug 09 02:20:33 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f')] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 02:20:33 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:33 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:33 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[278b29ff-d55a-463f-9a9d-4f3099bacd26]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:33 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:20:33 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f -p 172.16.0.220:3260 --logout {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f -p 172.16.0.220:3260 --logout" returned: 0 in 0.057s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ca8e48ff-531d-4d5b-a2a3-fc3b6343eacc]: (4, ('Logging out of session [sid: 13, target: iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f, portal: 172.16.0.220,3260]\nLogout of [sid: 13, target: iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 13, target: iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f, portal: 172.16.0.220,3260]
Aug 09 02:20:34 user nova-compute[72351]: Logout of [sid: 13, target: iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f, portal: 172.16.0.220,3260] successful.
Aug 09 02:20:34 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f -p 172.16.0.220:3260 --op delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f -p 172.16.0.220:3260 --op delete" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d9101239-3f74-47cd-af30-f78cb8d51da0]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.311s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] <== disconnect_volume: return (314ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Start _get_guest_xml network_info=[{"id": "74c98543-ce04-43d8-a58c-267e372db698", "address": "fa:16:3e:d4:4d:0a", "network": {"id": "7024d1f6-a400-4482-9364-78ae3b6adc80", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1603258958-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.62", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c2f67da658d94fabbab82ee6a6171c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap74c98543-ce", "ovs_interfaceid": "74c98543-ce04-43d8-a58c-267e372db698", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vdb': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} image_meta=ImageMeta(checksum=<?>,container_format='bare',created_at=<?>,direct_url=<?>,disk_format='qcow2',id=9eadc1b0-9cb8-4bc1-97d6-af86e590c548,min_disk=1,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=<?>,status=<?>,tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'size': 0, 'device_type': 'disk', 'encryption_secret_uuid': None, 'guest_format': None, 'encryption_options': None, 'disk_bus': 'virtio', 'encrypted': False, 'encryption_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'image_id': '9eadc1b0-9cb8-4bc1-97d6-af86e590c548'}], 'ephemerals': [], 'block_device_mapping': [{'device_type': 'disk', 'guest_format': None, 'disk_bus': 'virtio', 'mount_device': '/dev/vdb', 'boot_index': None, 'attachment_id': '37337a2e-29d1-4999-8d68-3d7a65691f1e', 'delete_on_termination': False, 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqns': ['iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'cIMBpXdn', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdh'}, 'status': 'reserved', 'instance': '97b13ccd-15b4-44bf-b865-71b4da861168', 'attached_at': '', 'detached_at': '', 'volume_id': 'cda288d2-cdeb-4b8f-abbf-acaa3fab484f', 'serial': 'cda288d2-cdeb-4b8f-abbf-acaa3fab484f'}, 'volume_type': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:20:34 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:20:34 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:20:34 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format='bare',created_at=<?>,direct_url=<?>,disk_format='qcow2',id=9eadc1b0-9cb8-4bc1-97d6-af86e590c548,min_disk=1,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=<?>,status=<?>,tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG nova.objects.instance [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lazy-loading 'vcpu_model' on Instance uuid 97b13ccd-15b4-44bf-b865-71b4da861168 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'cIMBpXdn', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path'}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:20:34 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:20:34 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:20:34 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:20:34 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:20:34 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 02:20:34 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f -p 172.16.0.220:3260" returned: 21 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[68dc4d70-c75b-496c-80c4-36a0b0efefdb]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Aug 09 02:20:34 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f -p 172.16.0.220:3260 --interface default --op new {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[65180987-abd0-44e8-aca6-b31023becf5b]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f] added\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f] added
Aug 09 02:20:34 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.027s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[25904437-ccf7-4f9b-b137-5bceb6bdaac1]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[c07b5628-548f-4a19-9dc1-068b6cc8c0f0]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f -p 172.16.0.220:3260 --op update -n node.session.auth.username -v cIMBpXdn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f -p 172.16.0.220:3260 --op update -n node.session.auth.username -v cIMBpXdn" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[770a5827-a830-496a-9f45-2052ad5dd768]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'cIMBpXdn'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[241ce9a2-5ef6-4766-9382-56458931d0be]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[0fc29f49-568d-4082-a728-3557e1f8bafc]: (4, ('tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:20:34 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] iscsi session list stdout=tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:20:34 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f -p 172.16.0.220:3260 --login {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f -p 172.16.0.220:3260 --login" returned: 0 in 0.065s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[424263c4-3041-47a9-8bbe-61c986a2616d]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f, portal: 172.16.0.220,3260]
Aug 09 02:20:34 user nova-compute[72351]: Login to [iface: default, target: iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f, portal: 172.16.0.220,3260] successful.
Aug 09 02:20:34 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[4efebe21-4754-4733-87aa-c7a012da16b4]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.027s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[2907a2db-cdcc-4807-83ad-9b12ae50cc1a]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f (non-flash)
Aug 09 02:20:34 user nova-compute[72351]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:20:34 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f (non-flash)
Aug 09 02:20:34 user nova-compute[72351]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:20:34 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.305s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] HCTL ('33', '-', '-', 0) found on session 14 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[be2324f7-e295-406b-9755-d9003c1cc62b]: (4, ('- - 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Searching for a device in session 14 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Connected to sdb using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'cIMBpXdn', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdh', 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG nova.compute.manager [req-588f085e-84ce-4f8b-a3e6-10b0dde039ad req-3a0611d4-7974-492b-9e72-38c20d7ce816 service nova] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Received event network-vif-unplugged-74c98543-ce04-43d8-a58c-267e372db698 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-588f085e-84ce-4f8b-a3e6-10b0dde039ad req-3a0611d4-7974-492b-9e72-38c20d7ce816 service nova] Acquiring lock "97b13ccd-15b4-44bf-b865-71b4da861168-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-588f085e-84ce-4f8b-a3e6-10b0dde039ad req-3a0611d4-7974-492b-9e72-38c20d7ce816 service nova] Lock "97b13ccd-15b4-44bf-b865-71b4da861168-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-588f085e-84ce-4f8b-a3e6-10b0dde039ad req-3a0611d4-7974-492b-9e72-38c20d7ce816 service nova] Lock "97b13ccd-15b4-44bf-b865-71b4da861168-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG nova.compute.manager [req-588f085e-84ce-4f8b-a3e6-10b0dde039ad req-3a0611d4-7974-492b-9e72-38c20d7ce816 service nova] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] No waiting events found dispatching network-vif-unplugged-74c98543-ce04-43d8-a58c-267e372db698 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:20:34 user nova-compute[72351]: WARNING nova.compute.manager [req-588f085e-84ce-4f8b-a3e6-10b0dde039ad req-3a0611d4-7974-492b-9e72-38c20d7ce816 service nova] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Received unexpected event network-vif-unplugged-74c98543-ce04-43d8-a58c-267e372db698 for instance with vm_state active and task_state reboot_started_hard.
Aug 09 02:20:34 user nova-compute[72351]: DEBUG nova.compute.manager [req-588f085e-84ce-4f8b-a3e6-10b0dde039ad req-3a0611d4-7974-492b-9e72-38c20d7ce816 service nova] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Received event network-vif-plugged-74c98543-ce04-43d8-a58c-267e372db698 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-588f085e-84ce-4f8b-a3e6-10b0dde039ad req-3a0611d4-7974-492b-9e72-38c20d7ce816 service nova] Acquiring lock "97b13ccd-15b4-44bf-b865-71b4da861168-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-588f085e-84ce-4f8b-a3e6-10b0dde039ad req-3a0611d4-7974-492b-9e72-38c20d7ce816 service nova] Lock "97b13ccd-15b4-44bf-b865-71b4da861168-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-588f085e-84ce-4f8b-a3e6-10b0dde039ad req-3a0611d4-7974-492b-9e72-38c20d7ce816 service nova] Lock "97b13ccd-15b4-44bf-b865-71b4da861168-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG nova.compute.manager [req-588f085e-84ce-4f8b-a3e6-10b0dde039ad req-3a0611d4-7974-492b-9e72-38c20d7ce816 service nova] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] No waiting events found dispatching network-vif-plugged-74c98543-ce04-43d8-a58c-267e372db698 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:20:34 user nova-compute[72351]: WARNING nova.compute.manager [req-588f085e-84ce-4f8b-a3e6-10b0dde039ad req-3a0611d4-7974-492b-9e72-38c20d7ce816 service nova] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Received unexpected event network-vif-plugged-74c98543-ce04-43d8-a58c-267e372db698 for instance with vm_state active and task_state reboot_started_hard.
Aug 09 02:20:34 user nova-compute[72351]: DEBUG nova.compute.manager [req-588f085e-84ce-4f8b-a3e6-10b0dde039ad req-3a0611d4-7974-492b-9e72-38c20d7ce816 service nova] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] Received event network-vif-plugged-4bd2f461-68b0-4882-98d6-2bb99a324210 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-588f085e-84ce-4f8b-a3e6-10b0dde039ad req-3a0611d4-7974-492b-9e72-38c20d7ce816 service nova] Acquiring lock "32a28ecb-af4b-4c6e-8f2d-a51857134b34-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-588f085e-84ce-4f8b-a3e6-10b0dde039ad req-3a0611d4-7974-492b-9e72-38c20d7ce816 service nova] Lock "32a28ecb-af4b-4c6e-8f2d-a51857134b34-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-588f085e-84ce-4f8b-a3e6-10b0dde039ad req-3a0611d4-7974-492b-9e72-38c20d7ce816 service nova] Lock "32a28ecb-af4b-4c6e-8f2d-a51857134b34-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG nova.compute.manager [req-588f085e-84ce-4f8b-a3e6-10b0dde039ad req-3a0611d4-7974-492b-9e72-38c20d7ce816 service nova] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] No waiting events found dispatching network-vif-plugged-4bd2f461-68b0-4882-98d6-2bb99a324210 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:20:34 user nova-compute[72351]: WARNING nova.compute.manager [req-588f085e-84ce-4f8b-a3e6-10b0dde039ad req-3a0611d4-7974-492b-9e72-38c20d7ce816 service nova] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] Received unexpected event network-vif-plugged-4bd2f461-68b0-4882-98d6-2bb99a324210 for instance with vm_state building and task_state spawning.
Aug 09 02:20:34 user nova-compute[72351]: DEBUG nova.compute.manager [req-588f085e-84ce-4f8b-a3e6-10b0dde039ad req-3a0611d4-7974-492b-9e72-38c20d7ce816 service nova] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] Received event network-vif-plugged-4bd2f461-68b0-4882-98d6-2bb99a324210 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-588f085e-84ce-4f8b-a3e6-10b0dde039ad req-3a0611d4-7974-492b-9e72-38c20d7ce816 service nova] Acquiring lock "32a28ecb-af4b-4c6e-8f2d-a51857134b34-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-588f085e-84ce-4f8b-a3e6-10b0dde039ad req-3a0611d4-7974-492b-9e72-38c20d7ce816 service nova] Lock "32a28ecb-af4b-4c6e-8f2d-a51857134b34-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-588f085e-84ce-4f8b-a3e6-10b0dde039ad req-3a0611d4-7974-492b-9e72-38c20d7ce816 service nova] Lock "32a28ecb-af4b-4c6e-8f2d-a51857134b34-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:20:34 user nova-compute[72351]: DEBUG nova.compute.manager [req-588f085e-84ce-4f8b-a3e6-10b0dde039ad req-3a0611d4-7974-492b-9e72-38c20d7ce816 service nova] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] No waiting events found dispatching network-vif-plugged-4bd2f461-68b0-4882-98d6-2bb99a324210 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:20:34 user nova-compute[72351]: WARNING nova.compute.manager [req-588f085e-84ce-4f8b-a3e6-10b0dde039ad req-3a0611d4-7974-492b-9e72-38c20d7ce816 service nova] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] Received unexpected event network-vif-plugged-4bd2f461-68b0-4882-98d6-2bb99a324210 for instance with vm_state building and task_state spawning.
Aug 09 02:20:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-9e64a2ea-7793-4e1a-a5f5-e23d770c0b2f tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquiring lock "a5ca6400-1125-431e-8a91-057e746d40b5" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:20:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-9e64a2ea-7793-4e1a-a5f5-e23d770c0b2f tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "a5ca6400-1125-431e-8a91-057e746d40b5" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:20:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-9e64a2ea-7793-4e1a-a5f5-e23d770c0b2f tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquiring lock "a5ca6400-1125-431e-8a91-057e746d40b5-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:20:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-9e64a2ea-7793-4e1a-a5f5-e23d770c0b2f tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "a5ca6400-1125-431e-8a91-057e746d40b5-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:20:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-9e64a2ea-7793-4e1a-a5f5-e23d770c0b2f tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "a5ca6400-1125-431e-8a91-057e746d40b5-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:20:35 user nova-compute[72351]: INFO nova.compute.manager [None req-9e64a2ea-7793-4e1a-a5f5-e23d770c0b2f tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Terminating instance
Aug 09 02:20:35 user nova-compute[72351]: DEBUG nova.compute.manager [None req-9e64a2ea-7793-4e1a-a5f5-e23d770c0b2f tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Start destroying the instance on the hypervisor. {{(pid=72351) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3124}}
Aug 09 02:20:35 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:35 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:35 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:35 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:35 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:35 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.346s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:20:35 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] <== connect_volume: return (1347ms) {'type': 'block', 'scsi_wwn': '23130366333646234', 'path': '/dev/sdb'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:20:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23130366333646234', 'path': '/dev/sdb'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 02:20:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:18:04Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestMinimumBasicScenario-server-1233822315',display_name='tempest-TestMinimumBasicScenario-server-1233822315',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testminimumbasicscenario-server-1233822315',id=8,image_ref='9eadc1b0-9cb8-4bc1-97d6-af86e590c548',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOk8lLSpSCaCDIKlT4lhWRaHWy+oi2s0IJNKXHyKayH1X4sCjRLnVKeip/ShH5Av4ZZCopMp+S7MD8OAGw8MyvDsE398yCbkjnI/7XCIasyAoPOQXBUUZ52swhczfVlPLg==',key_name='tempest-TestMinimumBasicScenario-1974156570',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:18:12Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='c2f67da658d94fabbab82ee6a6171c4b',ramdisk_id='',reservation_id='r-rhqfwdu0',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='9eadc1b0-9cb8-4bc1-97d6-af86e590c548',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',owner_project_name='tempest-TestMinimumBasicScenario-1380132717',owner_user_name='tempest-TestMinimumBasicScenario-1380132717-project-member'},tags=<?>,task_state='reboot_started_hard',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:20:33Z,user_data=None,user_id='bec5b5704e9b4e3480f428d5c037c20d',uuid=97b13ccd-15b4-44bf-b865-71b4da861168,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "74c98543-ce04-43d8-a58c-267e372db698", "address": "fa:16:3e:d4:4d:0a", "network": {"id": "7024d1f6-a400-4482-9364-78ae3b6adc80", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1603258958-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.62", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c2f67da658d94fabbab82ee6a6171c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap74c98543-ce", "ovs_interfaceid": "74c98543-ce04-43d8-a58c-267e372db698", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:20:35 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Converting VIF {"id": "74c98543-ce04-43d8-a58c-267e372db698", "address": "fa:16:3e:d4:4d:0a", "network": {"id": "7024d1f6-a400-4482-9364-78ae3b6adc80", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1603258958-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.62", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c2f67da658d94fabbab82ee6a6171c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap74c98543-ce", "ovs_interfaceid": "74c98543-ce04-43d8-a58c-267e372db698", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:20:35 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:d4:4d:0a,bridge_name='br-int',has_traffic_filtering=True,id=74c98543-ce04-43d8-a58c-267e372db698,network=Network(7024d1f6-a400-4482-9364-78ae3b6adc80),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap74c98543-ce') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:20:35 user nova-compute[72351]: DEBUG nova.objects.instance [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lazy-loading 'pci_devices' on Instance uuid 97b13ccd-15b4-44bf-b865-71b4da861168 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:20:35 user nova-compute[72351]: DEBUG nova.compute.manager [req-b5c6133b-6f5e-4250-b7a3-8842fcceac86 req-fb1c6021-588e-40cf-a825-7405a1a78b03 service nova] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Received event network-vif-unplugged-49802a57-3939-4cc2-956b-3d396b7c738e {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:20:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-b5c6133b-6f5e-4250-b7a3-8842fcceac86 req-fb1c6021-588e-40cf-a825-7405a1a78b03 service nova] Acquiring lock "a5ca6400-1125-431e-8a91-057e746d40b5-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:20:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-b5c6133b-6f5e-4250-b7a3-8842fcceac86 req-fb1c6021-588e-40cf-a825-7405a1a78b03 service nova] Lock "a5ca6400-1125-431e-8a91-057e746d40b5-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:20:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-b5c6133b-6f5e-4250-b7a3-8842fcceac86 req-fb1c6021-588e-40cf-a825-7405a1a78b03 service nova] Lock "a5ca6400-1125-431e-8a91-057e746d40b5-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:20:35 user nova-compute[72351]: DEBUG nova.compute.manager [req-b5c6133b-6f5e-4250-b7a3-8842fcceac86 req-fb1c6021-588e-40cf-a825-7405a1a78b03 service nova] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] No waiting events found dispatching network-vif-unplugged-49802a57-3939-4cc2-956b-3d396b7c738e {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:20:35 user nova-compute[72351]: DEBUG nova.compute.manager [req-b5c6133b-6f5e-4250-b7a3-8842fcceac86 req-fb1c6021-588e-40cf-a825-7405a1a78b03 service nova] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Received event network-vif-unplugged-49802a57-3939-4cc2-956b-3d396b7c738e for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:20:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:20:35 user nova-compute[72351]:   <uuid>97b13ccd-15b4-44bf-b865-71b4da861168</uuid>
Aug 09 02:20:35 user nova-compute[72351]:   <name>instance-00000008</name>
Aug 09 02:20:35 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:20:35 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:20:35 user nova-compute[72351]:   <metadata>
Aug 09 02:20:35 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:20:35 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:20:35 user nova-compute[72351]:       <nova:name>tempest-TestMinimumBasicScenario-server-1233822315</nova:name>
Aug 09 02:20:35 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:20:34</nova:creationTime>
Aug 09 02:20:35 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:20:35 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:20:35 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:20:35 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:20:35 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:20:35 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:20:35 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:20:35 user nova-compute[72351]:       <nova:owner>
Aug 09 02:20:35 user nova-compute[72351]:         <nova:user uuid="bec5b5704e9b4e3480f428d5c037c20d">tempest-TestMinimumBasicScenario-1380132717-project-member</nova:user>
Aug 09 02:20:35 user nova-compute[72351]:         <nova:project uuid="c2f67da658d94fabbab82ee6a6171c4b">tempest-TestMinimumBasicScenario-1380132717</nova:project>
Aug 09 02:20:35 user nova-compute[72351]:       </nova:owner>
Aug 09 02:20:35 user nova-compute[72351]:       <nova:root type="image" uuid="9eadc1b0-9cb8-4bc1-97d6-af86e590c548"/>
Aug 09 02:20:35 user nova-compute[72351]:       <nova:ports>
Aug 09 02:20:35 user nova-compute[72351]:         <nova:port uuid="74c98543-ce04-43d8-a58c-267e372db698">
Aug 09 02:20:35 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.12" ipVersion="4"/>
Aug 09 02:20:35 user nova-compute[72351]:         </nova:port>
Aug 09 02:20:35 user nova-compute[72351]:       </nova:ports>
Aug 09 02:20:35 user nova-compute[72351]:     </nova:instance>
Aug 09 02:20:35 user nova-compute[72351]:   </metadata>
Aug 09 02:20:35 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:20:35 user nova-compute[72351]:     <system>
Aug 09 02:20:35 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:20:35 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:20:35 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:20:35 user nova-compute[72351]:       <entry name="serial">97b13ccd-15b4-44bf-b865-71b4da861168</entry>
Aug 09 02:20:35 user nova-compute[72351]:       <entry name="uuid">97b13ccd-15b4-44bf-b865-71b4da861168</entry>
Aug 09 02:20:35 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:20:35 user nova-compute[72351]:     </system>
Aug 09 02:20:35 user nova-compute[72351]:   </sysinfo>
Aug 09 02:20:35 user nova-compute[72351]:   <os>
Aug 09 02:20:35 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:20:35 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:20:35 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:20:35 user nova-compute[72351]:   </os>
Aug 09 02:20:35 user nova-compute[72351]:   <features>
Aug 09 02:20:35 user nova-compute[72351]:     <acpi/>
Aug 09 02:20:35 user nova-compute[72351]:     <apic/>
Aug 09 02:20:35 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:20:35 user nova-compute[72351]:   </features>
Aug 09 02:20:35 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:20:35 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:20:35 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:20:35 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:20:35 user nova-compute[72351]:   </clock>
Aug 09 02:20:35 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:20:35 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:20:35 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:20:35 user nova-compute[72351]:   </cpu>
Aug 09 02:20:35 user nova-compute[72351]:   <devices>
Aug 09 02:20:35 user nova-compute[72351]:     <disk type="file" device="disk">
Aug 09 02:20:35 user nova-compute[72351]:       <driver name="qemu" type="qcow2" cache="none"/>
Aug 09 02:20:35 user nova-compute[72351]:       <source file="/opt/stack/data/nova/instances/97b13ccd-15b4-44bf-b865-71b4da861168/disk"/>
Aug 09 02:20:35 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:20:35 user nova-compute[72351]:     </disk>
Aug 09 02:20:35 user nova-compute[72351]:     <disk type="block" device="disk">
Aug 09 02:20:35 user nova-compute[72351]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:20:35 user nova-compute[72351]:       <source dev="/dev/sdb"/>
Aug 09 02:20:35 user nova-compute[72351]:       <target dev="vdb" bus="virtio"/>
Aug 09 02:20:35 user nova-compute[72351]:       <serial>cda288d2-cdeb-4b8f-abbf-acaa3fab484f</serial>
Aug 09 02:20:35 user nova-compute[72351]:     </disk>
Aug 09 02:20:35 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:20:35 user nova-compute[72351]:       <mac address="fa:16:3e:d4:4d:0a"/>
Aug 09 02:20:35 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:20:35 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:20:35 user nova-compute[72351]:       <target dev="tap74c98543-ce"/>
Aug 09 02:20:35 user nova-compute[72351]:     </interface>
Aug 09 02:20:35 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:20:35 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/97b13ccd-15b4-44bf-b865-71b4da861168/console.log" append="off"/>
Aug 09 02:20:35 user nova-compute[72351]:     </serial>
Aug 09 02:20:35 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:20:35 user nova-compute[72351]:     <video>
Aug 09 02:20:35 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:20:35 user nova-compute[72351]:     </video>
Aug 09 02:20:35 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:20:35 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:20:35 user nova-compute[72351]:     </rng>
Aug 09 02:20:35 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:20:35 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:20:35 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:20:35 user nova-compute[72351]:     </memballoon>
Aug 09 02:20:35 user nova-compute[72351]:   </devices>
Aug 09 02:20:35 user nova-compute[72351]: </domain>
Aug 09 02:20:35 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:20:35 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/97b13ccd-15b4-44bf-b865-71b4da861168/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:35 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547635.5679219, 32a28ecb-af4b-4c6e-8f2d-a51857134b34 => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:20:35 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] VM Resumed (Lifecycle Event)
Aug 09 02:20:35 user nova-compute[72351]: DEBUG nova.compute.manager [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:20:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] Guest created on hypervisor {{(pid=72351) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4411}}
Aug 09 02:20:35 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:20:35 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] Instance spawned successfully.
Aug 09 02:20:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:894}}
Aug 09 02:20:35 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:20:35 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:20:35 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547635.5688734, 32a28ecb-af4b-4c6e-8f2d-a51857134b34 => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:20:35 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] VM Started (Lifecycle Event)
Aug 09 02:20:35 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/97b13ccd-15b4-44bf-b865-71b4da861168/disk --force-share --output=json" returned: 0 in 0.142s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:35 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/97b13ccd-15b4-44bf-b865-71b4da861168/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:35 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:20:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] Found default for hw_cdrom_bus of ide {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:20:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] Found default for hw_disk_bus of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:20:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] Found default for hw_input_bus of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:20:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] Found default for hw_pointer_model of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:20:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] Found default for hw_video_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:20:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] Found default for hw_vif_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:20:35 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:20:35 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:20:35 user nova-compute[72351]: INFO nova.compute.manager [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] Took 7.33 seconds to spawn the instance on the hypervisor.
Aug 09 02:20:35 user nova-compute[72351]: DEBUG nova.compute.manager [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:20:35 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/97b13ccd-15b4-44bf-b865-71b4da861168/disk --force-share --output=json" returned: 0 in 0.221s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] skipping disk /dev/sdb (vdb) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:20:35 user nova-compute[72351]: DEBUG nova.objects.instance [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lazy-loading 'trusted_certs' on Instance uuid 97b13ccd-15b4-44bf-b865-71b4da861168 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:20:35 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/78680203fb06db2bb9e4fc0abad2df7c1c424054 --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:35 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Instance destroyed successfully.
Aug 09 02:20:36 user nova-compute[72351]: DEBUG nova.objects.instance [None req-9e64a2ea-7793-4e1a-a5f5-e23d770c0b2f tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lazy-loading 'resources' on Instance uuid a5ca6400-1125-431e-8a91-057e746d40b5 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-9e64a2ea-7793-4e1a-a5f5-e23d770c0b2f tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:18:43Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-server-1448258389',display_name='tempest-TestVolumeBootPattern-server-1448258389',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testvolumebootpattern-server-1448258389',id=10,image_ref='',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:19:07Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='a4c6bbfd8d904ba1ac69ba5da4564513',ramdisk_id='',reservation_id='r-o29ig997',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TestVolumeBootPattern-350743460',owner_user_name='tempest-TestVolumeBootPattern-350743460-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:19:08Z,user_data=None,user_id='07d8d5a634274df19c2db769d2d27b29',uuid=a5ca6400-1125-431e-8a91-057e746d40b5,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "49802a57-3939-4cc2-956b-3d396b7c738e", "address": "fa:16:3e:9e:eb:c0", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap49802a57-39", "ovs_interfaceid": "49802a57-3939-4cc2-956b-3d396b7c738e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-9e64a2ea-7793-4e1a-a5f5-e23d770c0b2f tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Converting VIF {"id": "49802a57-3939-4cc2-956b-3d396b7c738e", "address": "fa:16:3e:9e:eb:c0", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap49802a57-39", "ovs_interfaceid": "49802a57-3939-4cc2-956b-3d396b7c738e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-9e64a2ea-7793-4e1a-a5f5-e23d770c0b2f tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:9e:eb:c0,bridge_name='br-int',has_traffic_filtering=True,id=49802a57-3939-4cc2-956b-3d396b7c738e,network=Network(5ef428cf-e408-490c-8f60-70934f760f4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap49802a57-39') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG os_vif [None req-9e64a2ea-7793-4e1a-a5f5-e23d770c0b2f tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:9e:eb:c0,bridge_name='br-int',has_traffic_filtering=True,id=49802a57-3939-4cc2-956b-3d396b7c738e,network=Network(5ef428cf-e408-490c-8f60-70934f760f4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap49802a57-39') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:20:36 user nova-compute[72351]: INFO nova.compute.manager [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] Took 8.28 seconds to build instance.
Aug 09 02:20:36 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap49802a57-39, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:20:36 user nova-compute[72351]: INFO os_vif [None req-9e64a2ea-7793-4e1a-a5f5-e23d770c0b2f tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:9e:eb:c0,bridge_name='br-int',has_traffic_filtering=True,id=49802a57-3939-4cc2-956b-3d396b7c738e,network=Network(5ef428cf-e408-490c-8f60-70934f760f4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap49802a57-39')
Aug 09 02:20:36 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-9e64a2ea-7793-4e1a-a5f5-e23d770c0b2f tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-9e64a2ea-7793-4e1a-a5f5-e23d770c0b2f tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'lYLkVxRr', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sde'}, None), 'kwargs': True}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:20:36 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-9e64a2ea-7793-4e1a-a5f5-e23d770c0b2f tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:20:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-9e64a2ea-7793-4e1a-a5f5-e23d770c0b2f tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-9e64a2ea-7793-4e1a-a5f5-e23d770c0b2f tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-9e64a2ea-7793-4e1a-a5f5-e23d770c0b2f tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-50d8cfb9-4e95-4f01-ad43-6c1e2c697069 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Lock "32a28ecb-af4b-4c6e-8f2d-a51857134b34" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 8.411s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a39bd00f-98c2-48ff-8e67-e262da537546]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/78680203fb06db2bb9e4fc0abad2df7c1c424054 --force-share --output=json" returned: 0 in 0.163s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Checking if we can resize image /opt/stack/data/nova/instances/97b13ccd-15b4-44bf-b865-71b4da861168/disk. size=1073741824 {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/97b13ccd-15b4-44bf-b865-71b4da861168/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[1f51e1df-fc0d-439b-bb58-b53fa7c70744]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-9e64a2ea-7793-4e1a-a5f5-e23d770c0b2f tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f (non-flash)
Aug 09 02:20:36 user nova-compute[72351]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:20:36 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-9e64a2ea-7793-4e1a-a5f5-e23d770c0b2f tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f (non-flash)
Aug 09 02:20:36 user nova-compute[72351]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 (non-flash)
Aug 09 02:20:36 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-9e64a2ea-7793-4e1a-a5f5-e23d770c0b2f tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fcfdc18b9a0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7'): ({'sde'}, set())}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-9e64a2ea-7793-4e1a-a5f5-e23d770c0b2f tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Removing single pathed devices sde {{(pid=72351) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a44f09db-d0c5-45f5-ae4b-6488a5614e86]: (4, ('path checker states:\nup                  2\n\npaths: 0\nbusy: False\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sde {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sde" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d8cb8cea-92a8-4df0-84e5-8e029c45ec5a]: (4, ('ok\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-9e64a2ea-7793-4e1a-a5f5-e23d770c0b2f tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Flushing IO for device /dev/sde {{(pid=72351) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sde {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sde" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[343022a1-5a18-44cd-bfd5-e87fe587458e]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-9e64a2ea-7793-4e1a-a5f5-e23d770c0b2f tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Remove SCSI device /dev/sde with /sys/block/sde/device/delete {{(pid=72351) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sde/device/delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/97b13ccd-15b4-44bf-b865-71b4da861168/disk --force-share --output=json" returned: 0 in 0.148s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Cannot resize image /opt/stack/data/nova/instances/97b13ccd-15b4-44bf-b865-71b4da861168/disk to a smaller size. {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG nova.objects.instance [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lazy-loading 'migration_context' on Instance uuid 97b13ccd-15b4-44bf-b865-71b4da861168 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:18:04Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestMinimumBasicScenario-server-1233822315',display_name='tempest-TestMinimumBasicScenario-server-1233822315',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testminimumbasicscenario-server-1233822315',id=8,image_ref='9eadc1b0-9cb8-4bc1-97d6-af86e590c548',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOk8lLSpSCaCDIKlT4lhWRaHWy+oi2s0IJNKXHyKayH1X4sCjRLnVKeip/ShH5Av4ZZCopMp+S7MD8OAGw8MyvDsE398yCbkjnI/7XCIasyAoPOQXBUUZ52swhczfVlPLg==',key_name='tempest-TestMinimumBasicScenario-1974156570',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:18:12Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=<?>,power_state=1,progress=0,project_id='c2f67da658d94fabbab82ee6a6171c4b',ramdisk_id='',reservation_id='r-rhqfwdu0',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='9eadc1b0-9cb8-4bc1-97d6-af86e590c548',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',owner_project_name='tempest-TestMinimumBasicScenario-1380132717',owner_user_name='tempest-TestMinimumBasicScenario-1380132717-project-member'},tags=<?>,task_state='reboot_started_hard',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:20:33Z,user_data=None,user_id='bec5b5704e9b4e3480f428d5c037c20d',uuid=97b13ccd-15b4-44bf-b865-71b4da861168,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "74c98543-ce04-43d8-a58c-267e372db698", "address": "fa:16:3e:d4:4d:0a", "network": {"id": "7024d1f6-a400-4482-9364-78ae3b6adc80", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1603258958-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.62", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c2f67da658d94fabbab82ee6a6171c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap74c98543-ce", "ovs_interfaceid": "74c98543-ce04-43d8-a58c-267e372db698", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Converting VIF {"id": "74c98543-ce04-43d8-a58c-267e372db698", "address": "fa:16:3e:d4:4d:0a", "network": {"id": "7024d1f6-a400-4482-9364-78ae3b6adc80", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1603258958-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.62", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c2f67da658d94fabbab82ee6a6171c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap74c98543-ce", "ovs_interfaceid": "74c98543-ce04-43d8-a58c-267e372db698", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:d4:4d:0a,bridge_name='br-int',has_traffic_filtering=True,id=74c98543-ce04-43d8-a58c-267e372db698,network=Network(7024d1f6-a400-4482-9364-78ae3b6adc80),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap74c98543-ce') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG os_vif [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Plugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:d4:4d:0a,bridge_name='br-int',has_traffic_filtering=True,id=74c98543-ce04-43d8-a58c-267e372db698,network=Network(7024d1f6-a400-4482-9364-78ae3b6adc80),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap74c98543-ce') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sde/device/delete" returned: 0 in 0.042s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[db31808e-c1a1-4013-9032-baad1b450e0f]: (4, ('1', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-9e64a2ea-7793-4e1a-a5f5-e23d770c0b2f tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Checking to see if SCSI volumes sde have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-9e64a2ea-7793-4e1a-a5f5-e23d770c0b2f tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] SCSI volumes sde have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG os_brick.privileged.rootwrap [-] Unlinking ('/dev/disk/by-id/scsi-SSCST_BIO_Storage_ff25e235', '/dev/disk/by-id/scsi-23338353236663431', '/dev/disk/by-id/scsi-1SCST_BIOff25e235-38526f411d1e2085') {{(pid=83366) unlink_root /usr/local/lib/python3.10/dist-packages/os_brick/privileged/rootwrap.py:219}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[78e6717e-aeb3-43f4-bdde-7960a7c063cb]: (4, None) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-9e64a2ea-7793-4e1a-a5f5-e23d770c0b2f tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7')] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap74c98543-ce, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap74c98543-ce, col_values=(('external_ids', {'iface-id': '74c98543-ce04-43d8-a58c-267e372db698', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:d4:4d:0a', 'vm-uuid': '97b13ccd-15b4-44bf-b865-71b4da861168'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:36 user nova-compute[72351]: INFO os_vif [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Successfully plugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:d4:4d:0a,bridge_name='br-int',has_traffic_filtering=True,id=74c98543-ce04-43d8-a58c-267e372db698,network=Network(7024d1f6-a400-4482-9364-78ae3b6adc80),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap74c98543-ce')
Aug 09 02:20:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[0524fe98-b747-4483-aa93-37ba265ff6cd]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-9e64a2ea-7793-4e1a-a5f5-e23d770c0b2f tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 -p 172.16.0.220:3260 --logout {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 -p 172.16.0.220:3260 --logout" returned: 0 in 0.063s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e1927ab6-9192-42c4-b3e9-54bddf174b32]: (4, ('Logging out of session [sid: 5, target: iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7, portal: 172.16.0.220,3260]\nLogout of [sid: 5, target: iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-9e64a2ea-7793-4e1a-a5f5-e23d770c0b2f tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 5, target: iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7, portal: 172.16.0.220,3260]
Aug 09 02:20:36 user nova-compute[72351]: Logout of [sid: 5, target: iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7, portal: 172.16.0.220,3260] successful.
Aug 09 02:20:36 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 -p 172.16.0.220:3260 --op delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3f5c13ad-c735-40fa-89eb-c68adda906d7 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[274ffc2c-dfa1-4ead-bf1a-3400a93a7556]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-9e64a2ea-7793-4e1a-a5f5-e23d770c0b2f tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-9e64a2ea-7793-4e1a-a5f5-e23d770c0b2f tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.362s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-9e64a2ea-7793-4e1a-a5f5-e23d770c0b2f tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] <== disconnect_volume: return (363ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-9e64a2ea-7793-4e1a-a5f5-e23d770c0b2f tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 02:20:36 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-9e64a2ea-7793-4e1a-a5f5-e23d770c0b2f tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Deleting instance files /opt/stack/data/nova/instances/a5ca6400-1125-431e-8a91-057e746d40b5_del
Aug 09 02:20:36 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-9e64a2ea-7793-4e1a-a5f5-e23d770c0b2f tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Deletion of /opt/stack/data/nova/instances/a5ca6400-1125-431e-8a91-057e746d40b5_del complete
Aug 09 02:20:36 user nova-compute[72351]: INFO nova.compute.manager [None req-9e64a2ea-7793-4e1a-a5f5-e23d770c0b2f tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Took 1.36 seconds to destroy the instance on the hypervisor.
Aug 09 02:20:36 user nova-compute[72351]: DEBUG oslo.service.loopingcall [None req-9e64a2ea-7793-4e1a-a5f5-e23d770c0b2f tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72351) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG nova.compute.manager [-] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Deallocating network for instance {{(pid=72351) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2263}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] deallocate_for_instance() {{(pid=72351) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1795}}
Aug 09 02:20:36 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691547621.7068064, 629e177e-afbb-4765-a0b5-7a3f66f72ebf => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:20:36 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] VM Stopped (Lifecycle Event)
Aug 09 02:20:36 user nova-compute[72351]: DEBUG nova.compute.manager [None req-abd436f6-b912-4ff1-8494-81deba9a0985 None None] [instance: 629e177e-afbb-4765-a0b5-7a3f66f72ebf] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:20:37 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:37 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:20:37 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Took 0.81 seconds to deallocate network for instance.
Aug 09 02:20:37 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:37 user nova-compute[72351]: DEBUG nova.compute.manager [req-af7bbe66-69ed-4124-a101-0b1b61fdc72c req-dc966ff8-9fa2-4b31-9cab-98b4ac04d979 service nova] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Received event network-vif-plugged-49802a57-3939-4cc2-956b-3d396b7c738e {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:20:37 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-af7bbe66-69ed-4124-a101-0b1b61fdc72c req-dc966ff8-9fa2-4b31-9cab-98b4ac04d979 service nova] Acquiring lock "a5ca6400-1125-431e-8a91-057e746d40b5-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:20:37 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-af7bbe66-69ed-4124-a101-0b1b61fdc72c req-dc966ff8-9fa2-4b31-9cab-98b4ac04d979 service nova] Lock "a5ca6400-1125-431e-8a91-057e746d40b5-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:20:37 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-af7bbe66-69ed-4124-a101-0b1b61fdc72c req-dc966ff8-9fa2-4b31-9cab-98b4ac04d979 service nova] Lock "a5ca6400-1125-431e-8a91-057e746d40b5-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:20:37 user nova-compute[72351]: DEBUG nova.compute.manager [req-af7bbe66-69ed-4124-a101-0b1b61fdc72c req-dc966ff8-9fa2-4b31-9cab-98b4ac04d979 service nova] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] No waiting events found dispatching network-vif-plugged-49802a57-3939-4cc2-956b-3d396b7c738e {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:20:37 user nova-compute[72351]: WARNING nova.compute.manager [req-af7bbe66-69ed-4124-a101-0b1b61fdc72c req-dc966ff8-9fa2-4b31-9cab-98b4ac04d979 service nova] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Received unexpected event network-vif-plugged-49802a57-3939-4cc2-956b-3d396b7c738e for instance with vm_state active and task_state deleting.
Aug 09 02:20:37 user nova-compute[72351]: DEBUG nova.compute.manager [req-af7bbe66-69ed-4124-a101-0b1b61fdc72c req-dc966ff8-9fa2-4b31-9cab-98b4ac04d979 service nova] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Received event network-vif-deleted-49802a57-3939-4cc2-956b-3d396b7c738e {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:20:37 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:38 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:38 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:38 user nova-compute[72351]: INFO nova.compute.manager [None req-9e64a2ea-7793-4e1a-a5f5-e23d770c0b2f tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Took 1.15 seconds to detach 1 volumes for instance.
Aug 09 02:20:38 user nova-compute[72351]: DEBUG nova.compute.manager [None req-9e64a2ea-7793-4e1a-a5f5-e23d770c0b2f tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Deleting volume: 3f5c13ad-c735-40fa-89eb-c68adda906d7 {{(pid=72351) _cleanup_volumes /opt/stack/nova/nova/compute/manager.py:3221}}
Aug 09 02:20:38 user nova-compute[72351]: DEBUG nova.compute.manager [req-0d735d5c-1613-42b9-a590-1c3a5fa5089e req-c4e44c9d-473c-4955-84aa-8f1a6c578594 service nova] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Received event network-vif-plugged-74c98543-ce04-43d8-a58c-267e372db698 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:20:38 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-0d735d5c-1613-42b9-a590-1c3a5fa5089e req-c4e44c9d-473c-4955-84aa-8f1a6c578594 service nova] Acquiring lock "97b13ccd-15b4-44bf-b865-71b4da861168-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:20:38 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-0d735d5c-1613-42b9-a590-1c3a5fa5089e req-c4e44c9d-473c-4955-84aa-8f1a6c578594 service nova] Lock "97b13ccd-15b4-44bf-b865-71b4da861168-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:20:38 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-0d735d5c-1613-42b9-a590-1c3a5fa5089e req-c4e44c9d-473c-4955-84aa-8f1a6c578594 service nova] Lock "97b13ccd-15b4-44bf-b865-71b4da861168-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:20:38 user nova-compute[72351]: DEBUG nova.compute.manager [req-0d735d5c-1613-42b9-a590-1c3a5fa5089e req-c4e44c9d-473c-4955-84aa-8f1a6c578594 service nova] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] No waiting events found dispatching network-vif-plugged-74c98543-ce04-43d8-a58c-267e372db698 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:20:38 user nova-compute[72351]: WARNING nova.compute.manager [req-0d735d5c-1613-42b9-a590-1c3a5fa5089e req-c4e44c9d-473c-4955-84aa-8f1a6c578594 service nova] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Received unexpected event network-vif-plugged-74c98543-ce04-43d8-a58c-267e372db698 for instance with vm_state active and task_state reboot_started_hard.
Aug 09 02:20:38 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-9e64a2ea-7793-4e1a-a5f5-e23d770c0b2f tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:20:38 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-9e64a2ea-7793-4e1a-a5f5-e23d770c0b2f tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:20:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:39 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-9e64a2ea-7793-4e1a-a5f5-e23d770c0b2f tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:20:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:39 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-9e64a2ea-7793-4e1a-a5f5-e23d770c0b2f tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:20:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:39 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-9e64a2ea-7793-4e1a-a5f5-e23d770c0b2f tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.528s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:20:39 user nova-compute[72351]: INFO nova.scheduler.client.report [None req-9e64a2ea-7793-4e1a-a5f5-e23d770c0b2f tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Deleted allocations for instance a5ca6400-1125-431e-8a91-057e746d40b5
Aug 09 02:20:39 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-9e64a2ea-7793-4e1a-a5f5-e23d770c0b2f tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "a5ca6400-1125-431e-8a91-057e746d40b5" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 4.316s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:20:40 user nova-compute[72351]: DEBUG nova.compute.manager [req-d83ee2a7-d7f6-432f-8c1c-87c05a12acb9 req-d7cc49af-9612-49cb-a627-fb107d87c474 service nova] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Received event network-vif-plugged-74c98543-ce04-43d8-a58c-267e372db698 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:20:40 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d83ee2a7-d7f6-432f-8c1c-87c05a12acb9 req-d7cc49af-9612-49cb-a627-fb107d87c474 service nova] Acquiring lock "97b13ccd-15b4-44bf-b865-71b4da861168-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:20:40 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d83ee2a7-d7f6-432f-8c1c-87c05a12acb9 req-d7cc49af-9612-49cb-a627-fb107d87c474 service nova] Lock "97b13ccd-15b4-44bf-b865-71b4da861168-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:20:40 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d83ee2a7-d7f6-432f-8c1c-87c05a12acb9 req-d7cc49af-9612-49cb-a627-fb107d87c474 service nova] Lock "97b13ccd-15b4-44bf-b865-71b4da861168-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:20:40 user nova-compute[72351]: DEBUG nova.compute.manager [req-d83ee2a7-d7f6-432f-8c1c-87c05a12acb9 req-d7cc49af-9612-49cb-a627-fb107d87c474 service nova] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] No waiting events found dispatching network-vif-plugged-74c98543-ce04-43d8-a58c-267e372db698 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:20:40 user nova-compute[72351]: WARNING nova.compute.manager [req-d83ee2a7-d7f6-432f-8c1c-87c05a12acb9 req-d7cc49af-9612-49cb-a627-fb107d87c474 service nova] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Received unexpected event network-vif-plugged-74c98543-ce04-43d8-a58c-267e372db698 for instance with vm_state active and task_state reboot_started_hard.
Aug 09 02:20:40 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Removed pending event for 97b13ccd-15b4-44bf-b865-71b4da861168 due to event {{(pid=72351) _event_emit_delayed /opt/stack/nova/nova/virt/libvirt/host.py:438}}
Aug 09 02:20:40 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547640.7616794, 97b13ccd-15b4-44bf-b865-71b4da861168 => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:20:40 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] VM Resumed (Lifecycle Event)
Aug 09 02:20:40 user nova-compute[72351]: DEBUG nova.compute.manager [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:20:40 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Instance rebooted successfully.
Aug 09 02:20:40 user nova-compute[72351]: DEBUG nova.compute.manager [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:20:40 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:20:40 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: active, current task_state: reboot_started_hard, current DB power_state: 1, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:20:40 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] During sync_power_state the instance has a pending task (reboot_started_hard). Skip.
Aug 09 02:20:40 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547640.7617881, 97b13ccd-15b4-44bf-b865-71b4da861168 => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:20:40 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] VM Started (Lifecycle Event)
Aug 09 02:20:40 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:20:40 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Synchronizing instance power state after lifecycle event "Started"; current vm_state: active, current task_state: reboot_started_hard, current DB power_state: 1, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:20:40 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-9bcca557-39bf-4b81-977a-bab0b2653fcd tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "97b13ccd-15b4-44bf-b865-71b4da861168" "released" by "nova.compute.manager.ComputeManager.reboot_instance.<locals>.do_reboot_instance" :: held 8.879s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:20:41 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:42 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:42 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691547627.628111, 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0 => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:20:42 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] VM Stopped (Lifecycle Event)
Aug 09 02:20:42 user nova-compute[72351]: DEBUG nova.compute.manager [None req-6fda8766-bc6a-453e-b829-791450d6d9f0 None None] [instance: 7c5e633d-a32f-4c5e-bb9d-1e40da964bd0] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:20:46 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:50 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-45912413-9589-4a32-92a8-7c77bbf03376 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Acquiring lock "2d78475c-107d-4256-b12e-dbe9c8f3b9b2" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:20:50 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-45912413-9589-4a32-92a8-7c77bbf03376 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Lock "2d78475c-107d-4256-b12e-dbe9c8f3b9b2" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:20:50 user nova-compute[72351]: DEBUG nova.objects.instance [None req-45912413-9589-4a32-92a8-7c77bbf03376 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Lazy-loading 'flavor' on Instance uuid 2d78475c-107d-4256-b12e-dbe9c8f3b9b2 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:20:50 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-45912413-9589-4a32-92a8-7c77bbf03376 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Lock "2d78475c-107d-4256-b12e-dbe9c8f3b9b2" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.054s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:20:50 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-45912413-9589-4a32-92a8-7c77bbf03376 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Acquiring lock "2d78475c-107d-4256-b12e-dbe9c8f3b9b2" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:20:50 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-45912413-9589-4a32-92a8-7c77bbf03376 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Lock "2d78475c-107d-4256-b12e-dbe9c8f3b9b2" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.003s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:20:50 user nova-compute[72351]: INFO nova.compute.manager [None req-45912413-9589-4a32-92a8-7c77bbf03376 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Attaching volume 41ceb107-f270-4b7a-a2b0-14c2dbdd5497 to /dev/vdb
Aug 09 02:20:50 user nova-compute[72351]: DEBUG os_brick.utils [None req-45912413-9589-4a32-92a8-7c77bbf03376 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:20:50 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:50 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:50 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[325c0487-95db-48c0-8ed4-e0c97aacaccb]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:a52269602ff4\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:50 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:50 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.027s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:50 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[096290bb-07c0-4253-be34-b8138fad531a]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:50 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:50 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:50 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[fac059e0-e9b9-48ad-b61b-7f28e24dfcf8]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:50 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[fee50bfd-cbbe-4cd8-a0d8-5fdd45d933cc]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:50 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-45912413-9589-4a32-92a8-7c77bbf03376 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Running cmd (subprocess): nvme version {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:50 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-45912413-9589-4a32-92a8-7c77bbf03376 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] 'nvme version' failed. Not Retrying. {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:20:50 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.nvmeof [None req-45912413-9589-4a32-92a8-7c77bbf03376 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] nvme not present on system {{(pid=72351) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Aug 09 02:20:50 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:50 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:20:50 user nova-compute[72351]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Aug 09 02:20:50 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[1d3b7c2f-40a7-473a-bef5-41789908bd5a]: (4, '') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:50 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-45912413-9589-4a32-92a8-7c77bbf03376 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72351) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Aug 09 02:20:50 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-45912413-9589-4a32-92a8-7c77bbf03376 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Aug 09 02:20:50 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-45912413-9589-4a32-92a8-7c77bbf03376 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] LIGHTOS: no hostnqn found. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Aug 09 02:20:50 user nova-compute[72351]: DEBUG os_brick.utils [None req-45912413-9589-4a32-92a8-7c77bbf03376 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] <== get_connector_properties: return (147ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:a52269602ff4', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:20:50 user nova-compute[72351]: DEBUG nova.virt.block_device [None req-45912413-9589-4a32-92a8-7c77bbf03376 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Updating existing volume attachment record: 242e142b-552b-447c-8088-f77e546f99f5 {{(pid=72351) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Aug 09 02:20:50 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691547635.9640594, a5ca6400-1125-431e-8a91-057e746d40b5 => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:20:50 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] VM Stopped (Lifecycle Event)
Aug 09 02:20:51 user nova-compute[72351]: DEBUG nova.compute.manager [None req-9f65686d-183f-4112-80c1-b736f2a3fe6a None None] [instance: a5ca6400-1125-431e-8a91-057e746d40b5] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:20:51 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:20:51 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:20:52 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:52 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:52 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:20:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:20:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:20:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:20:52 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Auditing locally available compute resources for user (node: user) {{(pid=72351) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Aug 09 02:20:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b/disk --force-share --output=json" returned: 0 in 0.162s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b/disk --force-share --output=json" returned: 0 in 0.159s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/822e25c4-8dd1-4e21-b0ed-f340d59c6c98/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/822e25c4-8dd1-4e21-b0ed-f340d59c6c98/disk --force-share --output=json" returned: 0 in 0.197s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/822e25c4-8dd1-4e21-b0ed-f340d59c6c98/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:53 user nova-compute[72351]: DEBUG nova.compute.manager [req-c4f49d6d-0c31-43b0-8757-45b140a3d27e req-a9becf77-0a5b-4870-b332-f92754b3c8f7 service nova] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Received event network-changed-6bfee618-fc3b-43ba-b96f-c75db4b7a4e9 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:20:53 user nova-compute[72351]: DEBUG nova.compute.manager [req-c4f49d6d-0c31-43b0-8757-45b140a3d27e req-a9becf77-0a5b-4870-b332-f92754b3c8f7 service nova] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Refreshing instance network info cache due to event network-changed-6bfee618-fc3b-43ba-b96f-c75db4b7a4e9. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:20:53 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c4f49d6d-0c31-43b0-8757-45b140a3d27e req-a9becf77-0a5b-4870-b332-f92754b3c8f7 service nova] Acquiring lock "refresh_cache-822e25c4-8dd1-4e21-b0ed-f340d59c6c98" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:20:53 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c4f49d6d-0c31-43b0-8757-45b140a3d27e req-a9becf77-0a5b-4870-b332-f92754b3c8f7 service nova] Acquired lock "refresh_cache-822e25c4-8dd1-4e21-b0ed-f340d59c6c98" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:20:53 user nova-compute[72351]: DEBUG nova.network.neutron [req-c4f49d6d-0c31-43b0-8757-45b140a3d27e req-a9becf77-0a5b-4870-b332-f92754b3c8f7 service nova] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Refreshing network info cache for port 6bfee618-fc3b-43ba-b96f-c75db4b7a4e9 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:20:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/822e25c4-8dd1-4e21-b0ed-f340d59c6c98/disk --force-share --output=json" returned: 0 in 0.149s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2/disk --force-share --output=json" returned: 0 in 0.147s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2/disk --force-share --output=json" returned: 0 in 0.152s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a1d1f738-0edf-4184-aa56-020c07830492/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:54 user nova-compute[72351]: DEBUG nova.network.neutron [req-c4f49d6d-0c31-43b0-8757-45b140a3d27e req-a9becf77-0a5b-4870-b332-f92754b3c8f7 service nova] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Updated VIF entry in instance network info cache for port 6bfee618-fc3b-43ba-b96f-c75db4b7a4e9. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:20:54 user nova-compute[72351]: DEBUG nova.network.neutron [req-c4f49d6d-0c31-43b0-8757-45b140a3d27e req-a9becf77-0a5b-4870-b332-f92754b3c8f7 service nova] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Updating instance_info_cache with network_info: [{"id": "6bfee618-fc3b-43ba-b96f-c75db4b7a4e9", "address": "fa:16:3e:e4:67:dd", "network": {"id": "66067153-d668-48d7-9851-b33abf90f2e1", "bridge": "br-int", "label": "tempest-TestServerVolumeAttachScenarioOldVersion-1194345640-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.228", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5fd127f53fc54fdbbf8ae524f3578f91", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6bfee618-fc", "ovs_interfaceid": "6bfee618-fc3b-43ba-b96f-c75db4b7a4e9", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:20:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a1d1f738-0edf-4184-aa56-020c07830492/disk --force-share --output=json" returned: 0 in 0.192s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a1d1f738-0edf-4184-aa56-020c07830492/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c4f49d6d-0c31-43b0-8757-45b140a3d27e req-a9becf77-0a5b-4870-b332-f92754b3c8f7 service nova] Releasing lock "refresh_cache-822e25c4-8dd1-4e21-b0ed-f340d59c6c98" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:20:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a1d1f738-0edf-4184-aa56-020c07830492/disk --force-share --output=json" returned: 0 in 0.148s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/52f0d664-7767-4953-bc7a-496b26f22715/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/52f0d664-7767-4953-bc7a-496b26f22715/disk --force-share --output=json" returned: 0 in 0.180s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/52f0d664-7767-4953-bc7a-496b26f22715/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/52f0d664-7767-4953-bc7a-496b26f22715/disk --force-share --output=json" returned: 0 in 0.142s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/74728ca8-34d4-45e6-a1f2-21c0952ff835/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/74728ca8-34d4-45e6-a1f2-21c0952ff835/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/74728ca8-34d4-45e6-a1f2-21c0952ff835/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/74728ca8-34d4-45e6-a1f2-21c0952ff835/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/97b13ccd-15b4-44bf-b865-71b4da861168/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/97b13ccd-15b4-44bf-b865-71b4da861168/disk --force-share --output=json" returned: 0 in 0.147s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/97b13ccd-15b4-44bf-b865-71b4da861168/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/97b13ccd-15b4-44bf-b865-71b4da861168/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:55 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdb (vdb) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:20:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/32a28ecb-af4b-4c6e-8f2d-a51857134b34/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/32a28ecb-af4b-4c6e-8f2d-a51857134b34/disk --force-share --output=json" returned: 0 in 0.195s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/32a28ecb-af4b-4c6e-8f2d-a51857134b34/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/32a28ecb-af4b-4c6e-8f2d-a51857134b34/disk --force-share --output=json" returned: 0 in 0.184s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:56 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:56 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:20:56 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:20:56 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Hypervisor/Node resource view: name=user free_ram=7400MB free_disk=25.582611083984375GB free_vcpus=4 pci_devices=[{"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72351) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Aug 09 02:20:56 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:20:56 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:20:56 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 2d78475c-107d-4256-b12e-dbe9c8f3b9b2 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:20:56 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 74728ca8-34d4-45e6-a1f2-21c0952ff835 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:20:56 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:20:56 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 97b13ccd-15b4-44bf-b865-71b4da861168 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:20:56 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 822e25c4-8dd1-4e21-b0ed-f340d59c6c98 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:20:56 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 52f0d664-7767-4953-bc7a-496b26f22715 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:20:56 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance a1d1f738-0edf-4184-aa56-020c07830492 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:20:56 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 32a28ecb-af4b-4c6e-8f2d-a51857134b34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:20:56 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Total usable vcpus: 12, total allocated vcpus: 8 {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Aug 09 02:20:56 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Final resource view: name=user phys_ram=16011MB used_ram=4608MB phys_disk=40GB used_disk=8GB total_vcpus=12 used_vcpus=8 pci_stats=[] {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Aug 09 02:20:56 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:20:56 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:20:56 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Compute_service record updated for user:user {{(pid=72351) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Aug 09 02:20:56 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.400s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:20:57 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:20:57 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:20:57 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:20:57 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:20:57 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Starting heal instance info cache {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9842}}
Aug 09 02:20:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "refresh_cache-11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:20:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquired lock "refresh_cache-11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:20:57 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Forcefully refreshing network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1996}}
Aug 09 02:20:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-20131e30-410d-4ae0-8c83-1e5c1cdc6b4b tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Acquiring lock "822e25c4-8dd1-4e21-b0ed-f340d59c6c98" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:20:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-20131e30-410d-4ae0-8c83-1e5c1cdc6b4b tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Lock "822e25c4-8dd1-4e21-b0ed-f340d59c6c98" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:20:58 user nova-compute[72351]: DEBUG nova.objects.instance [None req-20131e30-410d-4ae0-8c83-1e5c1cdc6b4b tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Lazy-loading 'flavor' on Instance uuid 822e25c4-8dd1-4e21-b0ed-f340d59c6c98 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:20:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-20131e30-410d-4ae0-8c83-1e5c1cdc6b4b tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Lock "822e25c4-8dd1-4e21-b0ed-f340d59c6c98" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.081s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:20:58 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Updating instance_info_cache with network_info: [{"id": "a309cd5b-1679-48c6-a596-168b3af2d284", "address": "fa:16:3e:dc:f0:3c", "network": {"id": "da9c66d8-08c8-4c9c-aec7-c9ad980b89e4", "bridge": "br-int", "label": "tempest-TestVolumeSwap-1327534036-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.16", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "1277988bc35041249b8e315df5259f98", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa309cd5b-16", "ovs_interfaceid": "a309cd5b-1679-48c6-a596-168b3af2d284", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:20:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-20131e30-410d-4ae0-8c83-1e5c1cdc6b4b tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Acquiring lock "822e25c4-8dd1-4e21-b0ed-f340d59c6c98" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:20:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-20131e30-410d-4ae0-8c83-1e5c1cdc6b4b tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Lock "822e25c4-8dd1-4e21-b0ed-f340d59c6c98" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:20:58 user nova-compute[72351]: INFO nova.compute.manager [None req-20131e30-410d-4ae0-8c83-1e5c1cdc6b4b tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Attaching volume a4f329c4-9137-4cd0-971e-7a7402ba1785 to /dev/vdb
Aug 09 02:20:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Releasing lock "refresh_cache-11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:20:58 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Updated the network info_cache for instance {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9913}}
Aug 09 02:20:58 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:20:58 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:20:58 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:20:58 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:20:58 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:20:58 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72351) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10461}}
Aug 09 02:20:58 user nova-compute[72351]: DEBUG os_brick.utils [None req-20131e30-410d-4ae0-8c83-1e5c1cdc6b4b tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:20:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.015s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:58 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[c46d82e1-a3d1-4e79-b4f9-de653a55df38]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:a52269602ff4\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:58 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[f17a475c-13b7-44e8-b18c-2dafd6b0effe]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.016s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:58 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[95ba7e0d-7c91-4fa2-ac1e-0a1637070e4a]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:58 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d94d5e77-64aa-4d3f-951f-f1a9a4832fbf]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-20131e30-410d-4ae0-8c83-1e5c1cdc6b4b tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Running cmd (subprocess): nvme version {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-20131e30-410d-4ae0-8c83-1e5c1cdc6b4b tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] 'nvme version' failed. Not Retrying. {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:20:58 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.nvmeof [None req-20131e30-410d-4ae0-8c83-1e5c1cdc6b4b tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] nvme not present on system {{(pid=72351) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Aug 09 02:20:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:20:58 user nova-compute[72351]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Aug 09 02:20:58 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[8c850cff-3d44-454a-8424-3760425fc070]: (4, '') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:58 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-20131e30-410d-4ae0-8c83-1e5c1cdc6b4b tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72351) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Aug 09 02:20:58 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-20131e30-410d-4ae0-8c83-1e5c1cdc6b4b tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Aug 09 02:20:58 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-20131e30-410d-4ae0-8c83-1e5c1cdc6b4b tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] LIGHTOS: no hostnqn found. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Aug 09 02:20:58 user nova-compute[72351]: DEBUG os_brick.utils [None req-20131e30-410d-4ae0-8c83-1e5c1cdc6b4b tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] <== get_connector_properties: return (137ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:a52269602ff4', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:20:58 user nova-compute[72351]: DEBUG nova.virt.block_device [None req-20131e30-410d-4ae0-8c83-1e5c1cdc6b4b tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Updating existing volume attachment record: a2a49173-933a-411d-8924-772e784d0963 {{(pid=72351) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Aug 09 02:20:59 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-45912413-9589-4a32-92a8-7c77bbf03376 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 02:20:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-45912413-9589-4a32-92a8-7c77bbf03376 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'YaNtwOCg', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:20:59 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-45912413-9589-4a32-92a8-7c77bbf03376 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:20:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-45912413-9589-4a32-92a8-7c77bbf03376 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:20:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-45912413-9589-4a32-92a8-7c77bbf03376 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:20:59 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-45912413-9589-4a32-92a8-7c77bbf03376 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:20:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-45912413-9589-4a32-92a8-7c77bbf03376 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:20:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-45912413-9589-4a32-92a8-7c77bbf03376 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:20:59 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-45912413-9589-4a32-92a8-7c77bbf03376 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 02:20:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497 -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497 -p 172.16.0.220:3260" returned: 21 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:59 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[00771bcc-cbc8-45b8-8edc-375de89f7d3f]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-45912413-9589-4a32-92a8-7c77bbf03376 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Aug 09 02:20:59 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:20:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497 -p 172.16.0.220:3260 --interface default --op new {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:20:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.027s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:20:59 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d23b4231-0919-40d3-9822-9cd178773b68]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497] added\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:20:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-45912413-9589-4a32-92a8-7c77bbf03376 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497] added
Aug 09 02:20:59 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:20:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.024s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:00 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[c164174f-9a7e-4787-89cb-8da4f9624153]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:21:00 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-45912413-9589-4a32-92a8-7c77bbf03376 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:21:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:00 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[4702da01-3d30-431c-bac5-7a88111f3738]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:21:00 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-45912413-9589-4a32-92a8-7c77bbf03376 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:21:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v YaNtwOCg {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v YaNtwOCg" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:00 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[92ec497d-171b-4fbe-a881-52ef6fa204fb]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:21:00 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-45912413-9589-4a32-92a8-7c77bbf03376 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'YaNtwOCg'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:21:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.024s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:00 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d6331360-5880-4177-aa23-b1216f05e503]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:21:00 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-45912413-9589-4a32-92a8-7c77bbf03376 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:21:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:00 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[b687e6bc-7196-49e5-8149-bbe5d17aa67f]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:21:00 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-45912413-9589-4a32-92a8-7c77bbf03376 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f (non-flash)
Aug 09 02:21:00 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:21:00 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-45912413-9589-4a32-92a8-7c77bbf03376 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f (non-flash)
Aug 09 02:21:00 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:21:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497 -p 172.16.0.220:3260 --login {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497 -p 172.16.0.220:3260 --login" returned: 0 in 0.065s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:00 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[293463cc-7b1b-40ca-9849-47f3d20efd7b]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:21:00 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-45912413-9589-4a32-92a8-7c77bbf03376 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497, portal: 172.16.0.220,3260]
Aug 09 02:21:00 user nova-compute[72351]: Login to [iface: default, target: iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497, portal: 172.16.0.220,3260] successful.
Aug 09 02:21:00 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:21:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:00 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[f165c7ec-7f67-4c00-a067-821068fa4ed5]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:21:00 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-45912413-9589-4a32-92a8-7c77bbf03376 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:21:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:00 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[b20c4bbb-f4ff-466f-992f-ab2baa77b8d6]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f (non-flash)\ntcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:21:00 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-45912413-9589-4a32-92a8-7c77bbf03376 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f (non-flash)
Aug 09 02:21:00 user nova-compute[72351]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497 (non-flash)
Aug 09 02:21:00 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:21:00 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-45912413-9589-4a32-92a8-7c77bbf03376 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f (non-flash)
Aug 09 02:21:00 user nova-compute[72351]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497 (non-flash)
Aug 09 02:21:00 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:21:00 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-45912413-9589-4a32-92a8-7c77bbf03376 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.316s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:21:00 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-45912413-9589-4a32-92a8-7c77bbf03376 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 02:21:00 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-45912413-9589-4a32-92a8-7c77bbf03376 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] HCTL ('34', '-', '-', 0) found on session 15 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 02:21:00 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-45912413-9589-4a32-92a8-7c77bbf03376 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Scanning host 34 c: -, t: -, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 02:21:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:00 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ab2fa45d-1e2c-421e-8409-d74b4c0b3fa0]: (4, ('- - 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:21:00 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-45912413-9589-4a32-92a8-7c77bbf03376 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Searching for a device in session 15 and hctl ['34', '*', '*', 0] yield: sdc {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:21:00 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-45912413-9589-4a32-92a8-7c77bbf03376 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Connected to sdc using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'YaNtwOCg', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 02:21:01 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-45912413-9589-4a32-92a8-7c77bbf03376 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.360s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:21:01 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-45912413-9589-4a32-92a8-7c77bbf03376 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] <== connect_volume: return (1362ms) {'type': 'block', 'scsi_wwn': '26135653561653762', 'path': '/dev/sdc'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:21:01 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-45912413-9589-4a32-92a8-7c77bbf03376 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26135653561653762', 'path': '/dev/sdc'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 02:21:01 user nova-compute[72351]: DEBUG nova.objects.instance [None req-45912413-9589-4a32-92a8-7c77bbf03376 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Lazy-loading 'flavor' on Instance uuid 2d78475c-107d-4256-b12e-dbe9c8f3b9b2 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:21:01 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:01 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-45912413-9589-4a32-92a8-7c77bbf03376 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] attach device xml: <disk type="block" device="disk">
Aug 09 02:21:01 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:21:01 user nova-compute[72351]:   <source dev="/dev/sdc"/>
Aug 09 02:21:01 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:21:01 user nova-compute[72351]:   <serial>41ceb107-f270-4b7a-a2b0-14c2dbdd5497</serial>
Aug 09 02:21:01 user nova-compute[72351]: </disk>
Aug 09 02:21:01 user nova-compute[72351]:  {{(pid=72351) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Aug 09 02:21:01 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-45912413-9589-4a32-92a8-7c77bbf03376 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:21:01 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-45912413-9589-4a32-92a8-7c77bbf03376 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:21:01 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-45912413-9589-4a32-92a8-7c77bbf03376 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] No VIF found with MAC fa:16:3e:0a:89:6a, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:21:01 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-45912413-9589-4a32-92a8-7c77bbf03376 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Lock "2d78475c-107d-4256-b12e-dbe9c8f3b9b2" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 11.326s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:21:02 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:02 user nova-compute[72351]: INFO nova.compute.manager [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Rescuing
Aug 09 02:21:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Acquiring lock "refresh_cache-2d78475c-107d-4256-b12e-dbe9c8f3b9b2" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:21:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Acquired lock "refresh_cache-2d78475c-107d-4256-b12e-dbe9c8f3b9b2" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:21:02 user nova-compute[72351]: DEBUG nova.network.neutron [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:21:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:03 user nova-compute[72351]: DEBUG nova.network.neutron [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Updating instance_info_cache with network_info: [{"id": "ff25eb15-1af0-4110-9d9a-ecd0a1d4627d", "address": "fa:16:3e:0a:89:6a", "network": {"id": "fd1fe4e0-5d09-43c1-8dbc-1f60cb7f38d7", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1014783449-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.229", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "747d5750a2174aaea71849c70167a816", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapff25eb15-1a", "ovs_interfaceid": "ff25eb15-1af0-4110-9d9a-ecd0a1d4627d", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:21:03 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Releasing lock "refresh_cache-2d78475c-107d-4256-b12e-dbe9c8f3b9b2" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:21:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:03 user nova-compute[72351]: DEBUG nova.compute.manager [req-1dc30b19-4d12-4769-aa42-d8b59334e4d1 req-b8af66b5-f8c9-4ba9-b35f-a99b2c477131 service nova] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Received event network-vif-unplugged-ff25eb15-1af0-4110-9d9a-ecd0a1d4627d {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:21:03 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-1dc30b19-4d12-4769-aa42-d8b59334e4d1 req-b8af66b5-f8c9-4ba9-b35f-a99b2c477131 service nova] Acquiring lock "2d78475c-107d-4256-b12e-dbe9c8f3b9b2-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:21:03 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-1dc30b19-4d12-4769-aa42-d8b59334e4d1 req-b8af66b5-f8c9-4ba9-b35f-a99b2c477131 service nova] Lock "2d78475c-107d-4256-b12e-dbe9c8f3b9b2-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:21:03 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-1dc30b19-4d12-4769-aa42-d8b59334e4d1 req-b8af66b5-f8c9-4ba9-b35f-a99b2c477131 service nova] Lock "2d78475c-107d-4256-b12e-dbe9c8f3b9b2-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:21:03 user nova-compute[72351]: DEBUG nova.compute.manager [req-1dc30b19-4d12-4769-aa42-d8b59334e4d1 req-b8af66b5-f8c9-4ba9-b35f-a99b2c477131 service nova] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] No waiting events found dispatching network-vif-unplugged-ff25eb15-1af0-4110-9d9a-ecd0a1d4627d {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:21:03 user nova-compute[72351]: WARNING nova.compute.manager [req-1dc30b19-4d12-4769-aa42-d8b59334e4d1 req-b8af66b5-f8c9-4ba9-b35f-a99b2c477131 service nova] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Received unexpected event network-vif-unplugged-ff25eb15-1af0-4110-9d9a-ecd0a1d4627d for instance with vm_state active and task_state rescuing.
Aug 09 02:21:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:04 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Instance destroyed successfully.
Aug 09 02:21:04 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Attempting a stable device rescue
Aug 09 02:21:04 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] rescue generated disk_info: {'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vdb': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}, 'disk.rescue': {'bus': 'virtio', 'dev': 'vdc', 'type': 'disk'}}} {{(pid=72351) rescue /opt/stack/nova/nova/virt/libvirt/driver.py:4308}}
Aug 09 02:21:04 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Instance directory exists: not creating {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4713}}
Aug 09 02:21:04 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Creating image(s)
Aug 09 02:21:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Acquiring lock "/opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:21:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Lock "/opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:21:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Lock "/opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:21:04 user nova-compute[72351]: DEBUG nova.objects.instance [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Lazy-loading 'trusted_certs' on Instance uuid 2d78475c-107d-4256-b12e-dbe9c8f3b9b2 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:21:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Acquiring lock "90e420c550f5519f016964725db870eb284e3d12" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:21:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Lock "90e420c550f5519f016964725db870eb284e3d12" acquired by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:21:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/90e420c550f5519f016964725db870eb284e3d12.part --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/90e420c550f5519f016964725db870eb284e3d12.part --force-share --output=json" returned: 0 in 0.232s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:05 user nova-compute[72351]: DEBUG nova.virt.images [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] 05bf7646-45ca-4afe-bcb8-fa75842c5172 was qcow2, converting to raw {{(pid=72351) fetch_to_raw /opt/stack/nova/nova/virt/images.py:165}}
Aug 09 02:21:05 user nova-compute[72351]: DEBUG nova.privsep.utils [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=72351) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Aug 09 02:21:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Running cmd (subprocess): qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/90e420c550f5519f016964725db870eb284e3d12.part /opt/stack/data/nova/instances/_base/90e420c550f5519f016964725db870eb284e3d12.converted {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] CMD "qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/90e420c550f5519f016964725db870eb284e3d12.part /opt/stack/data/nova/instances/_base/90e420c550f5519f016964725db870eb284e3d12.converted" returned: 0 in 0.217s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/90e420c550f5519f016964725db870eb284e3d12.converted --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/90e420c550f5519f016964725db870eb284e3d12.converted --force-share --output=json" returned: 0 in 0.157s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Lock "90e420c550f5519f016964725db870eb284e3d12" "released" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: held 1.471s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:21:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Acquiring lock "90e420c550f5519f016964725db870eb284e3d12" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:21:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Lock "90e420c550f5519f016964725db870eb284e3d12" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:21:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/90e420c550f5519f016964725db870eb284e3d12 --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:05 user nova-compute[72351]: DEBUG nova.compute.manager [req-cea82199-0476-46ff-88d4-45498610ab21 req-2e7fd8dc-66f3-4283-a9c3-5150af69657c service nova] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Received event network-vif-plugged-ff25eb15-1af0-4110-9d9a-ecd0a1d4627d {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:21:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-cea82199-0476-46ff-88d4-45498610ab21 req-2e7fd8dc-66f3-4283-a9c3-5150af69657c service nova] Acquiring lock "2d78475c-107d-4256-b12e-dbe9c8f3b9b2-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:21:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-cea82199-0476-46ff-88d4-45498610ab21 req-2e7fd8dc-66f3-4283-a9c3-5150af69657c service nova] Lock "2d78475c-107d-4256-b12e-dbe9c8f3b9b2-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:21:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-cea82199-0476-46ff-88d4-45498610ab21 req-2e7fd8dc-66f3-4283-a9c3-5150af69657c service nova] Lock "2d78475c-107d-4256-b12e-dbe9c8f3b9b2-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:21:05 user nova-compute[72351]: DEBUG nova.compute.manager [req-cea82199-0476-46ff-88d4-45498610ab21 req-2e7fd8dc-66f3-4283-a9c3-5150af69657c service nova] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] No waiting events found dispatching network-vif-plugged-ff25eb15-1af0-4110-9d9a-ecd0a1d4627d {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:21:05 user nova-compute[72351]: WARNING nova.compute.manager [req-cea82199-0476-46ff-88d4-45498610ab21 req-2e7fd8dc-66f3-4283-a9c3-5150af69657c service nova] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Received unexpected event network-vif-plugged-ff25eb15-1af0-4110-9d9a-ecd0a1d4627d for instance with vm_state active and task_state rescuing.
Aug 09 02:21:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/90e420c550f5519f016964725db870eb284e3d12 --force-share --output=json" returned: 0 in 0.140s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/90e420c550f5519f016964725db870eb284e3d12,backing_fmt=raw /opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2/disk.rescue {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/90e420c550f5519f016964725db870eb284e3d12,backing_fmt=raw /opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2/disk.rescue" returned: 0 in 0.058s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Lock "90e420c550f5519f016964725db870eb284e3d12" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.206s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:21:05 user nova-compute[72351]: DEBUG nova.objects.instance [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Lazy-loading 'migration_context' on Instance uuid 2d78475c-107d-4256-b12e-dbe9c8f3b9b2 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Created local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4851}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Start _get_guest_xml network_info=[{"id": "ff25eb15-1af0-4110-9d9a-ecd0a1d4627d", "address": "fa:16:3e:0a:89:6a", "network": {"id": "fd1fe4e0-5d09-43c1-8dbc-1f60cb7f38d7", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1014783449-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.229", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerStableDeviceRescueTest-1014783449-network", "vif_mac": "fa:16:3e:0a:89:6a"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "747d5750a2174aaea71849c70167a816", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapff25eb15-1a", "ovs_interfaceid": "ff25eb15-1af0-4110-9d9a-ecd0a1d4627d", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vdb': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}, 'disk.rescue': {'bus': 'virtio', 'dev': 'vdc', 'type': 'disk'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>) rescue={'image_id': '05bf7646-45ca-4afe-bcb8-fa75842c5172', 'kernel_id': '', 'ramdisk_id': ''} block_device_info={'root_device_name': '/dev/vda', 'image': [{'size': 0, 'device_type': 'disk', 'encryption_secret_uuid': None, 'guest_format': None, 'encryption_options': None, 'disk_bus': 'virtio', 'encrypted': False, 'encryption_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'image_id': '18ccda7f-31d3-4978-ab9d-4ed2f74eff94'}], 'ephemerals': [], 'block_device_mapping': [{'device_type': 'disk', 'guest_format': None, 'disk_bus': 'virtio', 'mount_device': '/dev/vdb', 'boot_index': None, 'attachment_id': '242e142b-552b-447c-8088-f77e546f99f5', 'delete_on_termination': False, 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqns': ['iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'YaNtwOCg', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc'}, 'status': 'reserved', 'instance': '2d78475c-107d-4256-b12e-dbe9c8f3b9b2', 'attached_at': '', 'detached_at': '', 'volume_id': '41ceb107-f270-4b7a-a2b0-14c2dbdd5497', 'serial': '41ceb107-f270-4b7a-a2b0-14c2dbdd5497'}, 'volume_type': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG nova.objects.instance [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Lazy-loading 'resources' on Instance uuid 2d78475c-107d-4256-b12e-dbe9c8f3b9b2 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-2be30739-dcfa-425d-9440-7dfcce63ffaf tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Acquiring lock "11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-2be30739-dcfa-425d-9440-7dfcce63ffaf tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Lock "11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-2be30739-dcfa-425d-9440-7dfcce63ffaf tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Acquiring lock "11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-2be30739-dcfa-425d-9440-7dfcce63ffaf tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Lock "11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-2be30739-dcfa-425d-9440-7dfcce63ffaf tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Lock "11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:21:06 user nova-compute[72351]: INFO nova.compute.manager [None req-2be30739-dcfa-425d-9440-7dfcce63ffaf tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Terminating instance
Aug 09 02:21:06 user nova-compute[72351]: DEBUG nova.compute.manager [None req-2be30739-dcfa-425d-9440-7dfcce63ffaf tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Start destroying the instance on the hypervisor. {{(pid=72351) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3124}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG nova.objects.instance [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Lazy-loading 'numa_topology' on Instance uuid 2d78475c-107d-4256-b12e-dbe9c8f3b9b2 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:21:06 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:21:06 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:21:06 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG nova.objects.instance [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Lazy-loading 'vcpu_model' on Instance uuid 2d78475c-107d-4256-b12e-dbe9c8f3b9b2 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'YaNtwOCg', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path'}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:21:06 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:21:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:21:06 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:21:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:21:06 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 02:21:06 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497 -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497 -p 172.16.0.220:3260" returned: 0 in 0.026s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[f6654c5c-b2b3-45c8-94ac-3a35435b7f13]: (4, ('# BEGIN RECORD 2.1.5\nnode.name = iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497\nnode.tpgt = -1\nnode.startup = automatic\nnode.leading_login = No\niface.iscsi_ifacename = default\niface.net_ifacename = <empty>\niface.ipaddress = <empty>\niface.prefix_len = 0\niface.hwaddress = <empty>\niface.transport_name = tcp\niface.initiatorname = <empty>\niface.state = <empty>\niface.vlan_id = 0\niface.vlan_priority = 0\niface.vlan_state = <empty>\niface.iface_num = 0\niface.mtu = 0\niface.port = 0\niface.bootproto = <empty>\niface.subnet_mask = <empty>\niface.gateway = <empty>\niface.dhcp_alt_client_id_state = <empty>\niface.dhcp_alt_client_id = <empty>\niface.dhcp_dns = <empty>\niface.dhcp_learn_iqn = <empty>\niface.dhcp_req_vendor_id_state = <empty>\niface.dhcp_vendor_id_state = <empty>\niface.dhcp_vendor_id = <empty>\niface.dhcp_slp_da = <empty>\niface.fragmentation = <empty>\niface.gratuitous_arp = <empty>\niface.incoming_forwarding = <empty>\niface.tos_state = <empty>\niface.tos = 0\niface.ttl = 0\niface.delayed_ack = <empty>\niface.tcp_nagle = <empty>\niface.tcp_wsf_state = <empty>\niface.tcp_wsf = 0\niface.tcp_timer_scale = 0\niface.tcp_timestamp = <empty>\niface.redirect = <empty>\niface.def_task_mgmt_timeout = 0\niface.header_digest = <empty>\niface.data_digest = <empty>\niface.immediate_data = <empty>\niface.initial_r2t = <empty>\niface.data_seq_inorder = <empty>\niface.data_pdu_inorder = <empty>\niface.erl = 0\niface.max_receive_data_len = 0\niface.first_burst_len = 0\niface.max_outstanding_r2t = 0\niface.max_burst_len = 0\niface.chap_auth = <empty>\niface.bidi_chap = <empty>\niface.strict_login_compliance = <empty>\niface.discovery_auth = <empty>\niface.discovery_logout = <empty>\nnode.discovery_address = <empty>\nnode.discovery_port = 0\nnode.discovery_type = static\nnode.session.initial_cmdsn = 0\nnode.session.initial_login_retry_max = 8\nnode.session.xmit_thread_priority = -20\nnode.session.cmds_max = 128\nnode.session.queue_depth = 32\nnode.session.nr_sessions = 1\nnode.session.auth.authmethod = CHAP\nnode.session.auth.username = YaNtwOCg\nnode.session.auth.password = ********\nnode.session.auth.username_in = <empty>\nnode.session.auth.password_in = <empty>\nnode.session.auth.chap_algs = MD5\nnode.session.timeo.replacement_timeout = 120\nnode.session.err_timeo.abort_timeout = 15\nnode.session.err_timeo.lu_reset_timeout = 30\nnode.session.err_timeo.tgt_reset_timeout = 30\nnode.session.err_timeo.host_reset_timeout = 60\nnode.session.iscsi.FastAbort = Yes\nnode.session.iscsi.InitialR2T = No\nnode.session.iscsi.ImmediateData = Yes\nnode.session.iscsi.FirstBurstLength = 262144\nnode.session.iscsi.MaxBurstLength = 16776192\nnode.session.iscsi.DefaultTime2Retain = 0\nnode.session.iscsi.DefaultTime2Wait = 2\nnode.session.iscsi.MaxConnections = 1\nnode.session.iscsi.MaxOutstandingR2T = 1\nnode.session.iscsi.ERL = 0\nnode.session.scan = manual\nnode.session.reopen_max = 0\nnode.conn[0].address = 172.16.0.220\nnode.conn[0].port = 3260\nnode.conn[0].startup = manual\nnode.conn[0].tcp.window_size = 524288\nnode.conn[0].tcp.type_of_service = 0\nnode.conn[0].timeo.logout_timeout = 15\nnode.conn[0].timeo.login_timeout = 15\nnode.conn[0].timeo.auth_timeout = 45\nnode.conn[0].timeo.noop_out_interval = 5\nnode.conn[0].timeo.noop_out_timeout = 5\nnode.conn[0].iscsi.MaxXmitDataSegmentLength = 0\nnode.conn[0].iscsi.MaxRecvDataSegmentLength = 262144\nnode.conn[0].iscsi.HeaderDigest = None\nnode.conn[0].iscsi.DataDigest = None\nnode.conn[0].iscsi.IFMarker = No\nnode.conn[0].iscsi.OFMarker = No\n# END RECORD\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] iscsiadm (): stdout=# BEGIN RECORD 2.1.5
Aug 09 02:21:06 user nova-compute[72351]: node.name = iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497
Aug 09 02:21:06 user nova-compute[72351]: node.tpgt = -1
Aug 09 02:21:06 user nova-compute[72351]: node.startup = automatic
Aug 09 02:21:06 user nova-compute[72351]: node.leading_login = No
Aug 09 02:21:06 user nova-compute[72351]: iface.iscsi_ifacename = default
Aug 09 02:21:06 user nova-compute[72351]: iface.net_ifacename = <empty>
Aug 09 02:21:06 user nova-compute[72351]: iface.ipaddress = <empty>
Aug 09 02:21:06 user nova-compute[72351]: iface.prefix_len = 0
Aug 09 02:21:06 user nova-compute[72351]: iface.hwaddress = <empty>
Aug 09 02:21:06 user nova-compute[72351]: iface.transport_name = tcp
Aug 09 02:21:06 user nova-compute[72351]: iface.initiatorname = <empty>
Aug 09 02:21:06 user nova-compute[72351]: iface.state = <empty>
Aug 09 02:21:06 user nova-compute[72351]: iface.vlan_id = 0
Aug 09 02:21:06 user nova-compute[72351]: iface.vlan_priority = 0
Aug 09 02:21:06 user nova-compute[72351]: iface.vlan_state = <empty>
Aug 09 02:21:06 user nova-compute[72351]: iface.iface_num = 0
Aug 09 02:21:06 user nova-compute[72351]: iface.mtu = 0
Aug 09 02:21:06 user nova-compute[72351]: iface.port = 0
Aug 09 02:21:06 user nova-compute[72351]: iface.bootproto = <empty>
Aug 09 02:21:06 user nova-compute[72351]: iface.subnet_mask = <empty>
Aug 09 02:21:06 user nova-compute[72351]: iface.gateway = <empty>
Aug 09 02:21:06 user nova-compute[72351]: iface.dhcp_alt_client_id_state = <empty>
Aug 09 02:21:06 user nova-compute[72351]: iface.dhcp_alt_client_id = <empty>
Aug 09 02:21:06 user nova-compute[72351]: iface.dhcp_dns = <empty>
Aug 09 02:21:06 user nova-compute[72351]: iface.dhcp_learn_iqn = <empty>
Aug 09 02:21:06 user nova-compute[72351]: iface.dhcp_req_vendor_id_state = <empty>
Aug 09 02:21:06 user nova-compute[72351]: iface.dhcp_vendor_id_state = <empty>
Aug 09 02:21:06 user nova-compute[72351]: iface.dhcp_vendor_id = <empty>
Aug 09 02:21:06 user nova-compute[72351]: iface.dhcp_slp_da = <empty>
Aug 09 02:21:06 user nova-compute[72351]: iface.fragmentation = <empty>
Aug 09 02:21:06 user nova-compute[72351]: iface.gratuitous_arp = <empty>
Aug 09 02:21:06 user nova-compute[72351]: iface.incoming_forwarding = <empty>
Aug 09 02:21:06 user nova-compute[72351]: iface.tos_state = <empty>
Aug 09 02:21:06 user nova-compute[72351]: iface.tos = 0
Aug 09 02:21:06 user nova-compute[72351]: iface.ttl = 0
Aug 09 02:21:06 user nova-compute[72351]: iface.delayed_ack = <empty>
Aug 09 02:21:06 user nova-compute[72351]: iface.tcp_nagle = <empty>
Aug 09 02:21:06 user nova-compute[72351]: iface.tcp_wsf_state = <empty>
Aug 09 02:21:06 user nova-compute[72351]: iface.tcp_wsf = 0
Aug 09 02:21:06 user nova-compute[72351]: iface.tcp_timer_scale = 0
Aug 09 02:21:06 user nova-compute[72351]: iface.tcp_timestamp = <empty>
Aug 09 02:21:06 user nova-compute[72351]: iface.redirect = <empty>
Aug 09 02:21:06 user nova-compute[72351]: iface.def_task_mgmt_timeout = 0
Aug 09 02:21:06 user nova-compute[72351]: iface.header_digest = <empty>
Aug 09 02:21:06 user nova-compute[72351]: iface.data_digest = <empty>
Aug 09 02:21:06 user nova-compute[72351]: iface.immediate_data = <empty>
Aug 09 02:21:06 user nova-compute[72351]: iface.initial_r2t = <empty>
Aug 09 02:21:06 user nova-compute[72351]: iface.data_seq_inorder = <empty>
Aug 09 02:21:06 user nova-compute[72351]: iface.data_pdu_inorder = <empty>
Aug 09 02:21:06 user nova-compute[72351]: iface.erl = 0
Aug 09 02:21:06 user nova-compute[72351]: iface.max_receive_data_len = 0
Aug 09 02:21:06 user nova-compute[72351]: iface.first_burst_len = 0
Aug 09 02:21:06 user nova-compute[72351]: iface.max_outstanding_r2t = 0
Aug 09 02:21:06 user nova-compute[72351]: iface.max_burst_len = 0
Aug 09 02:21:06 user nova-compute[72351]: iface.chap_auth = <empty>
Aug 09 02:21:06 user nova-compute[72351]: iface.bidi_chap = <empty>
Aug 09 02:21:06 user nova-compute[72351]: iface.strict_login_compliance = <empty>
Aug 09 02:21:06 user nova-compute[72351]: iface.discovery_auth = <empty>
Aug 09 02:21:06 user nova-compute[72351]: iface.discovery_logout = <empty>
Aug 09 02:21:06 user nova-compute[72351]: node.discovery_address = <empty>
Aug 09 02:21:06 user nova-compute[72351]: node.discovery_port = 0
Aug 09 02:21:06 user nova-compute[72351]: node.discovery_type = static
Aug 09 02:21:06 user nova-compute[72351]: node.session.initial_cmdsn = 0
Aug 09 02:21:06 user nova-compute[72351]: node.session.initial_login_retry_max = 8
Aug 09 02:21:06 user nova-compute[72351]: node.session.xmit_thread_priority = -20
Aug 09 02:21:06 user nova-compute[72351]: node.session.cmds_max = 128
Aug 09 02:21:06 user nova-compute[72351]: node.session.queue_depth = 32
Aug 09 02:21:06 user nova-compute[72351]: node.session.nr_sessions = 1
Aug 09 02:21:06 user nova-compute[72351]: node.session.auth.authmethod = CHAP
Aug 09 02:21:06 user nova-compute[72351]: node.session.auth.username = YaNtwOCg
Aug 09 02:21:06 user nova-compute[72351]: node.session.auth.password = ***
Aug 09 02:21:06 user nova-compute[72351]: node.session.auth.username_in = <empty>
Aug 09 02:21:06 user nova-compute[72351]: node.session.auth.password_in = <empty>
Aug 09 02:21:06 user nova-compute[72351]: node.session.auth.chap_algs = MD5
Aug 09 02:21:06 user nova-compute[72351]: node.session.timeo.replacement_timeout = 120
Aug 09 02:21:06 user nova-compute[72351]: node.session.err_timeo.abort_timeout = 15
Aug 09 02:21:06 user nova-compute[72351]: node.session.err_timeo.lu_reset_timeout = 30
Aug 09 02:21:06 user nova-compute[72351]: node.session.err_timeo.tgt_reset_timeout = 30
Aug 09 02:21:06 user nova-compute[72351]: node.session.err_timeo.host_reset_timeout = 60
Aug 09 02:21:06 user nova-compute[72351]: node.session.iscsi.FastAbort = Yes
Aug 09 02:21:06 user nova-compute[72351]: node.session.iscsi.InitialR2T = No
Aug 09 02:21:06 user nova-compute[72351]: node.session.iscsi.ImmediateData = Yes
Aug 09 02:21:06 user nova-compute[72351]: node.session.iscsi.FirstBurstLength = 262144
Aug 09 02:21:06 user nova-compute[72351]: node.session.iscsi.MaxBurstLength = 16776192
Aug 09 02:21:06 user nova-compute[72351]: node.session.iscsi.DefaultTime2Retain = 0
Aug 09 02:21:06 user nova-compute[72351]: node.session.iscsi.DefaultTime2Wait = 2
Aug 09 02:21:06 user nova-compute[72351]: node.session.iscsi.MaxConnections = 1
Aug 09 02:21:06 user nova-compute[72351]: node.session.iscsi.MaxOutstandingR2T = 1
Aug 09 02:21:06 user nova-compute[72351]: node.session.iscsi.ERL = 0
Aug 09 02:21:06 user nova-compute[72351]: node.session.scan = manual
Aug 09 02:21:06 user nova-compute[72351]: node.session.reopen_max = 0
Aug 09 02:21:06 user nova-compute[72351]: node.conn[0].address = 172.16.0.220
Aug 09 02:21:06 user nova-compute[72351]: node.conn[0].port = 3260
Aug 09 02:21:06 user nova-compute[72351]: node.conn[0].startup = manual
Aug 09 02:21:06 user nova-compute[72351]: node.conn[0].tcp.window_size = 524288
Aug 09 02:21:06 user nova-compute[72351]: node.conn[0].tcp.type_of_service = 0
Aug 09 02:21:06 user nova-compute[72351]: node.conn[0].timeo.logout_timeout = 15
Aug 09 02:21:06 user nova-compute[72351]: node.conn[0].timeo.login_timeout = 15
Aug 09 02:21:06 user nova-compute[72351]: node.conn[0].timeo.auth_timeout = 45
Aug 09 02:21:06 user nova-compute[72351]: node.conn[0].timeo.noop_out_interval = 5
Aug 09 02:21:06 user nova-compute[72351]: node.conn[0].timeo.noop_out_timeout = 5
Aug 09 02:21:06 user nova-compute[72351]: node.conn[0].iscsi.MaxXmitDataSegmentLength = 0
Aug 09 02:21:06 user nova-compute[72351]: node.conn[0].iscsi.MaxRecvDataSegmentLength = 262144
Aug 09 02:21:06 user nova-compute[72351]: node.conn[0].iscsi.HeaderDigest = None
Aug 09 02:21:06 user nova-compute[72351]: node.conn[0].iscsi.DataDigest = None
Aug 09 02:21:06 user nova-compute[72351]: node.conn[0].iscsi.IFMarker = No
Aug 09 02:21:06 user nova-compute[72351]: node.conn[0].iscsi.OFMarker = No
Aug 09 02:21:06 user nova-compute[72351]: # END RECORD
Aug 09 02:21:06 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.027s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[18eb32e5-f65b-4a5e-bb77-0a75e4416e91]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.028s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ed20e832-733a-46d3-b957-fb2edd43c417]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v YaNtwOCg {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v YaNtwOCg" returned: 0 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e84347ad-63bc-46b6-91e8-8f14afbfa490]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'YaNtwOCg'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[86057bda-44ad-4085-b64e-021c512d5088]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.027s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[77e235ff-90fe-485e-a6aa-797977dbd240]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f (non-flash)\ntcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f (non-flash)
Aug 09 02:21:06 user nova-compute[72351]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497 (non-flash)
Aug 09 02:21:06 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f (non-flash)
Aug 09 02:21:06 user nova-compute[72351]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497 (non-flash)
Aug 09 02:21:06 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.182s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] HCTL ('34', '0', '0', 0) found on session 15 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Scanning host 34 c: 0, t: 0, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG nova.compute.manager [req-75894be8-45bd-4438-ae67-66495581629f req-2287a1fa-9dc4-4ac4-8604-4d2075cbeb31 service nova] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Received event network-vif-unplugged-a309cd5b-1679-48c6-a596-168b3af2d284 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-75894be8-45bd-4438-ae67-66495581629f req-2287a1fa-9dc4-4ac4-8604-4d2075cbeb31 service nova] Acquiring lock "11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-75894be8-45bd-4438-ae67-66495581629f req-2287a1fa-9dc4-4ac4-8604-4d2075cbeb31 service nova] Lock "11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-75894be8-45bd-4438-ae67-66495581629f req-2287a1fa-9dc4-4ac4-8604-4d2075cbeb31 service nova] Lock "11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG nova.compute.manager [req-75894be8-45bd-4438-ae67-66495581629f req-2287a1fa-9dc4-4ac4-8604-4d2075cbeb31 service nova] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] No waiting events found dispatching network-vif-unplugged-a309cd5b-1679-48c6-a596-168b3af2d284 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG nova.compute.manager [req-75894be8-45bd-4438-ae67-66495581629f req-2287a1fa-9dc4-4ac4-8604-4d2075cbeb31 service nova] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Received event network-vif-unplugged-a309cd5b-1679-48c6-a596-168b3af2d284 for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[04730c61-67c6-41f1-aad9-c30da95d9970]: (4, ('0 0 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Searching for a device in session 15 and hctl ('34', '0', '0', 0) yield: sdc {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Connected to sdc using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'YaNtwOCg', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc', 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 0.214s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] <== connect_volume: return (215ms) {'type': 'block', 'scsi_wwn': '26135653561653762', 'path': '/dev/sdc'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26135653561653762', 'path': '/dev/sdc'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:17:04Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerStableDeviceRescueTest-server-1763341201',display_name='tempest-ServerStableDeviceRescueTest-server-1763341201',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serverstabledevicerescuetest-server-1763341201',id=2,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBPLsnKPb6VYEzHtPIe2+hjQmOnmHJvA1Ev7/Puz1ZEEURwvwHmMhKqMbBr/N8dvlnbS9aRgyU274IVV3B6lTROiOE1e6tZ3774qYOPMP3+6C+tI9LZ4+zAxuQqNbLNbP+Q==',key_name='tempest-keypair-1212195529',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:17:18Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='747d5750a2174aaea71849c70167a816',ramdisk_id='',reservation_id='r-d0n2amoe',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerStableDeviceRescueTest-790832465',owner_user_name='tempest-ServerStableDeviceRescueTest-790832465-project-member'},tags=<?>,task_state='rescuing',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:19:04Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='157ad7fa9970477a91de3d1347dee441',uuid=2d78475c-107d-4256-b12e-dbe9c8f3b9b2,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "ff25eb15-1af0-4110-9d9a-ecd0a1d4627d", "address": "fa:16:3e:0a:89:6a", "network": {"id": "fd1fe4e0-5d09-43c1-8dbc-1f60cb7f38d7", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1014783449-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.229", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerStableDeviceRescueTest-1014783449-network", "vif_mac": "fa:16:3e:0a:89:6a"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "747d5750a2174aaea71849c70167a816", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapff25eb15-1a", "ovs_interfaceid": "ff25eb15-1af0-4110-9d9a-ecd0a1d4627d", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Converting VIF {"id": "ff25eb15-1af0-4110-9d9a-ecd0a1d4627d", "address": "fa:16:3e:0a:89:6a", "network": {"id": "fd1fe4e0-5d09-43c1-8dbc-1f60cb7f38d7", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1014783449-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.229", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerStableDeviceRescueTest-1014783449-network", "vif_mac": "fa:16:3e:0a:89:6a"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "747d5750a2174aaea71849c70167a816", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapff25eb15-1a", "ovs_interfaceid": "ff25eb15-1af0-4110-9d9a-ecd0a1d4627d", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:0a:89:6a,bridge_name='br-int',has_traffic_filtering=True,id=ff25eb15-1af0-4110-9d9a-ecd0a1d4627d,network=Network(fd1fe4e0-5d09-43c1-8dbc-1f60cb7f38d7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapff25eb15-1a') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG nova.objects.instance [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Lazy-loading 'pci_devices' on Instance uuid 2d78475c-107d-4256-b12e-dbe9c8f3b9b2 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:21:06 user nova-compute[72351]:   <uuid>2d78475c-107d-4256-b12e-dbe9c8f3b9b2</uuid>
Aug 09 02:21:06 user nova-compute[72351]:   <name>instance-00000002</name>
Aug 09 02:21:06 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:21:06 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:21:06 user nova-compute[72351]:   <metadata>
Aug 09 02:21:06 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:21:06 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:21:06 user nova-compute[72351]:       <nova:name>tempest-ServerStableDeviceRescueTest-server-1763341201</nova:name>
Aug 09 02:21:06 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:21:06</nova:creationTime>
Aug 09 02:21:06 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:21:06 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:21:06 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:21:06 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:21:06 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:21:06 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:21:06 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:21:06 user nova-compute[72351]:       <nova:owner>
Aug 09 02:21:06 user nova-compute[72351]:         <nova:user uuid="157ad7fa9970477a91de3d1347dee441">tempest-ServerStableDeviceRescueTest-790832465-project-member</nova:user>
Aug 09 02:21:06 user nova-compute[72351]:         <nova:project uuid="747d5750a2174aaea71849c70167a816">tempest-ServerStableDeviceRescueTest-790832465</nova:project>
Aug 09 02:21:06 user nova-compute[72351]:       </nova:owner>
Aug 09 02:21:06 user nova-compute[72351]:       <nova:root type="image" uuid="18ccda7f-31d3-4978-ab9d-4ed2f74eff94"/>
Aug 09 02:21:06 user nova-compute[72351]:       <nova:ports>
Aug 09 02:21:06 user nova-compute[72351]:         <nova:port uuid="ff25eb15-1af0-4110-9d9a-ecd0a1d4627d">
Aug 09 02:21:06 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.5" ipVersion="4"/>
Aug 09 02:21:06 user nova-compute[72351]:         </nova:port>
Aug 09 02:21:06 user nova-compute[72351]:       </nova:ports>
Aug 09 02:21:06 user nova-compute[72351]:     </nova:instance>
Aug 09 02:21:06 user nova-compute[72351]:   </metadata>
Aug 09 02:21:06 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:21:06 user nova-compute[72351]:     <system>
Aug 09 02:21:06 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:21:06 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:21:06 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:21:06 user nova-compute[72351]:       <entry name="serial">2d78475c-107d-4256-b12e-dbe9c8f3b9b2</entry>
Aug 09 02:21:06 user nova-compute[72351]:       <entry name="uuid">2d78475c-107d-4256-b12e-dbe9c8f3b9b2</entry>
Aug 09 02:21:06 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:21:06 user nova-compute[72351]:     </system>
Aug 09 02:21:06 user nova-compute[72351]:   </sysinfo>
Aug 09 02:21:06 user nova-compute[72351]:   <os>
Aug 09 02:21:06 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:21:06 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:21:06 user nova-compute[72351]:   </os>
Aug 09 02:21:06 user nova-compute[72351]:   <features>
Aug 09 02:21:06 user nova-compute[72351]:     <acpi/>
Aug 09 02:21:06 user nova-compute[72351]:     <apic/>
Aug 09 02:21:06 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:21:06 user nova-compute[72351]:   </features>
Aug 09 02:21:06 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:21:06 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:21:06 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:21:06 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:21:06 user nova-compute[72351]:   </clock>
Aug 09 02:21:06 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:21:06 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:21:06 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:21:06 user nova-compute[72351]:   </cpu>
Aug 09 02:21:06 user nova-compute[72351]:   <devices>
Aug 09 02:21:06 user nova-compute[72351]:     <disk type="file" device="disk">
Aug 09 02:21:06 user nova-compute[72351]:       <driver name="qemu" type="qcow2" cache="none"/>
Aug 09 02:21:06 user nova-compute[72351]:       <source file="/opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2/disk"/>
Aug 09 02:21:06 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:21:06 user nova-compute[72351]:     </disk>
Aug 09 02:21:06 user nova-compute[72351]:     <disk type="block" device="disk">
Aug 09 02:21:06 user nova-compute[72351]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:21:06 user nova-compute[72351]:       <source dev="/dev/sdc"/>
Aug 09 02:21:06 user nova-compute[72351]:       <target dev="vdb" bus="virtio"/>
Aug 09 02:21:06 user nova-compute[72351]:       <serial>41ceb107-f270-4b7a-a2b0-14c2dbdd5497</serial>
Aug 09 02:21:06 user nova-compute[72351]:     </disk>
Aug 09 02:21:06 user nova-compute[72351]:     <disk type="file" device="disk">
Aug 09 02:21:06 user nova-compute[72351]:       <driver name="qemu" type="qcow2" cache="none"/>
Aug 09 02:21:06 user nova-compute[72351]:       <source file="/opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2/disk.rescue"/>
Aug 09 02:21:06 user nova-compute[72351]:       <target dev="vdc" bus="virtio"/>
Aug 09 02:21:06 user nova-compute[72351]:       <boot order="1"/>
Aug 09 02:21:06 user nova-compute[72351]:     </disk>
Aug 09 02:21:06 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:21:06 user nova-compute[72351]:       <mac address="fa:16:3e:0a:89:6a"/>
Aug 09 02:21:06 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:21:06 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:21:06 user nova-compute[72351]:       <target dev="tapff25eb15-1a"/>
Aug 09 02:21:06 user nova-compute[72351]:     </interface>
Aug 09 02:21:06 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:21:06 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2/console.log" append="off"/>
Aug 09 02:21:06 user nova-compute[72351]:     </serial>
Aug 09 02:21:06 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:21:06 user nova-compute[72351]:     <video>
Aug 09 02:21:06 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:21:06 user nova-compute[72351]:     </video>
Aug 09 02:21:06 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:21:06 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:21:06 user nova-compute[72351]:     </rng>
Aug 09 02:21:06 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:21:06 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:21:06 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:21:06 user nova-compute[72351]:     </memballoon>
Aug 09 02:21:06 user nova-compute[72351]:   </devices>
Aug 09 02:21:06 user nova-compute[72351]: </domain>
Aug 09 02:21:06 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:21:06 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Instance destroyed successfully.
Aug 09 02:21:06 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] No BDM found with device name vdc, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] No VIF found with MAC fa:16:3e:0a:89:6a, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:06 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:07 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:07 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Instance destroyed successfully.
Aug 09 02:21:07 user nova-compute[72351]: DEBUG nova.objects.instance [None req-2be30739-dcfa-425d-9440-7dfcce63ffaf tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Lazy-loading 'resources' on Instance uuid 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:21:07 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-2be30739-dcfa-425d-9440-7dfcce63ffaf tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:17:19Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestVolumeSwap-server-119194024',display_name='tempest-TestVolumeSwap-server-119194024',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testvolumeswap-server-119194024',id=5,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBARbMYIq63VHbJ/MdcixbKY0L/h+74ffTkbUbHAN5B89Y/PUcG1NPaAWv1yx/WtztC1ALDJHBziTUBRaQrA68gE233M+Zzf5mD/gaihU2S129LWBcEuS8kxOAPQEOccAWw==',key_name='tempest-keypair-1608513669',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:17:36Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='1277988bc35041249b8e315df5259f98',ramdisk_id='',reservation_id='r-iosy2zx7',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TestVolumeSwap-646330225',owner_user_name='tempest-TestVolumeSwap-646330225-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:17:36Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='876134724b18462aba75896ddfad5705',uuid=11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "a309cd5b-1679-48c6-a596-168b3af2d284", "address": "fa:16:3e:dc:f0:3c", "network": {"id": "da9c66d8-08c8-4c9c-aec7-c9ad980b89e4", "bridge": "br-int", "label": "tempest-TestVolumeSwap-1327534036-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.16", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "1277988bc35041249b8e315df5259f98", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa309cd5b-16", "ovs_interfaceid": "a309cd5b-1679-48c6-a596-168b3af2d284", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:21:07 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-2be30739-dcfa-425d-9440-7dfcce63ffaf tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Converting VIF {"id": "a309cd5b-1679-48c6-a596-168b3af2d284", "address": "fa:16:3e:dc:f0:3c", "network": {"id": "da9c66d8-08c8-4c9c-aec7-c9ad980b89e4", "bridge": "br-int", "label": "tempest-TestVolumeSwap-1327534036-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.16", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "1277988bc35041249b8e315df5259f98", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa309cd5b-16", "ovs_interfaceid": "a309cd5b-1679-48c6-a596-168b3af2d284", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:21:07 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-2be30739-dcfa-425d-9440-7dfcce63ffaf tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:dc:f0:3c,bridge_name='br-int',has_traffic_filtering=True,id=a309cd5b-1679-48c6-a596-168b3af2d284,network=Network(da9c66d8-08c8-4c9c-aec7-c9ad980b89e4),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa309cd5b-16') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:21:07 user nova-compute[72351]: DEBUG os_vif [None req-2be30739-dcfa-425d-9440-7dfcce63ffaf tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:dc:f0:3c,bridge_name='br-int',has_traffic_filtering=True,id=a309cd5b-1679-48c6-a596-168b3af2d284,network=Network(da9c66d8-08c8-4c9c-aec7-c9ad980b89e4),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa309cd5b-16') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:21:07 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:07 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapa309cd5b-16, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:21:07 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:07 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:21:07 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:07 user nova-compute[72351]: INFO os_vif [None req-2be30739-dcfa-425d-9440-7dfcce63ffaf tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:dc:f0:3c,bridge_name='br-int',has_traffic_filtering=True,id=a309cd5b-1679-48c6-a596-168b3af2d284,network=Network(da9c66d8-08c8-4c9c-aec7-c9ad980b89e4),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa309cd5b-16')
Aug 09 02:21:07 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-2be30739-dcfa-425d-9440-7dfcce63ffaf tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Deleting instance files /opt/stack/data/nova/instances/11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b_del
Aug 09 02:21:07 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-2be30739-dcfa-425d-9440-7dfcce63ffaf tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Deletion of /opt/stack/data/nova/instances/11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b_del complete
Aug 09 02:21:07 user nova-compute[72351]: INFO nova.compute.manager [None req-2be30739-dcfa-425d-9440-7dfcce63ffaf tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Took 1.24 seconds to destroy the instance on the hypervisor.
Aug 09 02:21:07 user nova-compute[72351]: DEBUG oslo.service.loopingcall [None req-2be30739-dcfa-425d-9440-7dfcce63ffaf tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72351) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Aug 09 02:21:07 user nova-compute[72351]: DEBUG nova.compute.manager [-] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Deallocating network for instance {{(pid=72351) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2263}}
Aug 09 02:21:07 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] deallocate_for_instance() {{(pid=72351) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1795}}
Aug 09 02:21:07 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:07 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:07 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:07 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:07 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:07 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:07 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG nova.compute.manager [req-ec9cfb7c-0833-405a-acf7-97df832d5951 req-6c4ade80-67ac-4407-b318-fb4476c28f95 service nova] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Received event network-vif-plugged-a309cd5b-1679-48c6-a596-168b3af2d284 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ec9cfb7c-0833-405a-acf7-97df832d5951 req-6c4ade80-67ac-4407-b318-fb4476c28f95 service nova] Acquiring lock "11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ec9cfb7c-0833-405a-acf7-97df832d5951 req-6c4ade80-67ac-4407-b318-fb4476c28f95 service nova] Lock "11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ec9cfb7c-0833-405a-acf7-97df832d5951 req-6c4ade80-67ac-4407-b318-fb4476c28f95 service nova] Lock "11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG nova.compute.manager [req-ec9cfb7c-0833-405a-acf7-97df832d5951 req-6c4ade80-67ac-4407-b318-fb4476c28f95 service nova] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] No waiting events found dispatching network-vif-plugged-a309cd5b-1679-48c6-a596-168b3af2d284 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:21:08 user nova-compute[72351]: WARNING nova.compute.manager [req-ec9cfb7c-0833-405a-acf7-97df832d5951 req-6c4ade80-67ac-4407-b318-fb4476c28f95 service nova] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Received unexpected event network-vif-plugged-a309cd5b-1679-48c6-a596-168b3af2d284 for instance with vm_state active and task_state deleting.
Aug 09 02:21:08 user nova-compute[72351]: DEBUG nova.compute.manager [req-ec9cfb7c-0833-405a-acf7-97df832d5951 req-6c4ade80-67ac-4407-b318-fb4476c28f95 service nova] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Received event network-vif-plugged-a309cd5b-1679-48c6-a596-168b3af2d284 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ec9cfb7c-0833-405a-acf7-97df832d5951 req-6c4ade80-67ac-4407-b318-fb4476c28f95 service nova] Acquiring lock "11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ec9cfb7c-0833-405a-acf7-97df832d5951 req-6c4ade80-67ac-4407-b318-fb4476c28f95 service nova] Lock "11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ec9cfb7c-0833-405a-acf7-97df832d5951 req-6c4ade80-67ac-4407-b318-fb4476c28f95 service nova] Lock "11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG nova.compute.manager [req-ec9cfb7c-0833-405a-acf7-97df832d5951 req-6c4ade80-67ac-4407-b318-fb4476c28f95 service nova] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] No waiting events found dispatching network-vif-plugged-a309cd5b-1679-48c6-a596-168b3af2d284 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:21:08 user nova-compute[72351]: WARNING nova.compute.manager [req-ec9cfb7c-0833-405a-acf7-97df832d5951 req-6c4ade80-67ac-4407-b318-fb4476c28f95 service nova] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Received unexpected event network-vif-plugged-a309cd5b-1679-48c6-a596-168b3af2d284 for instance with vm_state active and task_state deleting.
Aug 09 02:21:08 user nova-compute[72351]: DEBUG nova.compute.manager [req-ec9cfb7c-0833-405a-acf7-97df832d5951 req-6c4ade80-67ac-4407-b318-fb4476c28f95 service nova] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Received event network-vif-plugged-a309cd5b-1679-48c6-a596-168b3af2d284 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ec9cfb7c-0833-405a-acf7-97df832d5951 req-6c4ade80-67ac-4407-b318-fb4476c28f95 service nova] Acquiring lock "11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ec9cfb7c-0833-405a-acf7-97df832d5951 req-6c4ade80-67ac-4407-b318-fb4476c28f95 service nova] Lock "11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ec9cfb7c-0833-405a-acf7-97df832d5951 req-6c4ade80-67ac-4407-b318-fb4476c28f95 service nova] Lock "11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG nova.compute.manager [req-ec9cfb7c-0833-405a-acf7-97df832d5951 req-6c4ade80-67ac-4407-b318-fb4476c28f95 service nova] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] No waiting events found dispatching network-vif-plugged-a309cd5b-1679-48c6-a596-168b3af2d284 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:21:08 user nova-compute[72351]: WARNING nova.compute.manager [req-ec9cfb7c-0833-405a-acf7-97df832d5951 req-6c4ade80-67ac-4407-b318-fb4476c28f95 service nova] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Received unexpected event network-vif-plugged-a309cd5b-1679-48c6-a596-168b3af2d284 for instance with vm_state active and task_state deleting.
Aug 09 02:21:08 user nova-compute[72351]: DEBUG nova.compute.manager [req-ec9cfb7c-0833-405a-acf7-97df832d5951 req-6c4ade80-67ac-4407-b318-fb4476c28f95 service nova] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Received event network-vif-unplugged-a309cd5b-1679-48c6-a596-168b3af2d284 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ec9cfb7c-0833-405a-acf7-97df832d5951 req-6c4ade80-67ac-4407-b318-fb4476c28f95 service nova] Acquiring lock "11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ec9cfb7c-0833-405a-acf7-97df832d5951 req-6c4ade80-67ac-4407-b318-fb4476c28f95 service nova] Lock "11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ec9cfb7c-0833-405a-acf7-97df832d5951 req-6c4ade80-67ac-4407-b318-fb4476c28f95 service nova] Lock "11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG nova.compute.manager [req-ec9cfb7c-0833-405a-acf7-97df832d5951 req-6c4ade80-67ac-4407-b318-fb4476c28f95 service nova] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] No waiting events found dispatching network-vif-unplugged-a309cd5b-1679-48c6-a596-168b3af2d284 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG nova.compute.manager [req-ec9cfb7c-0833-405a-acf7-97df832d5951 req-6c4ade80-67ac-4407-b318-fb4476c28f95 service nova] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Received event network-vif-unplugged-a309cd5b-1679-48c6-a596-168b3af2d284 for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG nova.compute.manager [req-cadeaeb6-30df-4f31-8179-2ecf5ffae0a0 req-f1bc2640-f08a-4344-95a4-e82f61d924cb service nova] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Received event network-vif-plugged-ff25eb15-1af0-4110-9d9a-ecd0a1d4627d {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-cadeaeb6-30df-4f31-8179-2ecf5ffae0a0 req-f1bc2640-f08a-4344-95a4-e82f61d924cb service nova] Acquiring lock "2d78475c-107d-4256-b12e-dbe9c8f3b9b2-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-cadeaeb6-30df-4f31-8179-2ecf5ffae0a0 req-f1bc2640-f08a-4344-95a4-e82f61d924cb service nova] Lock "2d78475c-107d-4256-b12e-dbe9c8f3b9b2-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-cadeaeb6-30df-4f31-8179-2ecf5ffae0a0 req-f1bc2640-f08a-4344-95a4-e82f61d924cb service nova] Lock "2d78475c-107d-4256-b12e-dbe9c8f3b9b2-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG nova.compute.manager [req-cadeaeb6-30df-4f31-8179-2ecf5ffae0a0 req-f1bc2640-f08a-4344-95a4-e82f61d924cb service nova] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] No waiting events found dispatching network-vif-plugged-ff25eb15-1af0-4110-9d9a-ecd0a1d4627d {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:21:08 user nova-compute[72351]: WARNING nova.compute.manager [req-cadeaeb6-30df-4f31-8179-2ecf5ffae0a0 req-f1bc2640-f08a-4344-95a4-e82f61d924cb service nova] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Received unexpected event network-vif-plugged-ff25eb15-1af0-4110-9d9a-ecd0a1d4627d for instance with vm_state active and task_state rescuing.
Aug 09 02:21:08 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:21:08 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Took 1.47 seconds to deallocate network for instance.
Aug 09 02:21:08 user nova-compute[72351]: DEBUG nova.compute.manager [req-28a29ef6-9379-421c-b051-0c60674ae686 req-d6dda885-742e-47d2-818f-9e5e445c57dc service nova] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Received event network-vif-deleted-a309cd5b-1679-48c6-a596-168b3af2d284 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-20131e30-410d-4ae0-8c83-1e5c1cdc6b4b tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-20131e30-410d-4ae0-8c83-1e5c1cdc6b4b tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:a4f329c4-9137-4cd0-971e-7a7402ba1785'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'FfmdKzlS', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:21:08 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-20131e30-410d-4ae0-8c83-1e5c1cdc6b4b tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:21:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-20131e30-410d-4ae0-8c83-1e5c1cdc6b4b tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-20131e30-410d-4ae0-8c83-1e5c1cdc6b4b tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:21:08 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-20131e30-410d-4ae0-8c83-1e5c1cdc6b4b tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:21:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-20131e30-410d-4ae0-8c83-1e5c1cdc6b4b tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:a4f329c4-9137-4cd0-971e-7a7402ba1785" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-20131e30-410d-4ae0-8c83-1e5c1cdc6b4b tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:a4f329c4-9137-4cd0-971e-7a7402ba1785" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:21:08 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-20131e30-410d-4ae0-8c83-1e5c1cdc6b4b tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 02:21:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a4f329c4-9137-4cd0-971e-7a7402ba1785 -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a4f329c4-9137-4cd0-971e-7a7402ba1785 -p 172.16.0.220:3260" returned: 21 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[965e0a77-4068-46b9-aa9e-089a7ccb9ac9]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-20131e30-410d-4ae0-8c83-1e5c1cdc6b4b tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Aug 09 02:21:08 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a4f329c4-9137-4cd0-971e-7a7402ba1785 -p 172.16.0.220:3260 --interface default --op new {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a4f329c4-9137-4cd0-971e-7a7402ba1785 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[fe9e0a56-382a-431c-b6da-764aabdda2d9]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:a4f329c4-9137-4cd0-971e-7a7402ba1785] added\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-20131e30-410d-4ae0-8c83-1e5c1cdc6b4b tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:a4f329c4-9137-4cd0-971e-7a7402ba1785] added
Aug 09 02:21:08 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a4f329c4-9137-4cd0-971e-7a7402ba1785 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a4f329c4-9137-4cd0-971e-7a7402ba1785 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[9cbff18a-6a6a-42fd-bd70-ebc7383c418d]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-20131e30-410d-4ae0-8c83-1e5c1cdc6b4b tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-2be30739-dcfa-425d-9440-7dfcce63ffaf tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-2be30739-dcfa-425d-9440-7dfcce63ffaf tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a4f329c4-9137-4cd0-971e-7a7402ba1785 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a4f329c4-9137-4cd0-971e-7a7402ba1785 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[9f977b39-4822-4cf1-80c8-de5f16c11634]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-20131e30-410d-4ae0-8c83-1e5c1cdc6b4b tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a4f329c4-9137-4cd0-971e-7a7402ba1785 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v FfmdKzlS {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a4f329c4-9137-4cd0-971e-7a7402ba1785 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v FfmdKzlS" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d063450d-8d9d-4b3c-ac38-6d4653c37e95]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-20131e30-410d-4ae0-8c83-1e5c1cdc6b4b tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'FfmdKzlS'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a4f329c4-9137-4cd0-971e-7a7402ba1785 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a4f329c4-9137-4cd0-971e-7a7402ba1785 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[2500966b-c8b7-4916-a32e-c6d27d5445d1]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-20131e30-410d-4ae0-8c83-1e5c1cdc6b4b tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[0d158406-70d5-403c-8259-3042e3e269c4]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f (non-flash)\ntcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-20131e30-410d-4ae0-8c83-1e5c1cdc6b4b tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f (non-flash)
Aug 09 02:21:08 user nova-compute[72351]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497 (non-flash)
Aug 09 02:21:08 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-20131e30-410d-4ae0-8c83-1e5c1cdc6b4b tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f (non-flash)
Aug 09 02:21:08 user nova-compute[72351]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497 (non-flash)
Aug 09 02:21:08 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a4f329c4-9137-4cd0-971e-7a7402ba1785 -p 172.16.0.220:3260 --login {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:09 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-2be30739-dcfa-425d-9440-7dfcce63ffaf tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:21:09 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-2be30739-dcfa-425d-9440-7dfcce63ffaf tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:21:09 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-2be30739-dcfa-425d-9440-7dfcce63ffaf tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.322s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:21:09 user nova-compute[72351]: INFO nova.scheduler.client.report [None req-2be30739-dcfa-425d-9440-7dfcce63ffaf tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Deleted allocations for instance 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b
Aug 09 02:21:09 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-2be30739-dcfa-425d-9440-7dfcce63ffaf tempest-TestVolumeSwap-646330225 tempest-TestVolumeSwap-646330225-project-member] Lock "11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 3.268s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:21:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Acquiring lock "75bbbead-8628-448b-9484-19bf4ab922ff" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:21:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Lock "75bbbead-8628-448b-9484-19bf4ab922ff" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:21:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a4f329c4-9137-4cd0-971e-7a7402ba1785 -p 172.16.0.220:3260 --login" returned: 0 in 1.079s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:10 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[8c5d304f-2c65-4abb-8d96-e4b2ccd9ee7a]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:a4f329c4-9137-4cd0-971e-7a7402ba1785, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:a4f329c4-9137-4cd0-971e-7a7402ba1785, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:21:10 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-20131e30-410d-4ae0-8c83-1e5c1cdc6b4b tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:a4f329c4-9137-4cd0-971e-7a7402ba1785, portal: 172.16.0.220,3260]
Aug 09 02:21:10 user nova-compute[72351]: Login to [iface: default, target: iqn.2010-10.org.openstack:a4f329c4-9137-4cd0-971e-7a7402ba1785, portal: 172.16.0.220,3260] successful.
Aug 09 02:21:10 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:21:10 user nova-compute[72351]: DEBUG nova.compute.manager [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Starting instance... {{(pid=72351) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2406}}
Aug 09 02:21:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a4f329c4-9137-4cd0-971e-7a7402ba1785 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a4f329c4-9137-4cd0-971e-7a7402ba1785 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.025s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:10 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[7677314c-b68e-4509-b59a-513b80af6c0e]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:21:10 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-20131e30-410d-4ae0-8c83-1e5c1cdc6b4b tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:21:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:10 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[c0248e0d-0bf3-47ff-b5a2-44402d2b0951]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f (non-flash)\ntcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497 (non-flash)\ntcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a4f329c4-9137-4cd0-971e-7a7402ba1785 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:21:10 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-20131e30-410d-4ae0-8c83-1e5c1cdc6b4b tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f (non-flash)
Aug 09 02:21:10 user nova-compute[72351]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497 (non-flash)
Aug 09 02:21:10 user nova-compute[72351]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a4f329c4-9137-4cd0-971e-7a7402ba1785 (non-flash)
Aug 09 02:21:10 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:21:10 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-20131e30-410d-4ae0-8c83-1e5c1cdc6b4b tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f (non-flash)
Aug 09 02:21:10 user nova-compute[72351]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497 (non-flash)
Aug 09 02:21:10 user nova-compute[72351]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a4f329c4-9137-4cd0-971e-7a7402ba1785 (non-flash)
Aug 09 02:21:10 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:21:10 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-20131e30-410d-4ae0-8c83-1e5c1cdc6b4b tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:a4f329c4-9137-4cd0-971e-7a7402ba1785" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 1.317s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:21:10 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-20131e30-410d-4ae0-8c83-1e5c1cdc6b4b tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 02:21:10 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-20131e30-410d-4ae0-8c83-1e5c1cdc6b4b tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] HCTL ('35', '-', '-', 0) found on session 16 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 02:21:10 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-20131e30-410d-4ae0-8c83-1e5c1cdc6b4b tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Scanning host 35 c: -, t: -, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 02:21:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host35/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host35/scan" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:10 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ce02620d-4b2d-4ae1-9017-383c31c6c60d]: (4, ('- - 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:21:10 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-20131e30-410d-4ae0-8c83-1e5c1cdc6b4b tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Searching for a device in session 16 and hctl ['35', '*', '*', 0] yield: sdd {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:21:10 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-20131e30-410d-4ae0-8c83-1e5c1cdc6b4b tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Connected to sdd using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'FfmdKzlS', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 02:21:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:21:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:21:10 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72351) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Aug 09 02:21:10 user nova-compute[72351]: INFO nova.compute.claims [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Claim successful on node user
Aug 09 02:21:10 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:21:10 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:21:10 user nova-compute[72351]: DEBUG nova.compute.manager [req-9cb0bbd9-b75e-4ebf-82e1-6418665dff9f req-66424ac9-0f86-4a9f-ba0e-eda4fb5db4f0 service nova] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Received event network-vif-plugged-a309cd5b-1679-48c6-a596-168b3af2d284 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:21:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-9cb0bbd9-b75e-4ebf-82e1-6418665dff9f req-66424ac9-0f86-4a9f-ba0e-eda4fb5db4f0 service nova] Acquiring lock "11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:21:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-9cb0bbd9-b75e-4ebf-82e1-6418665dff9f req-66424ac9-0f86-4a9f-ba0e-eda4fb5db4f0 service nova] Lock "11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:21:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-9cb0bbd9-b75e-4ebf-82e1-6418665dff9f req-66424ac9-0f86-4a9f-ba0e-eda4fb5db4f0 service nova] Lock "11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:21:10 user nova-compute[72351]: DEBUG nova.compute.manager [req-9cb0bbd9-b75e-4ebf-82e1-6418665dff9f req-66424ac9-0f86-4a9f-ba0e-eda4fb5db4f0 service nova] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] No waiting events found dispatching network-vif-plugged-a309cd5b-1679-48c6-a596-168b3af2d284 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:21:10 user nova-compute[72351]: WARNING nova.compute.manager [req-9cb0bbd9-b75e-4ebf-82e1-6418665dff9f req-66424ac9-0f86-4a9f-ba0e-eda4fb5db4f0 service nova] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Received unexpected event network-vif-plugged-a309cd5b-1679-48c6-a596-168b3af2d284 for instance with vm_state deleted and task_state None.
Aug 09 02:21:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.424s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:21:10 user nova-compute[72351]: DEBUG nova.compute.manager [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Start building networks asynchronously for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2803}}
Aug 09 02:21:10 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Removed pending event for 2d78475c-107d-4256-b12e-dbe9c8f3b9b2 due to event {{(pid=72351) _event_emit_delayed /opt/stack/nova/nova/virt/libvirt/host.py:438}}
Aug 09 02:21:10 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547670.6045005, 2d78475c-107d-4256-b12e-dbe9c8f3b9b2 => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:21:10 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] VM Resumed (Lifecycle Event)
Aug 09 02:21:10 user nova-compute[72351]: DEBUG nova.compute.manager [None req-338f52a9-efe3-4233-85c7-39938c431468 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:21:10 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:21:10 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: active, current task_state: rescuing, current DB power_state: 1, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:21:10 user nova-compute[72351]: DEBUG nova.compute.manager [req-65d30465-bc83-41a0-9532-a42b25ba8723 req-eda2c499-c3f3-444f-a1c1-3145e750ffd8 service nova] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Received event network-vif-plugged-ff25eb15-1af0-4110-9d9a-ecd0a1d4627d {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:21:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-65d30465-bc83-41a0-9532-a42b25ba8723 req-eda2c499-c3f3-444f-a1c1-3145e750ffd8 service nova] Acquiring lock "2d78475c-107d-4256-b12e-dbe9c8f3b9b2-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:21:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-65d30465-bc83-41a0-9532-a42b25ba8723 req-eda2c499-c3f3-444f-a1c1-3145e750ffd8 service nova] Lock "2d78475c-107d-4256-b12e-dbe9c8f3b9b2-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:21:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-65d30465-bc83-41a0-9532-a42b25ba8723 req-eda2c499-c3f3-444f-a1c1-3145e750ffd8 service nova] Lock "2d78475c-107d-4256-b12e-dbe9c8f3b9b2-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:21:10 user nova-compute[72351]: DEBUG nova.compute.manager [req-65d30465-bc83-41a0-9532-a42b25ba8723 req-eda2c499-c3f3-444f-a1c1-3145e750ffd8 service nova] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] No waiting events found dispatching network-vif-plugged-ff25eb15-1af0-4110-9d9a-ecd0a1d4627d {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:21:10 user nova-compute[72351]: WARNING nova.compute.manager [req-65d30465-bc83-41a0-9532-a42b25ba8723 req-eda2c499-c3f3-444f-a1c1-3145e750ffd8 service nova] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Received unexpected event network-vif-plugged-ff25eb15-1af0-4110-9d9a-ecd0a1d4627d for instance with vm_state active and task_state rescuing.
Aug 09 02:21:10 user nova-compute[72351]: DEBUG nova.compute.manager [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Allocating IP information in the background. {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1956}}
Aug 09 02:21:10 user nova-compute[72351]: DEBUG nova.network.neutron [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] allocate_for_instance() {{(pid=72351) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Aug 09 02:21:10 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] During sync_power_state the instance has a pending task (rescuing). Skip.
Aug 09 02:21:10 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547670.6063292, 2d78475c-107d-4256-b12e-dbe9c8f3b9b2 => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:21:10 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] VM Started (Lifecycle Event)
Aug 09 02:21:10 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Aug 09 02:21:10 user nova-compute[72351]: DEBUG nova.compute.manager [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Start building block device mappings for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2838}}
Aug 09 02:21:10 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:21:10 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Synchronizing instance power state after lifecycle event "Started"; current vm_state: rescued, current task_state: None, current DB power_state: 1, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:21:10 user nova-compute[72351]: DEBUG nova.policy [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '19668b0c790d49dd830c39e3c2be29bf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc90e94bbcb54fb6b86a9ac45edb596a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72351) authorize /opt/stack/nova/nova/policy.py:203}}
Aug 09 02:21:10 user nova-compute[72351]: DEBUG nova.compute.manager [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Start spawning the instance on the hypervisor. {{(pid=72351) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2612}}
Aug 09 02:21:10 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Creating instance directory {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4717}}
Aug 09 02:21:10 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Creating image(s)
Aug 09 02:21:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Acquiring lock "/opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:21:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Lock "/opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:21:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Lock "/opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:21:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.163s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Acquiring lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:21:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:21:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:11 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-20131e30-410d-4ae0-8c83-1e5c1cdc6b4b tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 2.354s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:21:11 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-20131e30-410d-4ae0-8c83-1e5c1cdc6b4b tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] <== connect_volume: return (2358ms) {'type': 'block', 'scsi_wwn': '23161386535316236', 'path': '/dev/sdd'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:21:11 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-20131e30-410d-4ae0-8c83-1e5c1cdc6b4b tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23161386535316236', 'path': '/dev/sdd'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 02:21:11 user nova-compute[72351]: DEBUG nova.objects.instance [None req-20131e30-410d-4ae0-8c83-1e5c1cdc6b4b tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Lazy-loading 'flavor' on Instance uuid 822e25c4-8dd1-4e21-b0ed-f340d59c6c98 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:21:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.198s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff/disk 1073741824 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff/disk 1073741824" returned: 0 in 0.055s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.264s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:21:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:11 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-20131e30-410d-4ae0-8c83-1e5c1cdc6b4b tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] attach device xml: <disk type="block" device="disk">
Aug 09 02:21:11 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:21:11 user nova-compute[72351]:   <source dev="/dev/sdd"/>
Aug 09 02:21:11 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:21:11 user nova-compute[72351]:   <serial>a4f329c4-9137-4cd0-971e-7a7402ba1785</serial>
Aug 09 02:21:11 user nova-compute[72351]: </disk>
Aug 09 02:21:11 user nova-compute[72351]:  {{(pid=72351) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Aug 09 02:21:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.165s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:11 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Checking if we can resize image /opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff/disk. size=1073741824 {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Aug 09 02:21:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:11 user nova-compute[72351]: DEBUG nova.network.neutron [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Successfully created port: be4ec483-8532-45f7-bb9f-0b50952ce94b {{(pid=72351) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Aug 09 02:21:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff/disk --force-share --output=json" returned: 0 in 0.153s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:11 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Cannot resize image /opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff/disk to a smaller size. {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Aug 09 02:21:11 user nova-compute[72351]: DEBUG nova.objects.instance [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Lazy-loading 'migration_context' on Instance uuid 75bbbead-8628-448b-9484-19bf4ab922ff {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:21:11 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Created local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4851}}
Aug 09 02:21:11 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Ensure instance console log exists: /opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff/console.log {{(pid=72351) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4603}}
Aug 09 02:21:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:21:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:21:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:21:11 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-20131e30-410d-4ae0-8c83-1e5c1cdc6b4b tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:21:11 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-20131e30-410d-4ae0-8c83-1e5c1cdc6b4b tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:21:11 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-20131e30-410d-4ae0-8c83-1e5c1cdc6b4b tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] No VIF found with MAC fa:16:3e:e4:67:dd, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:21:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-20131e30-410d-4ae0-8c83-1e5c1cdc6b4b tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Lock "822e25c4-8dd1-4e21-b0ed-f340d59c6c98" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 13.671s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:21:12 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:12 user nova-compute[72351]: DEBUG nova.network.neutron [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Successfully updated port: be4ec483-8532-45f7-bb9f-0b50952ce94b {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:21:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Acquiring lock "refresh_cache-75bbbead-8628-448b-9484-19bf4ab922ff" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:21:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Acquired lock "refresh_cache-75bbbead-8628-448b-9484-19bf4ab922ff" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:21:12 user nova-compute[72351]: DEBUG nova.network.neutron [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:21:12 user nova-compute[72351]: DEBUG nova.compute.manager [req-117b4ea2-0068-4c8f-9e78-6f2b1e9892e9 req-0d541e2e-754c-4480-88f5-04496811cca8 service nova] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Received event network-changed-be4ec483-8532-45f7-bb9f-0b50952ce94b {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:21:12 user nova-compute[72351]: DEBUG nova.compute.manager [req-117b4ea2-0068-4c8f-9e78-6f2b1e9892e9 req-0d541e2e-754c-4480-88f5-04496811cca8 service nova] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Refreshing instance network info cache due to event network-changed-be4ec483-8532-45f7-bb9f-0b50952ce94b. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:21:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-117b4ea2-0068-4c8f-9e78-6f2b1e9892e9 req-0d541e2e-754c-4480-88f5-04496811cca8 service nova] Acquiring lock "refresh_cache-75bbbead-8628-448b-9484-19bf4ab922ff" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:21:12 user nova-compute[72351]: DEBUG nova.network.neutron [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:21:12 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:12 user nova-compute[72351]: DEBUG nova.network.neutron [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Updating instance_info_cache with network_info: [{"id": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "address": "fa:16:3e:8b:b5:2e", "network": {"id": "8231f2c2-237e-41c6-8892-720b950c1453", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherB-1917380761-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fc90e94bbcb54fb6b86a9ac45edb596a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbe4ec483-85", "ovs_interfaceid": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:21:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Releasing lock "refresh_cache-75bbbead-8628-448b-9484-19bf4ab922ff" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:21:12 user nova-compute[72351]: DEBUG nova.compute.manager [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Instance network_info: |[{"id": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "address": "fa:16:3e:8b:b5:2e", "network": {"id": "8231f2c2-237e-41c6-8892-720b950c1453", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherB-1917380761-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fc90e94bbcb54fb6b86a9ac45edb596a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbe4ec483-85", "ovs_interfaceid": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1971}}
Aug 09 02:21:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-117b4ea2-0068-4c8f-9e78-6f2b1e9892e9 req-0d541e2e-754c-4480-88f5-04496811cca8 service nova] Acquired lock "refresh_cache-75bbbead-8628-448b-9484-19bf4ab922ff" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:21:12 user nova-compute[72351]: DEBUG nova.network.neutron [req-117b4ea2-0068-4c8f-9e78-6f2b1e9892e9 req-0d541e2e-754c-4480-88f5-04496811cca8 service nova] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Refreshing network info cache for port be4ec483-8532-45f7-bb9f-0b50952ce94b {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:21:12 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Start _get_guest_xml network_info=[{"id": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "address": "fa:16:3e:8b:b5:2e", "network": {"id": "8231f2c2-237e-41c6-8892-720b950c1453", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherB-1917380761-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fc90e94bbcb54fb6b86a9ac45edb596a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbe4ec483-85", "ovs_interfaceid": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'size': 0, 'device_type': 'disk', 'encryption_secret_uuid': None, 'guest_format': None, 'encryption_options': None, 'disk_bus': 'virtio', 'encrypted': False, 'encryption_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'image_id': '18ccda7f-31d3-4978-ab9d-4ed2f74eff94'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:21:12 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:21:12 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:21:12 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:21:12 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:21:12 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:21:12 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:21:12 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:21:12 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:21:12 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:21:12 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:21:12 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:21:12 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:21:12 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:21:12 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:21:12 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:21:12 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:21:12 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:21:12 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:21:12 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:21:09Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerActionsTestOtherB-server-634605118',display_name='tempest-ServerActionsTestOtherB-server-634605118',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serveractionstestotherb-server-634605118',id=15,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLdtpWQKmHav2tGxZemUVMEaT9TXyD7Duhy5QV84PPI+5KrgLVkbxVzWgryocScOebEprbcJ80bjBPnYhLtQJYSe5MXXz6GOQHLb+tJCZajhrDoMbn41/bPAnB5B3oFwCw==',key_name='tempest-keypair-644333596',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='fc90e94bbcb54fb6b86a9ac45edb596a',ramdisk_id='',reservation_id='r-ozz8gdfj',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerActionsTestOtherB-1726991791',owner_user_name='tempest-ServerActionsTestOtherB-1726991791-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:21:11Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='19668b0c790d49dd830c39e3c2be29bf',uuid=75bbbead-8628-448b-9484-19bf4ab922ff,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "address": "fa:16:3e:8b:b5:2e", "network": {"id": "8231f2c2-237e-41c6-8892-720b950c1453", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherB-1917380761-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fc90e94bbcb54fb6b86a9ac45edb596a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbe4ec483-85", "ovs_interfaceid": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:21:12 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Converting VIF {"id": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "address": "fa:16:3e:8b:b5:2e", "network": {"id": "8231f2c2-237e-41c6-8892-720b950c1453", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherB-1917380761-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fc90e94bbcb54fb6b86a9ac45edb596a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbe4ec483-85", "ovs_interfaceid": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:21:12 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:8b:b5:2e,bridge_name='br-int',has_traffic_filtering=True,id=be4ec483-8532-45f7-bb9f-0b50952ce94b,network=Network(8231f2c2-237e-41c6-8892-720b950c1453),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapbe4ec483-85') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:21:12 user nova-compute[72351]: DEBUG nova.objects.instance [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Lazy-loading 'pci_devices' on Instance uuid 75bbbead-8628-448b-9484-19bf4ab922ff {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:21:12 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:21:12 user nova-compute[72351]:   <uuid>75bbbead-8628-448b-9484-19bf4ab922ff</uuid>
Aug 09 02:21:12 user nova-compute[72351]:   <name>instance-0000000f</name>
Aug 09 02:21:12 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:21:12 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:21:12 user nova-compute[72351]:   <metadata>
Aug 09 02:21:12 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:21:12 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:21:12 user nova-compute[72351]:       <nova:name>tempest-ServerActionsTestOtherB-server-634605118</nova:name>
Aug 09 02:21:12 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:21:12</nova:creationTime>
Aug 09 02:21:12 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:21:12 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:21:12 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:21:12 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:21:12 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:21:12 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:21:12 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:21:12 user nova-compute[72351]:       <nova:owner>
Aug 09 02:21:12 user nova-compute[72351]:         <nova:user uuid="19668b0c790d49dd830c39e3c2be29bf">tempest-ServerActionsTestOtherB-1726991791-project-member</nova:user>
Aug 09 02:21:12 user nova-compute[72351]:         <nova:project uuid="fc90e94bbcb54fb6b86a9ac45edb596a">tempest-ServerActionsTestOtherB-1726991791</nova:project>
Aug 09 02:21:12 user nova-compute[72351]:       </nova:owner>
Aug 09 02:21:12 user nova-compute[72351]:       <nova:root type="image" uuid="18ccda7f-31d3-4978-ab9d-4ed2f74eff94"/>
Aug 09 02:21:12 user nova-compute[72351]:       <nova:ports>
Aug 09 02:21:12 user nova-compute[72351]:         <nova:port uuid="be4ec483-8532-45f7-bb9f-0b50952ce94b">
Aug 09 02:21:12 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.14" ipVersion="4"/>
Aug 09 02:21:12 user nova-compute[72351]:         </nova:port>
Aug 09 02:21:12 user nova-compute[72351]:       </nova:ports>
Aug 09 02:21:12 user nova-compute[72351]:     </nova:instance>
Aug 09 02:21:12 user nova-compute[72351]:   </metadata>
Aug 09 02:21:12 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:21:12 user nova-compute[72351]:     <system>
Aug 09 02:21:12 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:21:12 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:21:12 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:21:12 user nova-compute[72351]:       <entry name="serial">75bbbead-8628-448b-9484-19bf4ab922ff</entry>
Aug 09 02:21:12 user nova-compute[72351]:       <entry name="uuid">75bbbead-8628-448b-9484-19bf4ab922ff</entry>
Aug 09 02:21:12 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:21:12 user nova-compute[72351]:     </system>
Aug 09 02:21:12 user nova-compute[72351]:   </sysinfo>
Aug 09 02:21:12 user nova-compute[72351]:   <os>
Aug 09 02:21:12 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:21:12 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:21:12 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:21:12 user nova-compute[72351]:   </os>
Aug 09 02:21:12 user nova-compute[72351]:   <features>
Aug 09 02:21:12 user nova-compute[72351]:     <acpi/>
Aug 09 02:21:12 user nova-compute[72351]:     <apic/>
Aug 09 02:21:12 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:21:12 user nova-compute[72351]:   </features>
Aug 09 02:21:12 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:21:12 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:21:12 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:21:12 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:21:12 user nova-compute[72351]:   </clock>
Aug 09 02:21:12 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:21:12 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:21:12 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:21:12 user nova-compute[72351]:   </cpu>
Aug 09 02:21:12 user nova-compute[72351]:   <devices>
Aug 09 02:21:12 user nova-compute[72351]:     <disk type="file" device="disk">
Aug 09 02:21:12 user nova-compute[72351]:       <driver name="qemu" type="qcow2" cache="none"/>
Aug 09 02:21:12 user nova-compute[72351]:       <source file="/opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff/disk"/>
Aug 09 02:21:12 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:21:12 user nova-compute[72351]:     </disk>
Aug 09 02:21:12 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:21:12 user nova-compute[72351]:       <mac address="fa:16:3e:8b:b5:2e"/>
Aug 09 02:21:12 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:21:12 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:21:12 user nova-compute[72351]:       <target dev="tapbe4ec483-85"/>
Aug 09 02:21:12 user nova-compute[72351]:     </interface>
Aug 09 02:21:12 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:21:12 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff/console.log" append="off"/>
Aug 09 02:21:12 user nova-compute[72351]:     </serial>
Aug 09 02:21:12 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:21:12 user nova-compute[72351]:     <video>
Aug 09 02:21:12 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:21:12 user nova-compute[72351]:     </video>
Aug 09 02:21:12 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:21:12 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:21:12 user nova-compute[72351]:     </rng>
Aug 09 02:21:12 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:21:12 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:21:12 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:21:12 user nova-compute[72351]:     </memballoon>
Aug 09 02:21:12 user nova-compute[72351]:   </devices>
Aug 09 02:21:12 user nova-compute[72351]: </domain>
Aug 09 02:21:12 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:21:12 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:21:09Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerActionsTestOtherB-server-634605118',display_name='tempest-ServerActionsTestOtherB-server-634605118',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serveractionstestotherb-server-634605118',id=15,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLdtpWQKmHav2tGxZemUVMEaT9TXyD7Duhy5QV84PPI+5KrgLVkbxVzWgryocScOebEprbcJ80bjBPnYhLtQJYSe5MXXz6GOQHLb+tJCZajhrDoMbn41/bPAnB5B3oFwCw==',key_name='tempest-keypair-644333596',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='fc90e94bbcb54fb6b86a9ac45edb596a',ramdisk_id='',reservation_id='r-ozz8gdfj',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerActionsTestOtherB-1726991791',owner_user_name='tempest-ServerActionsTestOtherB-1726991791-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:21:11Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='19668b0c790d49dd830c39e3c2be29bf',uuid=75bbbead-8628-448b-9484-19bf4ab922ff,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "address": "fa:16:3e:8b:b5:2e", "network": {"id": "8231f2c2-237e-41c6-8892-720b950c1453", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherB-1917380761-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fc90e94bbcb54fb6b86a9ac45edb596a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbe4ec483-85", "ovs_interfaceid": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:21:12 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Converting VIF {"id": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "address": "fa:16:3e:8b:b5:2e", "network": {"id": "8231f2c2-237e-41c6-8892-720b950c1453", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherB-1917380761-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fc90e94bbcb54fb6b86a9ac45edb596a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbe4ec483-85", "ovs_interfaceid": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:21:12 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:8b:b5:2e,bridge_name='br-int',has_traffic_filtering=True,id=be4ec483-8532-45f7-bb9f-0b50952ce94b,network=Network(8231f2c2-237e-41c6-8892-720b950c1453),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapbe4ec483-85') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:21:12 user nova-compute[72351]: DEBUG os_vif [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:8b:b5:2e,bridge_name='br-int',has_traffic_filtering=True,id=be4ec483-8532-45f7-bb9f-0b50952ce94b,network=Network(8231f2c2-237e-41c6-8892-720b950c1453),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapbe4ec483-85') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:21:12 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:12 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:21:12 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:21:12 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:12 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapbe4ec483-85, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:21:12 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapbe4ec483-85, col_values=(('external_ids', {'iface-id': 'be4ec483-8532-45f7-bb9f-0b50952ce94b', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:8b:b5:2e', 'vm-uuid': '75bbbead-8628-448b-9484-19bf4ab922ff'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:21:12 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:12 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:21:12 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:12 user nova-compute[72351]: INFO os_vif [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:8b:b5:2e,bridge_name='br-int',has_traffic_filtering=True,id=be4ec483-8532-45f7-bb9f-0b50952ce94b,network=Network(8231f2c2-237e-41c6-8892-720b950c1453),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapbe4ec483-85')
Aug 09 02:21:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:21:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] No VIF found with MAC fa:16:3e:8b:b5:2e, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:21:13 user nova-compute[72351]: DEBUG nova.network.neutron [req-117b4ea2-0068-4c8f-9e78-6f2b1e9892e9 req-0d541e2e-754c-4480-88f5-04496811cca8 service nova] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Updated VIF entry in instance network info cache for port be4ec483-8532-45f7-bb9f-0b50952ce94b. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:21:13 user nova-compute[72351]: DEBUG nova.network.neutron [req-117b4ea2-0068-4c8f-9e78-6f2b1e9892e9 req-0d541e2e-754c-4480-88f5-04496811cca8 service nova] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Updating instance_info_cache with network_info: [{"id": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "address": "fa:16:3e:8b:b5:2e", "network": {"id": "8231f2c2-237e-41c6-8892-720b950c1453", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherB-1917380761-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fc90e94bbcb54fb6b86a9ac45edb596a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbe4ec483-85", "ovs_interfaceid": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:21:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-117b4ea2-0068-4c8f-9e78-6f2b1e9892e9 req-0d541e2e-754c-4480-88f5-04496811cca8 service nova] Releasing lock "refresh_cache-75bbbead-8628-448b-9484-19bf4ab922ff" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:21:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-15094d96-81e0-4e94-b4e4-cbdd3cb16ddb tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Acquiring lock "822e25c4-8dd1-4e21-b0ed-f340d59c6c98" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:21:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-15094d96-81e0-4e94-b4e4-cbdd3cb16ddb tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Lock "822e25c4-8dd1-4e21-b0ed-f340d59c6c98" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:21:14 user nova-compute[72351]: INFO nova.compute.manager [None req-15094d96-81e0-4e94-b4e4-cbdd3cb16ddb tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Detaching volume a4f329c4-9137-4cd0-971e-7a7402ba1785
Aug 09 02:21:14 user nova-compute[72351]: INFO nova.virt.block_device [None req-15094d96-81e0-4e94-b4e4-cbdd3cb16ddb tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Attempting to driver detach volume a4f329c4-9137-4cd0-971e-7a7402ba1785 from mountpoint /dev/vdb
Aug 09 02:21:14 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-15094d96-81e0-4e94-b4e4-cbdd3cb16ddb tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Attempting to detach device vdb from instance 822e25c4-8dd1-4e21-b0ed-f340d59c6c98 from the persistent domain config. {{(pid=72351) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2477}}
Aug 09 02:21:14 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-15094d96-81e0-4e94-b4e4-cbdd3cb16ddb tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 02:21:14 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:21:14 user nova-compute[72351]:   <source dev="/dev/sdd"/>
Aug 09 02:21:14 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:21:14 user nova-compute[72351]:   <serial>a4f329c4-9137-4cd0-971e-7a7402ba1785</serial>
Aug 09 02:21:14 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Aug 09 02:21:14 user nova-compute[72351]: </disk>
Aug 09 02:21:14 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 02:21:14 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-15094d96-81e0-4e94-b4e4-cbdd3cb16ddb tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Successfully detached device vdb from instance 822e25c4-8dd1-4e21-b0ed-f340d59c6c98 from the persistent domain config.
Aug 09 02:21:14 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-15094d96-81e0-4e94-b4e4-cbdd3cb16ddb tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 822e25c4-8dd1-4e21-b0ed-f340d59c6c98 from the live domain config. {{(pid=72351) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2513}}
Aug 09 02:21:14 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-15094d96-81e0-4e94-b4e4-cbdd3cb16ddb tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 02:21:14 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:21:14 user nova-compute[72351]:   <source dev="/dev/sdd"/>
Aug 09 02:21:14 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:21:14 user nova-compute[72351]:   <serial>a4f329c4-9137-4cd0-971e-7a7402ba1785</serial>
Aug 09 02:21:14 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Aug 09 02:21:14 user nova-compute[72351]: </disk>
Aug 09 02:21:14 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 02:21:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:14 user nova-compute[72351]: DEBUG nova.compute.manager [req-d988b54b-8b44-4a10-8974-d304ab264e89 req-62644cbf-3d15-4dc8-b3c5-f1a321530369 service nova] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Received event network-vif-plugged-be4ec483-8532-45f7-bb9f-0b50952ce94b {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:21:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d988b54b-8b44-4a10-8974-d304ab264e89 req-62644cbf-3d15-4dc8-b3c5-f1a321530369 service nova] Acquiring lock "75bbbead-8628-448b-9484-19bf4ab922ff-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:21:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d988b54b-8b44-4a10-8974-d304ab264e89 req-62644cbf-3d15-4dc8-b3c5-f1a321530369 service nova] Lock "75bbbead-8628-448b-9484-19bf4ab922ff-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:21:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d988b54b-8b44-4a10-8974-d304ab264e89 req-62644cbf-3d15-4dc8-b3c5-f1a321530369 service nova] Lock "75bbbead-8628-448b-9484-19bf4ab922ff-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:21:14 user nova-compute[72351]: DEBUG nova.compute.manager [req-d988b54b-8b44-4a10-8974-d304ab264e89 req-62644cbf-3d15-4dc8-b3c5-f1a321530369 service nova] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] No waiting events found dispatching network-vif-plugged-be4ec483-8532-45f7-bb9f-0b50952ce94b {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:21:14 user nova-compute[72351]: WARNING nova.compute.manager [req-d988b54b-8b44-4a10-8974-d304ab264e89 req-62644cbf-3d15-4dc8-b3c5-f1a321530369 service nova] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Received unexpected event network-vif-plugged-be4ec483-8532-45f7-bb9f-0b50952ce94b for instance with vm_state building and task_state spawning.
Aug 09 02:21:15 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:15 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Received event <DeviceRemovedEvent: 1691547675.192262, 822e25c4-8dd1-4e21-b0ed-f340d59c6c98 => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=72351) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2360}}
Aug 09 02:21:15 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:15 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-15094d96-81e0-4e94-b4e4-cbdd3cb16ddb tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 822e25c4-8dd1-4e21-b0ed-f340d59c6c98 {{(pid=72351) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2589}}
Aug 09 02:21:15 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-15094d96-81e0-4e94-b4e4-cbdd3cb16ddb tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Successfully detached device vdb from instance 822e25c4-8dd1-4e21-b0ed-f340d59c6c98 from the live domain config.
Aug 09 02:21:15 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-15094d96-81e0-4e94-b4e4-cbdd3cb16ddb tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 02:21:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-15094d96-81e0-4e94-b4e4-cbdd3cb16ddb tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:a4f329c4-9137-4cd0-971e-7a7402ba1785'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'FfmdKzlS', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdd'}, None), 'kwargs': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:21:15 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-15094d96-81e0-4e94-b4e4-cbdd3cb16ddb tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:21:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-15094d96-81e0-4e94-b4e4-cbdd3cb16ddb tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:21:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-15094d96-81e0-4e94-b4e4-cbdd3cb16ddb tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:21:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-15094d96-81e0-4e94-b4e4-cbdd3cb16ddb tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:a4f329c4-9137-4cd0-971e-7a7402ba1785', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 02:21:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:15 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:15 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[9afbfe56-f06a-4298-b249-22f159bd7c42]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:a4f329c4-9137-4cd0-971e-7a7402ba1785\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:21:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:15 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d55ffd74-7fb4-4219-881d-74a5d97f6e28]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f (non-flash)\ntcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497 (non-flash)\ntcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a4f329c4-9137-4cd0-971e-7a7402ba1785 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:21:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-15094d96-81e0-4e94-b4e4-cbdd3cb16ddb tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f (non-flash)
Aug 09 02:21:15 user nova-compute[72351]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497 (non-flash)
Aug 09 02:21:15 user nova-compute[72351]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a4f329c4-9137-4cd0-971e-7a7402ba1785 (non-flash)
Aug 09 02:21:15 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:21:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-15094d96-81e0-4e94-b4e4-cbdd3cb16ddb tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f (non-flash)
Aug 09 02:21:15 user nova-compute[72351]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497 (non-flash)
Aug 09 02:21:15 user nova-compute[72351]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a4f329c4-9137-4cd0-971e-7a7402ba1785 (non-flash)
Aug 09 02:21:15 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:21:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-15094d96-81e0-4e94-b4e4-cbdd3cb16ddb tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fcffc276200>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:a4f329c4-9137-4cd0-971e-7a7402ba1785'): ({'sdd'}, set())}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 02:21:15 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-15094d96-81e0-4e94-b4e4-cbdd3cb16ddb tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Removing single pathed devices sdd {{(pid=72351) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Aug 09 02:21:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:15 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[459215fb-f6b6-41c7-ab3f-ac425f8a93c3]: (4, ('path checker states:\nup                  3\n\npaths: 0\nbusy: False\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:21:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdd {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdd" returned: 0 in 0.026s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:15 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[cd8f226d-0760-4e48-9fcd-f7d1d28246fb]: (4, ('ok\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:21:15 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-15094d96-81e0-4e94-b4e4-cbdd3cb16ddb tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Flushing IO for device /dev/sdd {{(pid=72351) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Aug 09 02:21:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdd {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdd" returned: 0 in 0.014s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:15 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[edf2082e-aece-44ce-ae82-ac984c553732]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:21:15 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-15094d96-81e0-4e94-b4e4-cbdd3cb16ddb tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Remove SCSI device /dev/sdd with /sys/block/sdd/device/delete {{(pid=72351) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Aug 09 02:21:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdd/device/delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdd/device/delete" returned: 0 in 0.068s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:15 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[7db328e3-a6cc-4ff8-8899-a4a2f8cdc7c1]: (4, ('1', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:21:15 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-15094d96-81e0-4e94-b4e4-cbdd3cb16ddb tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Checking to see if SCSI volumes sdd have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Aug 09 02:21:15 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-15094d96-81e0-4e94-b4e4-cbdd3cb16ddb tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] SCSI volumes sdd have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Aug 09 02:21:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-15094d96-81e0-4e94-b4e4-cbdd3cb16ddb tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:a4f329c4-9137-4cd0-971e-7a7402ba1785')] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 02:21:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a4f329c4-9137-4cd0-971e-7a7402ba1785 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a4f329c4-9137-4cd0-971e-7a7402ba1785 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:15 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[6bd3f3a5-538f-45fe-93b8-1f4de5b1fee4]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:21:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-15094d96-81e0-4e94-b4e4-cbdd3cb16ddb tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:21:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a4f329c4-9137-4cd0-971e-7a7402ba1785 -p 172.16.0.220:3260 --logout {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a4f329c4-9137-4cd0-971e-7a7402ba1785 -p 172.16.0.220:3260 --logout" returned: 0 in 0.075s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:15 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e84255e8-f65b-4786-a165-bd580bc5d9eb]: (4, ('Logging out of session [sid: 16, target: iqn.2010-10.org.openstack:a4f329c4-9137-4cd0-971e-7a7402ba1785, portal: 172.16.0.220,3260]\nLogout of [sid: 16, target: iqn.2010-10.org.openstack:a4f329c4-9137-4cd0-971e-7a7402ba1785, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:21:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-15094d96-81e0-4e94-b4e4-cbdd3cb16ddb tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 16, target: iqn.2010-10.org.openstack:a4f329c4-9137-4cd0-971e-7a7402ba1785, portal: 172.16.0.220,3260]
Aug 09 02:21:15 user nova-compute[72351]: Logout of [sid: 16, target: iqn.2010-10.org.openstack:a4f329c4-9137-4cd0-971e-7a7402ba1785, portal: 172.16.0.220,3260] successful.
Aug 09 02:21:15 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:21:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a4f329c4-9137-4cd0-971e-7a7402ba1785 -p 172.16.0.220:3260 --op delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a4f329c4-9137-4cd0-971e-7a7402ba1785 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.031s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:15 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[5cd0ceb8-23c6-41cb-8aec-853aa43ae1aa]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:21:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-15094d96-81e0-4e94-b4e4-cbdd3cb16ddb tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:21:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-15094d96-81e0-4e94-b4e4-cbdd3cb16ddb tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.345s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:21:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-15094d96-81e0-4e94-b4e4-cbdd3cb16ddb tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] <== disconnect_volume: return (346ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:21:15 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-15094d96-81e0-4e94-b4e4-cbdd3cb16ddb tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 02:21:16 user nova-compute[72351]: DEBUG nova.objects.instance [None req-15094d96-81e0-4e94-b4e4-cbdd3cb16ddb tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Lazy-loading 'flavor' on Instance uuid 822e25c4-8dd1-4e21-b0ed-f340d59c6c98 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:21:16 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-15094d96-81e0-4e94-b4e4-cbdd3cb16ddb tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Lock "822e25c4-8dd1-4e21-b0ed-f340d59c6c98" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 2.127s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:21:16 user nova-compute[72351]: DEBUG nova.compute.manager [req-2e655271-ddfe-45c2-9be7-33ffb3e00b80 req-4c4310ee-dc13-4245-bb36-4182ca836b44 service nova] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Received event network-vif-plugged-be4ec483-8532-45f7-bb9f-0b50952ce94b {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:21:16 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-2e655271-ddfe-45c2-9be7-33ffb3e00b80 req-4c4310ee-dc13-4245-bb36-4182ca836b44 service nova] Acquiring lock "75bbbead-8628-448b-9484-19bf4ab922ff-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:21:16 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-2e655271-ddfe-45c2-9be7-33ffb3e00b80 req-4c4310ee-dc13-4245-bb36-4182ca836b44 service nova] Lock "75bbbead-8628-448b-9484-19bf4ab922ff-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:21:16 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-2e655271-ddfe-45c2-9be7-33ffb3e00b80 req-4c4310ee-dc13-4245-bb36-4182ca836b44 service nova] Lock "75bbbead-8628-448b-9484-19bf4ab922ff-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:21:16 user nova-compute[72351]: DEBUG nova.compute.manager [req-2e655271-ddfe-45c2-9be7-33ffb3e00b80 req-4c4310ee-dc13-4245-bb36-4182ca836b44 service nova] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] No waiting events found dispatching network-vif-plugged-be4ec483-8532-45f7-bb9f-0b50952ce94b {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:21:16 user nova-compute[72351]: WARNING nova.compute.manager [req-2e655271-ddfe-45c2-9be7-33ffb3e00b80 req-4c4310ee-dc13-4245-bb36-4182ca836b44 service nova] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Received unexpected event network-vif-plugged-be4ec483-8532-45f7-bb9f-0b50952ce94b for instance with vm_state building and task_state spawning.
Aug 09 02:21:17 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547677.223453, 75bbbead-8628-448b-9484-19bf4ab922ff => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:21:17 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] VM Resumed (Lifecycle Event)
Aug 09 02:21:17 user nova-compute[72351]: DEBUG nova.compute.manager [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:21:17 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Guest created on hypervisor {{(pid=72351) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4411}}
Aug 09 02:21:17 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Instance spawned successfully.
Aug 09 02:21:17 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:894}}
Aug 09 02:21:17 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:21:17 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:21:17 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Found default for hw_cdrom_bus of ide {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:21:17 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Found default for hw_disk_bus of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:21:17 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Found default for hw_input_bus of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:21:17 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Found default for hw_pointer_model of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:21:17 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Found default for hw_video_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:21:17 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Found default for hw_vif_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:21:17 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:21:17 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547677.2271132, 75bbbead-8628-448b-9484-19bf4ab922ff => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:21:17 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] VM Started (Lifecycle Event)
Aug 09 02:21:17 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:21:17 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:21:17 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:21:17 user nova-compute[72351]: INFO nova.compute.manager [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Took 6.54 seconds to spawn the instance on the hypervisor.
Aug 09 02:21:17 user nova-compute[72351]: DEBUG nova.compute.manager [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:21:17 user nova-compute[72351]: INFO nova.compute.manager [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Took 7.36 seconds to build instance.
Aug 09 02:21:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-49f9adac-5186-4159-805e-d035810de7a8 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Lock "75bbbead-8628-448b-9484-19bf4ab922ff" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 7.470s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:21:17 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:17 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:22 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691547667.1237638, 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:21:22 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] VM Stopped (Lifecycle Event)
Aug 09 02:21:22 user nova-compute[72351]: DEBUG nova.compute.manager [None req-6d812257-4603-44c1-ae32-f4c329c28361 None None] [instance: 11ff6c2c-9836-4ec2-9f28-76f1f54dcf8b] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:21:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:31 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:32 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:32 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:34 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Acquiring lock "e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:21:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:21:35 user nova-compute[72351]: DEBUG nova.compute.manager [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Starting instance... {{(pid=72351) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2406}}
Aug 09 02:21:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:21:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:21:35 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72351) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Aug 09 02:21:35 user nova-compute[72351]: INFO nova.compute.claims [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Claim successful on node user
Aug 09 02:21:35 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:21:36 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:21:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.451s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:21:36 user nova-compute[72351]: DEBUG nova.compute.manager [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Start building networks asynchronously for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2803}}
Aug 09 02:21:36 user nova-compute[72351]: DEBUG nova.compute.manager [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Allocating IP information in the background. {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1956}}
Aug 09 02:21:36 user nova-compute[72351]: DEBUG nova.network.neutron [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] allocate_for_instance() {{(pid=72351) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Aug 09 02:21:36 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Aug 09 02:21:36 user nova-compute[72351]: DEBUG nova.compute.manager [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Start building block device mappings for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2838}}
Aug 09 02:21:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Acquiring lock "eb86cb49-2018-42c2-bd97-c662b829775d" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:21:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "eb86cb49-2018-42c2-bd97-c662b829775d" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:21:36 user nova-compute[72351]: DEBUG nova.compute.manager [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Starting instance... {{(pid=72351) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2406}}
Aug 09 02:21:36 user nova-compute[72351]: DEBUG nova.policy [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '419b28d5a46c481db435715a30ab2614', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3aea1e66805146f7a8cd1d97e64a24af', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72351) authorize /opt/stack/nova/nova/policy.py:203}}
Aug 09 02:21:36 user nova-compute[72351]: DEBUG nova.compute.manager [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Start spawning the instance on the hypervisor. {{(pid=72351) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2612}}
Aug 09 02:21:36 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Creating instance directory {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4717}}
Aug 09 02:21:36 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Creating image(s)
Aug 09 02:21:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Acquiring lock "/opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:21:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "/opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:21:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "/opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:21:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:21:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:21:36 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72351) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Aug 09 02:21:36 user nova-compute[72351]: INFO nova.compute.claims [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Claim successful on node user
Aug 09 02:21:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.145s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Acquiring lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:21:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:21:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.152s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk 1073741824 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk 1073741824" returned: 0 in 0.053s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.211s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:21:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.141s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:36 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Checking if we can resize image /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk. size=1073741824 {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Aug 09 02:21:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:36 user nova-compute[72351]: DEBUG nova.network.neutron [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Successfully created port: 9481e457-9c23-4c45-b0bd-1faf63f8b7b8 {{(pid=72351) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Aug 09 02:21:36 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:21:36 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:21:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json" returned: 0 in 0.184s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:36 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Cannot resize image /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk to a smaller size. {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Aug 09 02:21:36 user nova-compute[72351]: DEBUG nova.objects.instance [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lazy-loading 'migration_context' on Instance uuid e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:21:37 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Created local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4851}}
Aug 09 02:21:37 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Ensure instance console log exists: /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/console.log {{(pid=72351) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4603}}
Aug 09 02:21:37 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:21:37 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:21:37 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:21:37 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.657s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:21:37 user nova-compute[72351]: DEBUG nova.compute.manager [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Start building networks asynchronously for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2803}}
Aug 09 02:21:37 user nova-compute[72351]: DEBUG nova.compute.manager [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Allocating IP information in the background. {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1956}}
Aug 09 02:21:37 user nova-compute[72351]: DEBUG nova.network.neutron [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] allocate_for_instance() {{(pid=72351) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Aug 09 02:21:37 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Aug 09 02:21:37 user nova-compute[72351]: DEBUG nova.compute.manager [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Start building block device mappings for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2838}}
Aug 09 02:21:37 user nova-compute[72351]: DEBUG nova.policy [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '419b28d5a46c481db435715a30ab2614', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3aea1e66805146f7a8cd1d97e64a24af', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72351) authorize /opt/stack/nova/nova/policy.py:203}}
Aug 09 02:21:37 user nova-compute[72351]: DEBUG nova.compute.manager [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Start spawning the instance on the hypervisor. {{(pid=72351) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2612}}
Aug 09 02:21:37 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Creating instance directory {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4717}}
Aug 09 02:21:37 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Creating image(s)
Aug 09 02:21:37 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Acquiring lock "/opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:21:37 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "/opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:21:37 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "/opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:21:37 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:37 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:37 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.215s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:37 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Acquiring lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:21:37 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:21:37 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:37 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:37 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.199s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:37 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk 1073741824 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:37 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk 1073741824" returned: 0 in 0.079s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:37 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.285s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:21:37 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:37 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:38 user nova-compute[72351]: DEBUG nova.network.neutron [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Successfully updated port: 9481e457-9c23-4c45-b0bd-1faf63f8b7b8 {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:21:38 user nova-compute[72351]: DEBUG nova.compute.manager [req-2bcaff5b-40db-4810-9a2c-c1ffbf03c390 req-60d5ddb6-ef26-4f9e-bb56-9b3c52d8d040 service nova] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Received event network-changed-9481e457-9c23-4c45-b0bd-1faf63f8b7b8 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:21:38 user nova-compute[72351]: DEBUG nova.compute.manager [req-2bcaff5b-40db-4810-9a2c-c1ffbf03c390 req-60d5ddb6-ef26-4f9e-bb56-9b3c52d8d040 service nova] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Refreshing instance network info cache due to event network-changed-9481e457-9c23-4c45-b0bd-1faf63f8b7b8. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:21:38 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-2bcaff5b-40db-4810-9a2c-c1ffbf03c390 req-60d5ddb6-ef26-4f9e-bb56-9b3c52d8d040 service nova] Acquiring lock "refresh_cache-e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:21:38 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-2bcaff5b-40db-4810-9a2c-c1ffbf03c390 req-60d5ddb6-ef26-4f9e-bb56-9b3c52d8d040 service nova] Acquired lock "refresh_cache-e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:21:38 user nova-compute[72351]: DEBUG nova.network.neutron [req-2bcaff5b-40db-4810-9a2c-c1ffbf03c390 req-60d5ddb6-ef26-4f9e-bb56-9b3c52d8d040 service nova] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Refreshing network info cache for port 9481e457-9c23-4c45-b0bd-1faf63f8b7b8 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:21:38 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Acquiring lock "refresh_cache-e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:21:38 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.236s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:38 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Checking if we can resize image /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk. size=1073741824 {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Aug 09 02:21:38 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:38 user nova-compute[72351]: DEBUG nova.network.neutron [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Successfully created port: 5f97db70-9e5b-4d42-b765-b832fb522f41 {{(pid=72351) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Aug 09 02:21:38 user nova-compute[72351]: DEBUG nova.network.neutron [req-2bcaff5b-40db-4810-9a2c-c1ffbf03c390 req-60d5ddb6-ef26-4f9e-bb56-9b3c52d8d040 service nova] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:21:38 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json" returned: 0 in 0.157s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:38 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Cannot resize image /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk to a smaller size. {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Aug 09 02:21:38 user nova-compute[72351]: DEBUG nova.objects.instance [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lazy-loading 'migration_context' on Instance uuid eb86cb49-2018-42c2-bd97-c662b829775d {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:21:38 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Created local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4851}}
Aug 09 02:21:38 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Ensure instance console log exists: /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/console.log {{(pid=72351) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4603}}
Aug 09 02:21:38 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:21:38 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:21:38 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:21:38 user nova-compute[72351]: DEBUG nova.network.neutron [req-2bcaff5b-40db-4810-9a2c-c1ffbf03c390 req-60d5ddb6-ef26-4f9e-bb56-9b3c52d8d040 service nova] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:21:38 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-2bcaff5b-40db-4810-9a2c-c1ffbf03c390 req-60d5ddb6-ef26-4f9e-bb56-9b3c52d8d040 service nova] Releasing lock "refresh_cache-e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:21:38 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Acquired lock "refresh_cache-e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:21:38 user nova-compute[72351]: DEBUG nova.network.neutron [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:21:38 user nova-compute[72351]: DEBUG nova.network.neutron [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:21:38 user nova-compute[72351]: DEBUG nova.network.neutron [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Updating instance_info_cache with network_info: [{"id": "9481e457-9c23-4c45-b0bd-1faf63f8b7b8", "address": "fa:16:3e:69:03:30", "network": {"id": "49e8751c-e119-430f-9110-3dc6069b1589", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-899275135-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3aea1e66805146f7a8cd1d97e64a24af", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9481e457-9c", "ovs_interfaceid": "9481e457-9c23-4c45-b0bd-1faf63f8b7b8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:21:38 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Releasing lock "refresh_cache-e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:21:38 user nova-compute[72351]: DEBUG nova.compute.manager [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Instance network_info: |[{"id": "9481e457-9c23-4c45-b0bd-1faf63f8b7b8", "address": "fa:16:3e:69:03:30", "network": {"id": "49e8751c-e119-430f-9110-3dc6069b1589", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-899275135-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3aea1e66805146f7a8cd1d97e64a24af", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9481e457-9c", "ovs_interfaceid": "9481e457-9c23-4c45-b0bd-1faf63f8b7b8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1971}}
Aug 09 02:21:38 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Start _get_guest_xml network_info=[{"id": "9481e457-9c23-4c45-b0bd-1faf63f8b7b8", "address": "fa:16:3e:69:03:30", "network": {"id": "49e8751c-e119-430f-9110-3dc6069b1589", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-899275135-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3aea1e66805146f7a8cd1d97e64a24af", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9481e457-9c", "ovs_interfaceid": "9481e457-9c23-4c45-b0bd-1faf63f8b7b8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'size': 0, 'device_type': 'disk', 'encryption_secret_uuid': None, 'guest_format': None, 'encryption_options': None, 'disk_bus': 'virtio', 'encrypted': False, 'encryption_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'image_id': '18ccda7f-31d3-4978-ab9d-4ed2f74eff94'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:21:38 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:21:38 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:21:38 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:21:38 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:21:38 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:21:38 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:21:38 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:21:38 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:21:38 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:21:38 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:21:38 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:21:38 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:21:38 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:21:38 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:21:38 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:21:38 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:21:38 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:21:38 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:21:38 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:21:34Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerRescueNegativeTestJSON-server-938229174',display_name='tempest-ServerRescueNegativeTestJSON-server-938229174',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serverrescuenegativetestjson-server-938229174',id=16,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='3aea1e66805146f7a8cd1d97e64a24af',ramdisk_id='',reservation_id='r-14yazlt3',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerRescueNegativeTestJSON-289364438',owner_user_name='tempest-ServerRescueNegativeTestJSON-289364438-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:21:36Z,user_data=None,user_id='419b28d5a46c481db435715a30ab2614',uuid=e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "9481e457-9c23-4c45-b0bd-1faf63f8b7b8", "address": "fa:16:3e:69:03:30", "network": {"id": "49e8751c-e119-430f-9110-3dc6069b1589", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-899275135-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3aea1e66805146f7a8cd1d97e64a24af", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9481e457-9c", "ovs_interfaceid": "9481e457-9c23-4c45-b0bd-1faf63f8b7b8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:21:38 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Converting VIF {"id": "9481e457-9c23-4c45-b0bd-1faf63f8b7b8", "address": "fa:16:3e:69:03:30", "network": {"id": "49e8751c-e119-430f-9110-3dc6069b1589", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-899275135-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3aea1e66805146f7a8cd1d97e64a24af", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9481e457-9c", "ovs_interfaceid": "9481e457-9c23-4c45-b0bd-1faf63f8b7b8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:21:38 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:69:03:30,bridge_name='br-int',has_traffic_filtering=True,id=9481e457-9c23-4c45-b0bd-1faf63f8b7b8,network=Network(49e8751c-e119-430f-9110-3dc6069b1589),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap9481e457-9c') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG nova.objects.instance [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lazy-loading 'pci_devices' on Instance uuid e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:21:39 user nova-compute[72351]:   <uuid>e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335</uuid>
Aug 09 02:21:39 user nova-compute[72351]:   <name>instance-00000010</name>
Aug 09 02:21:39 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:21:39 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:21:39 user nova-compute[72351]:   <metadata>
Aug 09 02:21:39 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:21:39 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:21:39 user nova-compute[72351]:       <nova:name>tempest-ServerRescueNegativeTestJSON-server-938229174</nova:name>
Aug 09 02:21:39 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:21:38</nova:creationTime>
Aug 09 02:21:39 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:21:39 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:21:39 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:21:39 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:21:39 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:21:39 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:21:39 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:21:39 user nova-compute[72351]:       <nova:owner>
Aug 09 02:21:39 user nova-compute[72351]:         <nova:user uuid="419b28d5a46c481db435715a30ab2614">tempest-ServerRescueNegativeTestJSON-289364438-project-member</nova:user>
Aug 09 02:21:39 user nova-compute[72351]:         <nova:project uuid="3aea1e66805146f7a8cd1d97e64a24af">tempest-ServerRescueNegativeTestJSON-289364438</nova:project>
Aug 09 02:21:39 user nova-compute[72351]:       </nova:owner>
Aug 09 02:21:39 user nova-compute[72351]:       <nova:root type="image" uuid="18ccda7f-31d3-4978-ab9d-4ed2f74eff94"/>
Aug 09 02:21:39 user nova-compute[72351]:       <nova:ports>
Aug 09 02:21:39 user nova-compute[72351]:         <nova:port uuid="9481e457-9c23-4c45-b0bd-1faf63f8b7b8">
Aug 09 02:21:39 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.12" ipVersion="4"/>
Aug 09 02:21:39 user nova-compute[72351]:         </nova:port>
Aug 09 02:21:39 user nova-compute[72351]:       </nova:ports>
Aug 09 02:21:39 user nova-compute[72351]:     </nova:instance>
Aug 09 02:21:39 user nova-compute[72351]:   </metadata>
Aug 09 02:21:39 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:21:39 user nova-compute[72351]:     <system>
Aug 09 02:21:39 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:21:39 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:21:39 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:21:39 user nova-compute[72351]:       <entry name="serial">e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335</entry>
Aug 09 02:21:39 user nova-compute[72351]:       <entry name="uuid">e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335</entry>
Aug 09 02:21:39 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:21:39 user nova-compute[72351]:     </system>
Aug 09 02:21:39 user nova-compute[72351]:   </sysinfo>
Aug 09 02:21:39 user nova-compute[72351]:   <os>
Aug 09 02:21:39 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:21:39 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:21:39 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:21:39 user nova-compute[72351]:   </os>
Aug 09 02:21:39 user nova-compute[72351]:   <features>
Aug 09 02:21:39 user nova-compute[72351]:     <acpi/>
Aug 09 02:21:39 user nova-compute[72351]:     <apic/>
Aug 09 02:21:39 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:21:39 user nova-compute[72351]:   </features>
Aug 09 02:21:39 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:21:39 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:21:39 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:21:39 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:21:39 user nova-compute[72351]:   </clock>
Aug 09 02:21:39 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:21:39 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:21:39 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:21:39 user nova-compute[72351]:   </cpu>
Aug 09 02:21:39 user nova-compute[72351]:   <devices>
Aug 09 02:21:39 user nova-compute[72351]:     <disk type="file" device="disk">
Aug 09 02:21:39 user nova-compute[72351]:       <driver name="qemu" type="qcow2" cache="none"/>
Aug 09 02:21:39 user nova-compute[72351]:       <source file="/opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk"/>
Aug 09 02:21:39 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:21:39 user nova-compute[72351]:     </disk>
Aug 09 02:21:39 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:21:39 user nova-compute[72351]:       <mac address="fa:16:3e:69:03:30"/>
Aug 09 02:21:39 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:21:39 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:21:39 user nova-compute[72351]:       <target dev="tap9481e457-9c"/>
Aug 09 02:21:39 user nova-compute[72351]:     </interface>
Aug 09 02:21:39 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:21:39 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/console.log" append="off"/>
Aug 09 02:21:39 user nova-compute[72351]:     </serial>
Aug 09 02:21:39 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:21:39 user nova-compute[72351]:     <video>
Aug 09 02:21:39 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:21:39 user nova-compute[72351]:     </video>
Aug 09 02:21:39 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:21:39 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:21:39 user nova-compute[72351]:     </rng>
Aug 09 02:21:39 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:21:39 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:21:39 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:21:39 user nova-compute[72351]:     </memballoon>
Aug 09 02:21:39 user nova-compute[72351]:   </devices>
Aug 09 02:21:39 user nova-compute[72351]: </domain>
Aug 09 02:21:39 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:21:34Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerRescueNegativeTestJSON-server-938229174',display_name='tempest-ServerRescueNegativeTestJSON-server-938229174',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serverrescuenegativetestjson-server-938229174',id=16,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='3aea1e66805146f7a8cd1d97e64a24af',ramdisk_id='',reservation_id='r-14yazlt3',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerRescueNegativeTestJSON-289364438',owner_user_name='tempest-ServerRescueNegativeTestJSON-289364438-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:21:36Z,user_data=None,user_id='419b28d5a46c481db435715a30ab2614',uuid=e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "9481e457-9c23-4c45-b0bd-1faf63f8b7b8", "address": "fa:16:3e:69:03:30", "network": {"id": "49e8751c-e119-430f-9110-3dc6069b1589", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-899275135-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3aea1e66805146f7a8cd1d97e64a24af", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9481e457-9c", "ovs_interfaceid": "9481e457-9c23-4c45-b0bd-1faf63f8b7b8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Converting VIF {"id": "9481e457-9c23-4c45-b0bd-1faf63f8b7b8", "address": "fa:16:3e:69:03:30", "network": {"id": "49e8751c-e119-430f-9110-3dc6069b1589", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-899275135-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3aea1e66805146f7a8cd1d97e64a24af", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9481e457-9c", "ovs_interfaceid": "9481e457-9c23-4c45-b0bd-1faf63f8b7b8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:69:03:30,bridge_name='br-int',has_traffic_filtering=True,id=9481e457-9c23-4c45-b0bd-1faf63f8b7b8,network=Network(49e8751c-e119-430f-9110-3dc6069b1589),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap9481e457-9c') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG os_vif [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:69:03:30,bridge_name='br-int',has_traffic_filtering=True,id=9481e457-9c23-4c45-b0bd-1faf63f8b7b8,network=Network(49e8751c-e119-430f-9110-3dc6069b1589),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap9481e457-9c') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap9481e457-9c, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap9481e457-9c, col_values=(('external_ids', {'iface-id': '9481e457-9c23-4c45-b0bd-1faf63f8b7b8', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:69:03:30', 'vm-uuid': 'e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:39 user nova-compute[72351]: INFO os_vif [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:69:03:30,bridge_name='br-int',has_traffic_filtering=True,id=9481e457-9c23-4c45-b0bd-1faf63f8b7b8,network=Network(49e8751c-e119-430f-9110-3dc6069b1589),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap9481e457-9c')
Aug 09 02:21:39 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] No VIF found with MAC fa:16:3e:69:03:30, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG nova.network.neutron [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Successfully updated port: 5f97db70-9e5b-4d42-b765-b832fb522f41 {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Acquiring lock "refresh_cache-eb86cb49-2018-42c2-bd97-c662b829775d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Acquired lock "refresh_cache-eb86cb49-2018-42c2-bd97-c662b829775d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG nova.network.neutron [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG nova.compute.manager [req-12a5a427-542a-4c55-9bec-13adee076ffa req-cb02d63d-ef17-45f9-b2ad-6366d99039d0 service nova] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Received event network-changed-5f97db70-9e5b-4d42-b765-b832fb522f41 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG nova.compute.manager [req-12a5a427-542a-4c55-9bec-13adee076ffa req-cb02d63d-ef17-45f9-b2ad-6366d99039d0 service nova] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Refreshing instance network info cache due to event network-changed-5f97db70-9e5b-4d42-b765-b832fb522f41. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-12a5a427-542a-4c55-9bec-13adee076ffa req-cb02d63d-ef17-45f9-b2ad-6366d99039d0 service nova] Acquiring lock "refresh_cache-eb86cb49-2018-42c2-bd97-c662b829775d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG nova.network.neutron [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG nova.network.neutron [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Updating instance_info_cache with network_info: [{"id": "5f97db70-9e5b-4d42-b765-b832fb522f41", "address": "fa:16:3e:dc:da:c7", "network": {"id": "49e8751c-e119-430f-9110-3dc6069b1589", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-899275135-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3aea1e66805146f7a8cd1d97e64a24af", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5f97db70-9e", "ovs_interfaceid": "5f97db70-9e5b-4d42-b765-b832fb522f41", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Releasing lock "refresh_cache-eb86cb49-2018-42c2-bd97-c662b829775d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG nova.compute.manager [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Instance network_info: |[{"id": "5f97db70-9e5b-4d42-b765-b832fb522f41", "address": "fa:16:3e:dc:da:c7", "network": {"id": "49e8751c-e119-430f-9110-3dc6069b1589", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-899275135-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3aea1e66805146f7a8cd1d97e64a24af", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5f97db70-9e", "ovs_interfaceid": "5f97db70-9e5b-4d42-b765-b832fb522f41", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1971}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-12a5a427-542a-4c55-9bec-13adee076ffa req-cb02d63d-ef17-45f9-b2ad-6366d99039d0 service nova] Acquired lock "refresh_cache-eb86cb49-2018-42c2-bd97-c662b829775d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG nova.network.neutron [req-12a5a427-542a-4c55-9bec-13adee076ffa req-cb02d63d-ef17-45f9-b2ad-6366d99039d0 service nova] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Refreshing network info cache for port 5f97db70-9e5b-4d42-b765-b832fb522f41 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Start _get_guest_xml network_info=[{"id": "5f97db70-9e5b-4d42-b765-b832fb522f41", "address": "fa:16:3e:dc:da:c7", "network": {"id": "49e8751c-e119-430f-9110-3dc6069b1589", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-899275135-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3aea1e66805146f7a8cd1d97e64a24af", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5f97db70-9e", "ovs_interfaceid": "5f97db70-9e5b-4d42-b765-b832fb522f41", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'size': 0, 'device_type': 'disk', 'encryption_secret_uuid': None, 'guest_format': None, 'encryption_options': None, 'disk_bus': 'virtio', 'encrypted': False, 'encryption_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'image_id': '18ccda7f-31d3-4978-ab9d-4ed2f74eff94'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:21:39 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:21:39 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:21:39 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:21:35Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerRescueNegativeTestJSON-server-69135895',display_name='tempest-ServerRescueNegativeTestJSON-server-69135895',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serverrescuenegativetestjson-server-69135895',id=17,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='3aea1e66805146f7a8cd1d97e64a24af',ramdisk_id='',reservation_id='r-b0vsogov',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerRescueNegativeTestJSON-289364438',owner_user_name='tempest-ServerRescueNegativeTestJSON-289364438-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:21:37Z,user_data=None,user_id='419b28d5a46c481db435715a30ab2614',uuid=eb86cb49-2018-42c2-bd97-c662b829775d,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "5f97db70-9e5b-4d42-b765-b832fb522f41", "address": "fa:16:3e:dc:da:c7", "network": {"id": "49e8751c-e119-430f-9110-3dc6069b1589", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-899275135-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3aea1e66805146f7a8cd1d97e64a24af", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5f97db70-9e", "ovs_interfaceid": "5f97db70-9e5b-4d42-b765-b832fb522f41", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Converting VIF {"id": "5f97db70-9e5b-4d42-b765-b832fb522f41", "address": "fa:16:3e:dc:da:c7", "network": {"id": "49e8751c-e119-430f-9110-3dc6069b1589", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-899275135-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3aea1e66805146f7a8cd1d97e64a24af", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5f97db70-9e", "ovs_interfaceid": "5f97db70-9e5b-4d42-b765-b832fb522f41", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:dc:da:c7,bridge_name='br-int',has_traffic_filtering=True,id=5f97db70-9e5b-4d42-b765-b832fb522f41,network=Network(49e8751c-e119-430f-9110-3dc6069b1589),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap5f97db70-9e') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG nova.objects.instance [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lazy-loading 'pci_devices' on Instance uuid eb86cb49-2018-42c2-bd97-c662b829775d {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:21:39 user nova-compute[72351]:   <uuid>eb86cb49-2018-42c2-bd97-c662b829775d</uuid>
Aug 09 02:21:39 user nova-compute[72351]:   <name>instance-00000011</name>
Aug 09 02:21:39 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:21:39 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:21:39 user nova-compute[72351]:   <metadata>
Aug 09 02:21:39 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:21:39 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:21:39 user nova-compute[72351]:       <nova:name>tempest-ServerRescueNegativeTestJSON-server-69135895</nova:name>
Aug 09 02:21:39 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:21:39</nova:creationTime>
Aug 09 02:21:39 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:21:39 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:21:39 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:21:39 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:21:39 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:21:39 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:21:39 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:21:39 user nova-compute[72351]:       <nova:owner>
Aug 09 02:21:39 user nova-compute[72351]:         <nova:user uuid="419b28d5a46c481db435715a30ab2614">tempest-ServerRescueNegativeTestJSON-289364438-project-member</nova:user>
Aug 09 02:21:39 user nova-compute[72351]:         <nova:project uuid="3aea1e66805146f7a8cd1d97e64a24af">tempest-ServerRescueNegativeTestJSON-289364438</nova:project>
Aug 09 02:21:39 user nova-compute[72351]:       </nova:owner>
Aug 09 02:21:39 user nova-compute[72351]:       <nova:root type="image" uuid="18ccda7f-31d3-4978-ab9d-4ed2f74eff94"/>
Aug 09 02:21:39 user nova-compute[72351]:       <nova:ports>
Aug 09 02:21:39 user nova-compute[72351]:         <nova:port uuid="5f97db70-9e5b-4d42-b765-b832fb522f41">
Aug 09 02:21:39 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.5" ipVersion="4"/>
Aug 09 02:21:39 user nova-compute[72351]:         </nova:port>
Aug 09 02:21:39 user nova-compute[72351]:       </nova:ports>
Aug 09 02:21:39 user nova-compute[72351]:     </nova:instance>
Aug 09 02:21:39 user nova-compute[72351]:   </metadata>
Aug 09 02:21:39 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:21:39 user nova-compute[72351]:     <system>
Aug 09 02:21:39 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:21:39 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:21:39 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:21:39 user nova-compute[72351]:       <entry name="serial">eb86cb49-2018-42c2-bd97-c662b829775d</entry>
Aug 09 02:21:39 user nova-compute[72351]:       <entry name="uuid">eb86cb49-2018-42c2-bd97-c662b829775d</entry>
Aug 09 02:21:39 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:21:39 user nova-compute[72351]:     </system>
Aug 09 02:21:39 user nova-compute[72351]:   </sysinfo>
Aug 09 02:21:39 user nova-compute[72351]:   <os>
Aug 09 02:21:39 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:21:39 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:21:39 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:21:39 user nova-compute[72351]:   </os>
Aug 09 02:21:39 user nova-compute[72351]:   <features>
Aug 09 02:21:39 user nova-compute[72351]:     <acpi/>
Aug 09 02:21:39 user nova-compute[72351]:     <apic/>
Aug 09 02:21:39 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:21:39 user nova-compute[72351]:   </features>
Aug 09 02:21:39 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:21:39 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:21:39 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:21:39 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:21:39 user nova-compute[72351]:   </clock>
Aug 09 02:21:39 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:21:39 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:21:39 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:21:39 user nova-compute[72351]:   </cpu>
Aug 09 02:21:39 user nova-compute[72351]:   <devices>
Aug 09 02:21:39 user nova-compute[72351]:     <disk type="file" device="disk">
Aug 09 02:21:39 user nova-compute[72351]:       <driver name="qemu" type="qcow2" cache="none"/>
Aug 09 02:21:39 user nova-compute[72351]:       <source file="/opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk"/>
Aug 09 02:21:39 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:21:39 user nova-compute[72351]:     </disk>
Aug 09 02:21:39 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:21:39 user nova-compute[72351]:       <mac address="fa:16:3e:dc:da:c7"/>
Aug 09 02:21:39 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:21:39 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:21:39 user nova-compute[72351]:       <target dev="tap5f97db70-9e"/>
Aug 09 02:21:39 user nova-compute[72351]:     </interface>
Aug 09 02:21:39 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:21:39 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/console.log" append="off"/>
Aug 09 02:21:39 user nova-compute[72351]:     </serial>
Aug 09 02:21:39 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:21:39 user nova-compute[72351]:     <video>
Aug 09 02:21:39 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:21:39 user nova-compute[72351]:     </video>
Aug 09 02:21:39 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:21:39 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:21:39 user nova-compute[72351]:     </rng>
Aug 09 02:21:39 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:21:39 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:21:39 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:21:39 user nova-compute[72351]:     </memballoon>
Aug 09 02:21:39 user nova-compute[72351]:   </devices>
Aug 09 02:21:39 user nova-compute[72351]: </domain>
Aug 09 02:21:39 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:21:35Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerRescueNegativeTestJSON-server-69135895',display_name='tempest-ServerRescueNegativeTestJSON-server-69135895',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serverrescuenegativetestjson-server-69135895',id=17,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='3aea1e66805146f7a8cd1d97e64a24af',ramdisk_id='',reservation_id='r-b0vsogov',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerRescueNegativeTestJSON-289364438',owner_user_name='tempest-ServerRescueNegativeTestJSON-289364438-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:21:37Z,user_data=None,user_id='419b28d5a46c481db435715a30ab2614',uuid=eb86cb49-2018-42c2-bd97-c662b829775d,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "5f97db70-9e5b-4d42-b765-b832fb522f41", "address": "fa:16:3e:dc:da:c7", "network": {"id": "49e8751c-e119-430f-9110-3dc6069b1589", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-899275135-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3aea1e66805146f7a8cd1d97e64a24af", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5f97db70-9e", "ovs_interfaceid": "5f97db70-9e5b-4d42-b765-b832fb522f41", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Converting VIF {"id": "5f97db70-9e5b-4d42-b765-b832fb522f41", "address": "fa:16:3e:dc:da:c7", "network": {"id": "49e8751c-e119-430f-9110-3dc6069b1589", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-899275135-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3aea1e66805146f7a8cd1d97e64a24af", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5f97db70-9e", "ovs_interfaceid": "5f97db70-9e5b-4d42-b765-b832fb522f41", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:dc:da:c7,bridge_name='br-int',has_traffic_filtering=True,id=5f97db70-9e5b-4d42-b765-b832fb522f41,network=Network(49e8751c-e119-430f-9110-3dc6069b1589),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap5f97db70-9e') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG os_vif [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:dc:da:c7,bridge_name='br-int',has_traffic_filtering=True,id=5f97db70-9e5b-4d42-b765-b832fb522f41,network=Network(49e8751c-e119-430f-9110-3dc6069b1589),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap5f97db70-9e') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap5f97db70-9e, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap5f97db70-9e, col_values=(('external_ids', {'iface-id': '5f97db70-9e5b-4d42-b765-b832fb522f41', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:dc:da:c7', 'vm-uuid': 'eb86cb49-2018-42c2-bd97-c662b829775d'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:39 user nova-compute[72351]: INFO os_vif [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:dc:da:c7,bridge_name='br-int',has_traffic_filtering=True,id=5f97db70-9e5b-4d42-b765-b832fb522f41,network=Network(49e8751c-e119-430f-9110-3dc6069b1589),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap5f97db70-9e')
Aug 09 02:21:39 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:21:39 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] No VIF found with MAC fa:16:3e:dc:da:c7, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:21:40 user nova-compute[72351]: DEBUG nova.network.neutron [req-12a5a427-542a-4c55-9bec-13adee076ffa req-cb02d63d-ef17-45f9-b2ad-6366d99039d0 service nova] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Updated VIF entry in instance network info cache for port 5f97db70-9e5b-4d42-b765-b832fb522f41. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:21:40 user nova-compute[72351]: DEBUG nova.network.neutron [req-12a5a427-542a-4c55-9bec-13adee076ffa req-cb02d63d-ef17-45f9-b2ad-6366d99039d0 service nova] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Updating instance_info_cache with network_info: [{"id": "5f97db70-9e5b-4d42-b765-b832fb522f41", "address": "fa:16:3e:dc:da:c7", "network": {"id": "49e8751c-e119-430f-9110-3dc6069b1589", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-899275135-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3aea1e66805146f7a8cd1d97e64a24af", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5f97db70-9e", "ovs_interfaceid": "5f97db70-9e5b-4d42-b765-b832fb522f41", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:21:40 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-12a5a427-542a-4c55-9bec-13adee076ffa req-cb02d63d-ef17-45f9-b2ad-6366d99039d0 service nova] Releasing lock "refresh_cache-eb86cb49-2018-42c2-bd97-c662b829775d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:21:41 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:41 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:41 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:41 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:41 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:41 user nova-compute[72351]: DEBUG nova.compute.manager [req-9d7afe42-b875-4bb6-ba2b-d940c9f04909 req-b997f045-6160-4013-883d-f866d2fe10ea service nova] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Received event network-vif-plugged-9481e457-9c23-4c45-b0bd-1faf63f8b7b8 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:21:41 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-9d7afe42-b875-4bb6-ba2b-d940c9f04909 req-b997f045-6160-4013-883d-f866d2fe10ea service nova] Acquiring lock "e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:21:41 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-9d7afe42-b875-4bb6-ba2b-d940c9f04909 req-b997f045-6160-4013-883d-f866d2fe10ea service nova] Lock "e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:21:41 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-9d7afe42-b875-4bb6-ba2b-d940c9f04909 req-b997f045-6160-4013-883d-f866d2fe10ea service nova] Lock "e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:21:41 user nova-compute[72351]: DEBUG nova.compute.manager [req-9d7afe42-b875-4bb6-ba2b-d940c9f04909 req-b997f045-6160-4013-883d-f866d2fe10ea service nova] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] No waiting events found dispatching network-vif-plugged-9481e457-9c23-4c45-b0bd-1faf63f8b7b8 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:21:41 user nova-compute[72351]: WARNING nova.compute.manager [req-9d7afe42-b875-4bb6-ba2b-d940c9f04909 req-b997f045-6160-4013-883d-f866d2fe10ea service nova] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Received unexpected event network-vif-plugged-9481e457-9c23-4c45-b0bd-1faf63f8b7b8 for instance with vm_state building and task_state spawning.
Aug 09 02:21:41 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:41 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:42 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:42 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:42 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:42 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:42 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:42 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:42 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:43 user nova-compute[72351]: DEBUG nova.compute.manager [req-461e6e59-53df-421b-8185-9b9484ad99bc req-0b8c17e6-6b6a-454b-a5b5-8a1140e59ce3 service nova] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Received event network-vif-plugged-9481e457-9c23-4c45-b0bd-1faf63f8b7b8 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:21:43 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-461e6e59-53df-421b-8185-9b9484ad99bc req-0b8c17e6-6b6a-454b-a5b5-8a1140e59ce3 service nova] Acquiring lock "e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:21:43 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-461e6e59-53df-421b-8185-9b9484ad99bc req-0b8c17e6-6b6a-454b-a5b5-8a1140e59ce3 service nova] Lock "e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:21:43 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-461e6e59-53df-421b-8185-9b9484ad99bc req-0b8c17e6-6b6a-454b-a5b5-8a1140e59ce3 service nova] Lock "e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:21:43 user nova-compute[72351]: DEBUG nova.compute.manager [req-461e6e59-53df-421b-8185-9b9484ad99bc req-0b8c17e6-6b6a-454b-a5b5-8a1140e59ce3 service nova] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] No waiting events found dispatching network-vif-plugged-9481e457-9c23-4c45-b0bd-1faf63f8b7b8 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:21:43 user nova-compute[72351]: WARNING nova.compute.manager [req-461e6e59-53df-421b-8185-9b9484ad99bc req-0b8c17e6-6b6a-454b-a5b5-8a1140e59ce3 service nova] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Received unexpected event network-vif-plugged-9481e457-9c23-4c45-b0bd-1faf63f8b7b8 for instance with vm_state building and task_state spawning.
Aug 09 02:21:43 user nova-compute[72351]: DEBUG nova.compute.manager [req-461e6e59-53df-421b-8185-9b9484ad99bc req-0b8c17e6-6b6a-454b-a5b5-8a1140e59ce3 service nova] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Received event network-vif-plugged-5f97db70-9e5b-4d42-b765-b832fb522f41 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:21:43 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-461e6e59-53df-421b-8185-9b9484ad99bc req-0b8c17e6-6b6a-454b-a5b5-8a1140e59ce3 service nova] Acquiring lock "eb86cb49-2018-42c2-bd97-c662b829775d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:21:43 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-461e6e59-53df-421b-8185-9b9484ad99bc req-0b8c17e6-6b6a-454b-a5b5-8a1140e59ce3 service nova] Lock "eb86cb49-2018-42c2-bd97-c662b829775d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:21:43 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-461e6e59-53df-421b-8185-9b9484ad99bc req-0b8c17e6-6b6a-454b-a5b5-8a1140e59ce3 service nova] Lock "eb86cb49-2018-42c2-bd97-c662b829775d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:21:43 user nova-compute[72351]: DEBUG nova.compute.manager [req-461e6e59-53df-421b-8185-9b9484ad99bc req-0b8c17e6-6b6a-454b-a5b5-8a1140e59ce3 service nova] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] No waiting events found dispatching network-vif-plugged-5f97db70-9e5b-4d42-b765-b832fb522f41 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:21:43 user nova-compute[72351]: WARNING nova.compute.manager [req-461e6e59-53df-421b-8185-9b9484ad99bc req-0b8c17e6-6b6a-454b-a5b5-8a1140e59ce3 service nova] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Received unexpected event network-vif-plugged-5f97db70-9e5b-4d42-b765-b832fb522f41 for instance with vm_state building and task_state spawning.
Aug 09 02:21:43 user nova-compute[72351]: DEBUG nova.compute.manager [req-461e6e59-53df-421b-8185-9b9484ad99bc req-0b8c17e6-6b6a-454b-a5b5-8a1140e59ce3 service nova] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Received event network-vif-plugged-5f97db70-9e5b-4d42-b765-b832fb522f41 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:21:43 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-461e6e59-53df-421b-8185-9b9484ad99bc req-0b8c17e6-6b6a-454b-a5b5-8a1140e59ce3 service nova] Acquiring lock "eb86cb49-2018-42c2-bd97-c662b829775d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:21:43 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-461e6e59-53df-421b-8185-9b9484ad99bc req-0b8c17e6-6b6a-454b-a5b5-8a1140e59ce3 service nova] Lock "eb86cb49-2018-42c2-bd97-c662b829775d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:21:43 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-461e6e59-53df-421b-8185-9b9484ad99bc req-0b8c17e6-6b6a-454b-a5b5-8a1140e59ce3 service nova] Lock "eb86cb49-2018-42c2-bd97-c662b829775d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:21:43 user nova-compute[72351]: DEBUG nova.compute.manager [req-461e6e59-53df-421b-8185-9b9484ad99bc req-0b8c17e6-6b6a-454b-a5b5-8a1140e59ce3 service nova] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] No waiting events found dispatching network-vif-plugged-5f97db70-9e5b-4d42-b765-b832fb522f41 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:21:43 user nova-compute[72351]: WARNING nova.compute.manager [req-461e6e59-53df-421b-8185-9b9484ad99bc req-0b8c17e6-6b6a-454b-a5b5-8a1140e59ce3 service nova] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Received unexpected event network-vif-plugged-5f97db70-9e5b-4d42-b765-b832fb522f41 for instance with vm_state building and task_state spawning.
Aug 09 02:21:44 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547704.4787536, e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335 => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:21:44 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] VM Resumed (Lifecycle Event)
Aug 09 02:21:44 user nova-compute[72351]: DEBUG nova.compute.manager [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:21:44 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Guest created on hypervisor {{(pid=72351) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4411}}
Aug 09 02:21:44 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Instance spawned successfully.
Aug 09 02:21:44 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:894}}
Aug 09 02:21:44 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:21:44 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:21:44 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Found default for hw_cdrom_bus of ide {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:21:44 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Found default for hw_disk_bus of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:21:44 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Found default for hw_input_bus of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:21:44 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Found default for hw_pointer_model of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:21:44 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Found default for hw_video_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:21:44 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Found default for hw_vif_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:21:44 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:21:44 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547704.4799068, e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335 => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:21:44 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] VM Started (Lifecycle Event)
Aug 09 02:21:44 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:21:44 user nova-compute[72351]: DEBUG nova.compute.manager [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:21:44 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Guest created on hypervisor {{(pid=72351) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4411}}
Aug 09 02:21:44 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:21:44 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Instance spawned successfully.
Aug 09 02:21:44 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:894}}
Aug 09 02:21:44 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:21:44 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547704.5769083, eb86cb49-2018-42c2-bd97-c662b829775d => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:21:44 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] VM Resumed (Lifecycle Event)
Aug 09 02:21:44 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Found default for hw_cdrom_bus of ide {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:21:44 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Found default for hw_disk_bus of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:21:44 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Found default for hw_input_bus of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:21:44 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Found default for hw_pointer_model of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:21:44 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Found default for hw_video_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:21:44 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Found default for hw_vif_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:21:44 user nova-compute[72351]: INFO nova.compute.manager [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Took 8.33 seconds to spawn the instance on the hypervisor.
Aug 09 02:21:44 user nova-compute[72351]: DEBUG nova.compute.manager [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:21:44 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:21:44 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:21:44 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:21:44 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547704.5782428, eb86cb49-2018-42c2-bd97-c662b829775d => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:21:44 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] VM Started (Lifecycle Event)
Aug 09 02:21:44 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:21:44 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:21:44 user nova-compute[72351]: INFO nova.compute.manager [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Took 7.34 seconds to spawn the instance on the hypervisor.
Aug 09 02:21:44 user nova-compute[72351]: DEBUG nova.compute.manager [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:21:44 user nova-compute[72351]: INFO nova.compute.manager [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Took 9.16 seconds to build instance.
Aug 09 02:21:44 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:21:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-90bab2ad-8e52-49cd-949e-1c4f5077851d tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 9.279s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:21:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:44 user nova-compute[72351]: INFO nova.compute.manager [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Took 8.48 seconds to build instance.
Aug 09 02:21:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-0e29c364-7ff3-4e2f-873f-6b81d0dbdbfc tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "eb86cb49-2018-42c2-bd97-c662b829775d" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 8.592s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:21:47 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:52 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:21:52 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:21:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:21:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:21:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:21:52 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Auditing locally available compute resources for user (node: user) {{(pid=72351) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Aug 09 02:21:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json" returned: 0 in 0.262s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json" returned: 0 in 0.153s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:53 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-d4631790-8972-49a3-a38f-73cc72c7df9c tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Acquiring lock "52f0d664-7767-4953-bc7a-496b26f22715" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:21:53 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-d4631790-8972-49a3-a38f-73cc72c7df9c tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Lock "52f0d664-7767-4953-bc7a-496b26f22715" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:21:53 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-d4631790-8972-49a3-a38f-73cc72c7df9c tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Acquiring lock "52f0d664-7767-4953-bc7a-496b26f22715-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:21:53 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-d4631790-8972-49a3-a38f-73cc72c7df9c tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Lock "52f0d664-7767-4953-bc7a-496b26f22715-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:21:53 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-d4631790-8972-49a3-a38f-73cc72c7df9c tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Lock "52f0d664-7767-4953-bc7a-496b26f22715-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:21:53 user nova-compute[72351]: INFO nova.compute.manager [None req-d4631790-8972-49a3-a38f-73cc72c7df9c tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] Terminating instance
Aug 09 02:21:53 user nova-compute[72351]: DEBUG nova.compute.manager [None req-d4631790-8972-49a3-a38f-73cc72c7df9c tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] Start destroying the instance on the hypervisor. {{(pid=72351) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3124}}
Aug 09 02:21:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/822e25c4-8dd1-4e21-b0ed-f340d59c6c98/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/822e25c4-8dd1-4e21-b0ed-f340d59c6c98/disk --force-share --output=json" returned: 0 in 0.167s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/822e25c4-8dd1-4e21-b0ed-f340d59c6c98/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/822e25c4-8dd1-4e21-b0ed-f340d59c6c98/disk --force-share --output=json" returned: 0 in 0.186s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:54 user nova-compute[72351]: DEBUG nova.compute.manager [req-d192c75f-98e2-4bb2-81fb-e7fc6dd6eda4 req-78bea821-f661-4681-8910-354d435a5fef service nova] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] Received event network-vif-unplugged-84074c82-d423-45d8-9665-bd92d13b6380 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:21:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d192c75f-98e2-4bb2-81fb-e7fc6dd6eda4 req-78bea821-f661-4681-8910-354d435a5fef service nova] Acquiring lock "52f0d664-7767-4953-bc7a-496b26f22715-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:21:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d192c75f-98e2-4bb2-81fb-e7fc6dd6eda4 req-78bea821-f661-4681-8910-354d435a5fef service nova] Lock "52f0d664-7767-4953-bc7a-496b26f22715-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:21:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d192c75f-98e2-4bb2-81fb-e7fc6dd6eda4 req-78bea821-f661-4681-8910-354d435a5fef service nova] Lock "52f0d664-7767-4953-bc7a-496b26f22715-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:21:54 user nova-compute[72351]: DEBUG nova.compute.manager [req-d192c75f-98e2-4bb2-81fb-e7fc6dd6eda4 req-78bea821-f661-4681-8910-354d435a5fef service nova] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] No waiting events found dispatching network-vif-unplugged-84074c82-d423-45d8-9665-bd92d13b6380 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:21:54 user nova-compute[72351]: DEBUG nova.compute.manager [req-d192c75f-98e2-4bb2-81fb-e7fc6dd6eda4 req-78bea821-f661-4681-8910-354d435a5fef service nova] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] Received event network-vif-unplugged-84074c82-d423-45d8-9665-bd92d13b6380 for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:21:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json" returned: 0 in 0.180s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json" returned: 0 in 0.156s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:54 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] Instance destroyed successfully.
Aug 09 02:21:54 user nova-compute[72351]: DEBUG nova.objects.instance [None req-d4631790-8972-49a3-a38f-73cc72c7df9c tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Lazy-loading 'resources' on Instance uuid 52f0d664-7767-4953-bc7a-496b26f22715 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:21:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-d4631790-8972-49a3-a38f-73cc72c7df9c tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:19:51Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-VolumesActionsTest-instance-1745726771',display_name='tempest-VolumesActionsTest-instance-1745726771',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-volumesactionstest-instance-1745726771',id=12,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:19:58Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='d6461f5cffd540b2b7f6797390561a80',ramdisk_id='',reservation_id='r-g9lvwc12',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-VolumesActionsTest-1803517492',owner_user_name='tempest-VolumesActionsTest-1803517492-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:19:59Z,user_data=None,user_id='ae566c7921844ee0b146d36259fbaf36',uuid=52f0d664-7767-4953-bc7a-496b26f22715,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "84074c82-d423-45d8-9665-bd92d13b6380", "address": "fa:16:3e:63:b2:2a", "network": {"id": "885b8ede-6326-4f34-aaaf-159df1b5e4e5", "bridge": "br-int", "label": "tempest-VolumesActionsTest-661898154-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d6461f5cffd540b2b7f6797390561a80", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap84074c82-d4", "ovs_interfaceid": "84074c82-d423-45d8-9665-bd92d13b6380", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:21:54 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-d4631790-8972-49a3-a38f-73cc72c7df9c tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Converting VIF {"id": "84074c82-d423-45d8-9665-bd92d13b6380", "address": "fa:16:3e:63:b2:2a", "network": {"id": "885b8ede-6326-4f34-aaaf-159df1b5e4e5", "bridge": "br-int", "label": "tempest-VolumesActionsTest-661898154-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d6461f5cffd540b2b7f6797390561a80", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap84074c82-d4", "ovs_interfaceid": "84074c82-d423-45d8-9665-bd92d13b6380", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:21:54 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-d4631790-8972-49a3-a38f-73cc72c7df9c tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:63:b2:2a,bridge_name='br-int',has_traffic_filtering=True,id=84074c82-d423-45d8-9665-bd92d13b6380,network=Network(885b8ede-6326-4f34-aaaf-159df1b5e4e5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap84074c82-d4') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:21:54 user nova-compute[72351]: DEBUG os_vif [None req-d4631790-8972-49a3-a38f-73cc72c7df9c tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:63:b2:2a,bridge_name='br-int',has_traffic_filtering=True,id=84074c82-d423-45d8-9665-bd92d13b6380,network=Network(885b8ede-6326-4f34-aaaf-159df1b5e4e5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap84074c82-d4') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:21:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2/disk --force-share --output=json" returned: 0 in 0.172s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap84074c82-d4, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:21:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:21:54 user nova-compute[72351]: INFO os_vif [None req-d4631790-8972-49a3-a38f-73cc72c7df9c tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:63:b2:2a,bridge_name='br-int',has_traffic_filtering=True,id=84074c82-d423-45d8-9665-bd92d13b6380,network=Network(885b8ede-6326-4f34-aaaf-159df1b5e4e5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap84074c82-d4')
Aug 09 02:21:54 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-d4631790-8972-49a3-a38f-73cc72c7df9c tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] Deleting instance files /opt/stack/data/nova/instances/52f0d664-7767-4953-bc7a-496b26f22715_del
Aug 09 02:21:54 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-d4631790-8972-49a3-a38f-73cc72c7df9c tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] Deletion of /opt/stack/data/nova/instances/52f0d664-7767-4953-bc7a-496b26f22715_del complete
Aug 09 02:21:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2/disk --force-share --output=json" returned: 0 in 0.133s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdc (vdb) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:21:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2/disk.rescue --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:54 user nova-compute[72351]: INFO nova.compute.manager [None req-d4631790-8972-49a3-a38f-73cc72c7df9c tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] Took 1.07 seconds to destroy the instance on the hypervisor.
Aug 09 02:21:54 user nova-compute[72351]: DEBUG oslo.service.loopingcall [None req-d4631790-8972-49a3-a38f-73cc72c7df9c tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72351) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Aug 09 02:21:54 user nova-compute[72351]: DEBUG nova.compute.manager [-] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] Deallocating network for instance {{(pid=72351) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2263}}
Aug 09 02:21:54 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] deallocate_for_instance() {{(pid=72351) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1795}}
Aug 09 02:21:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2/disk.rescue --force-share --output=json" returned: 0 in 0.144s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2/disk.rescue --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2/disk.rescue --force-share --output=json" returned: 0 in 0.178s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a1d1f738-0edf-4184-aa56-020c07830492/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a1d1f738-0edf-4184-aa56-020c07830492/disk --force-share --output=json" returned: 0 in 0.157s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a1d1f738-0edf-4184-aa56-020c07830492/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:55 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:21:55 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] Took 0.56 seconds to deallocate network for instance.
Aug 09 02:21:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a1d1f738-0edf-4184-aa56-020c07830492/disk --force-share --output=json" returned: 0 in 0.150s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:55 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Error from libvirt while getting description of instance-0000000c: [Error Code 42] Domain not found: no domain with matching uuid '52f0d664-7767-4953-bc7a-496b26f22715' (instance-0000000c): libvirt.libvirtError: Domain not found: no domain with matching uuid '52f0d664-7767-4953-bc7a-496b26f22715' (instance-0000000c)
Aug 09 02:21:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/74728ca8-34d4-45e6-a1f2-21c0952ff835/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-d4631790-8972-49a3-a38f-73cc72c7df9c tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:21:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-d4631790-8972-49a3-a38f-73cc72c7df9c tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.003s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:21:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/74728ca8-34d4-45e6-a1f2-21c0952ff835/disk --force-share --output=json" returned: 0 in 0.183s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/74728ca8-34d4-45e6-a1f2-21c0952ff835/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/74728ca8-34d4-45e6-a1f2-21c0952ff835/disk --force-share --output=json" returned: 0 in 0.162s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/97b13ccd-15b4-44bf-b865-71b4da861168/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:55 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-d4631790-8972-49a3-a38f-73cc72c7df9c tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:21:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/97b13ccd-15b4-44bf-b865-71b4da861168/disk --force-share --output=json" returned: 0 in 0.147s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/97b13ccd-15b4-44bf-b865-71b4da861168/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:55 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-d4631790-8972-49a3-a38f-73cc72c7df9c tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:21:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-d4631790-8972-49a3-a38f-73cc72c7df9c tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.557s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:21:55 user nova-compute[72351]: INFO nova.scheduler.client.report [None req-d4631790-8972-49a3-a38f-73cc72c7df9c tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Deleted allocations for instance 52f0d664-7767-4953-bc7a-496b26f22715
Aug 09 02:21:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/97b13ccd-15b4-44bf-b865-71b4da861168/disk --force-share --output=json" returned: 0 in 0.187s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:55 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdb (vdb) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:21:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:56 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-d4631790-8972-49a3-a38f-73cc72c7df9c tempest-VolumesActionsTest-1803517492 tempest-VolumesActionsTest-1803517492-project-member] Lock "52f0d664-7767-4953-bc7a-496b26f22715" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.495s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:21:56 user nova-compute[72351]: DEBUG nova.compute.manager [req-52755b1e-fd34-4737-b1a9-e601bfc8d801 req-4a94e804-f1ee-4203-bcd0-50c0cb4ba590 service nova] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] Received event network-vif-plugged-84074c82-d423-45d8-9665-bd92d13b6380 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:21:56 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-52755b1e-fd34-4737-b1a9-e601bfc8d801 req-4a94e804-f1ee-4203-bcd0-50c0cb4ba590 service nova] Acquiring lock "52f0d664-7767-4953-bc7a-496b26f22715-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:21:56 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-52755b1e-fd34-4737-b1a9-e601bfc8d801 req-4a94e804-f1ee-4203-bcd0-50c0cb4ba590 service nova] Lock "52f0d664-7767-4953-bc7a-496b26f22715-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:21:56 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-52755b1e-fd34-4737-b1a9-e601bfc8d801 req-4a94e804-f1ee-4203-bcd0-50c0cb4ba590 service nova] Lock "52f0d664-7767-4953-bc7a-496b26f22715-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:21:56 user nova-compute[72351]: DEBUG nova.compute.manager [req-52755b1e-fd34-4737-b1a9-e601bfc8d801 req-4a94e804-f1ee-4203-bcd0-50c0cb4ba590 service nova] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] No waiting events found dispatching network-vif-plugged-84074c82-d423-45d8-9665-bd92d13b6380 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:21:56 user nova-compute[72351]: WARNING nova.compute.manager [req-52755b1e-fd34-4737-b1a9-e601bfc8d801 req-4a94e804-f1ee-4203-bcd0-50c0cb4ba590 service nova] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] Received unexpected event network-vif-plugged-84074c82-d423-45d8-9665-bd92d13b6380 for instance with vm_state deleted and task_state None.
Aug 09 02:21:56 user nova-compute[72351]: DEBUG nova.compute.manager [req-52755b1e-fd34-4737-b1a9-e601bfc8d801 req-4a94e804-f1ee-4203-bcd0-50c0cb4ba590 service nova] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] Received event network-vif-deleted-84074c82-d423-45d8-9665-bd92d13b6380 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:21:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff/disk --force-share --output=json" returned: 0 in 0.176s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff/disk --force-share --output=json" returned: 0 in 0.133s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/32a28ecb-af4b-4c6e-8f2d-a51857134b34/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/32a28ecb-af4b-4c6e-8f2d-a51857134b34/disk --force-share --output=json" returned: 0 in 0.155s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/32a28ecb-af4b-4c6e-8f2d-a51857134b34/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/32a28ecb-af4b-4c6e-8f2d-a51857134b34/disk --force-share --output=json" returned: 0 in 0.142s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:57 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:21:57 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:21:57 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Hypervisor/Node resource view: name=user free_ram=7084MB free_disk=25.45849609375GB free_vcpus=2 pci_devices=[{"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72351) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Aug 09 02:21:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:21:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:21:57 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 2d78475c-107d-4256-b12e-dbe9c8f3b9b2 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:21:57 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 74728ca8-34d4-45e6-a1f2-21c0952ff835 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:21:57 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 97b13ccd-15b4-44bf-b865-71b4da861168 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:21:57 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 822e25c4-8dd1-4e21-b0ed-f340d59c6c98 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:21:57 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance a1d1f738-0edf-4184-aa56-020c07830492 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:21:57 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 32a28ecb-af4b-4c6e-8f2d-a51857134b34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:21:57 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 75bbbead-8628-448b-9484-19bf4ab922ff actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:21:57 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:21:57 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance eb86cb49-2018-42c2-bd97-c662b829775d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:21:57 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Total usable vcpus: 12, total allocated vcpus: 9 {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Aug 09 02:21:57 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Final resource view: name=user phys_ram=16011MB used_ram=5120MB phys_disk=40GB used_disk=9GB total_vcpus=12 used_vcpus=9 pci_stats=[] {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Aug 09 02:21:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquiring lock "1d772c10-1c8a-4101-9a61-f8a60bb19c66" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:21:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "1d772c10-1c8a-4101-9a61-f8a60bb19c66" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:21:57 user nova-compute[72351]: DEBUG nova.compute.manager [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Starting instance... {{(pid=72351) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2406}}
Aug 09 02:21:57 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:21:57 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:21:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:21:57 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Compute_service record updated for user:user {{(pid=72351) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Aug 09 02:21:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.530s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:21:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.012s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:21:57 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72351) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Aug 09 02:21:57 user nova-compute[72351]: INFO nova.compute.claims [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Claim successful on node user
Aug 09 02:21:58 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:21:58 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:21:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.489s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:21:58 user nova-compute[72351]: DEBUG nova.compute.manager [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Start building networks asynchronously for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2803}}
Aug 09 02:21:58 user nova-compute[72351]: DEBUG nova.compute.manager [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Allocating IP information in the background. {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1956}}
Aug 09 02:21:58 user nova-compute[72351]: DEBUG nova.network.neutron [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] allocate_for_instance() {{(pid=72351) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Aug 09 02:21:58 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Aug 09 02:21:58 user nova-compute[72351]: DEBUG nova.compute.manager [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Start building block device mappings for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2838}}
Aug 09 02:21:58 user nova-compute[72351]: INFO nova.virt.block_device [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Booting with volume ba022fd7-964b-4325-9d01-74fb60f45274 at /dev/vda
Aug 09 02:21:58 user nova-compute[72351]: DEBUG os_brick.utils [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:21:58 user nova-compute[72351]: DEBUG nova.policy [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '07d8d5a634274df19c2db769d2d27b29', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4c6bbfd8d904ba1ac69ba5da4564513', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72351) authorize /opt/stack/nova/nova/policy.py:203}}
Aug 09 02:21:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.015s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:58 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[82d7e9e1-cf50-408c-9950-cf335b58e1ea]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:a52269602ff4\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:21:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:58 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[6775cb43-a159-4df7-b2dc-48cdb818e04f]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:21:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:21:58 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[aa12e715-50e3-4111-b91b-75857e977b87]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:21:58 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d42df5ee-1907-4940-aa63-edf49261f8fd]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:21:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Running cmd (subprocess): nvme version {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] 'nvme version' failed. Not Retrying. {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:21:58 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.nvmeof [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] nvme not present on system {{(pid=72351) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Aug 09 02:21:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:21:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:21:58 user nova-compute[72351]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Aug 09 02:21:58 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ecfa1733-f615-4033-ad23-9b0b27725981]: (4, '') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:21:58 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72351) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Aug 09 02:21:58 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Aug 09 02:21:58 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] LIGHTOS: no hostnqn found. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Aug 09 02:21:58 user nova-compute[72351]: DEBUG os_brick.utils [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] <== get_connector_properties: return (130ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:a52269602ff4', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:21:58 user nova-compute[72351]: DEBUG nova.virt.block_device [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Updating existing volume attachment record: 3c5507cf-deb9-4005-885b-2390e403031d {{(pid=72351) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Aug 09 02:21:59 user nova-compute[72351]: DEBUG nova.network.neutron [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Successfully created port: 55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7 {{(pid=72351) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Aug 09 02:21:59 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:21:59 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:21:59 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:21:59 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Starting heal instance info cache {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9842}}
Aug 09 02:21:59 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Didn't find any instances for network info cache update. {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9928}}
Aug 09 02:21:59 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:21:59 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:21:59 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:21:59 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:21:59 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:21:59 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72351) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10461}}
Aug 09 02:22:00 user nova-compute[72351]: DEBUG nova.network.neutron [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Successfully updated port: 55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7 {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:22:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquiring lock "refresh_cache-1d772c10-1c8a-4101-9a61-f8a60bb19c66" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:22:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquired lock "refresh_cache-1d772c10-1c8a-4101-9a61-f8a60bb19c66" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:22:00 user nova-compute[72351]: DEBUG nova.network.neutron [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:22:00 user nova-compute[72351]: DEBUG nova.compute.manager [req-cb9a5b0c-7849-4e0d-aa03-d6e07fe87d7b req-b4ec6dab-22a3-4cd1-8e5b-6dee2e7bf13f service nova] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Received event network-changed-55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:22:00 user nova-compute[72351]: DEBUG nova.compute.manager [req-cb9a5b0c-7849-4e0d-aa03-d6e07fe87d7b req-b4ec6dab-22a3-4cd1-8e5b-6dee2e7bf13f service nova] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Refreshing instance network info cache due to event network-changed-55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:22:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-cb9a5b0c-7849-4e0d-aa03-d6e07fe87d7b req-b4ec6dab-22a3-4cd1-8e5b-6dee2e7bf13f service nova] Acquiring lock "refresh_cache-1d772c10-1c8a-4101-9a61-f8a60bb19c66" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:22:00 user nova-compute[72351]: DEBUG nova.network.neutron [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:22:00 user nova-compute[72351]: DEBUG nova.network.neutron [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Updating instance_info_cache with network_info: [{"id": "55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7", "address": "fa:16:3e:75:70:ff", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap55e3cfd3-c6", "ovs_interfaceid": "55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:22:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Releasing lock "refresh_cache-1d772c10-1c8a-4101-9a61-f8a60bb19c66" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:22:00 user nova-compute[72351]: DEBUG nova.compute.manager [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Instance network_info: |[{"id": "55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7", "address": "fa:16:3e:75:70:ff", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap55e3cfd3-c6", "ovs_interfaceid": "55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1971}}
Aug 09 02:22:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-cb9a5b0c-7849-4e0d-aa03-d6e07fe87d7b req-b4ec6dab-22a3-4cd1-8e5b-6dee2e7bf13f service nova] Acquired lock "refresh_cache-1d772c10-1c8a-4101-9a61-f8a60bb19c66" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:22:00 user nova-compute[72351]: DEBUG nova.network.neutron [req-cb9a5b0c-7849-4e0d-aa03-d6e07fe87d7b req-b4ec6dab-22a3-4cd1-8e5b-6dee2e7bf13f service nova] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Refreshing network info cache for port 55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:22:00 user nova-compute[72351]: DEBUG nova.network.neutron [req-cb9a5b0c-7849-4e0d-aa03-d6e07fe87d7b req-b4ec6dab-22a3-4cd1-8e5b-6dee2e7bf13f service nova] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Updated VIF entry in instance network info cache for port 55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:22:00 user nova-compute[72351]: DEBUG nova.network.neutron [req-cb9a5b0c-7849-4e0d-aa03-d6e07fe87d7b req-b4ec6dab-22a3-4cd1-8e5b-6dee2e7bf13f service nova] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Updating instance_info_cache with network_info: [{"id": "55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7", "address": "fa:16:3e:75:70:ff", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap55e3cfd3-c6", "ovs_interfaceid": "55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:22:01 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-cb9a5b0c-7849-4e0d-aa03-d6e07fe87d7b req-b4ec6dab-22a3-4cd1-8e5b-6dee2e7bf13f service nova] Releasing lock "refresh_cache-1d772c10-1c8a-4101-9a61-f8a60bb19c66" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:22:02 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:06 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:07 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:09 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691547714.4465308, 52f0d664-7767-4953-bc7a-496b26f22715 => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:22:09 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] VM Stopped (Lifecycle Event)
Aug 09 02:22:09 user nova-compute[72351]: DEBUG nova.compute.manager [None req-bbd8b453-0aea-4838-9ffa-e8e517ea6eef None None] [instance: 52f0d664-7767-4953-bc7a-496b26f22715] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:22:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:12 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:12 user nova-compute[72351]: DEBUG nova.compute.manager [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Start spawning the instance on the hypervisor. {{(pid=72351) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2612}}
Aug 09 02:22:12 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Creating instance directory {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4717}}
Aug 09 02:22:12 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Creating image(s)
Aug 09 02:22:12 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Did not create local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4853}}
Aug 09 02:22:12 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Ensure instance console log exists: /opt/stack/data/nova/instances/1d772c10-1c8a-4101-9a61-f8a60bb19c66/console.log {{(pid=72351) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4603}}
Aug 09 02:22:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:22:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:22:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:22:12 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Start _get_guest_xml network_info=[{"id": "55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7", "address": "fa:16:3e:75:70:ff", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap55e3cfd3-c6", "ovs_interfaceid": "55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'device_type': 'disk', 'guest_format': None, 'disk_bus': 'virtio', 'mount_device': '/dev/vda', 'boot_index': 0, 'attachment_id': '3c5507cf-deb9-4005-885b-2390e403031d', 'delete_on_termination': True, 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqns': ['iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'xbIDCRQN', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '1d772c10-1c8a-4101-9a61-f8a60bb19c66', 'attached_at': '', 'detached_at': '', 'volume_id': 'ba022fd7-964b-4325-9d01-74fb60f45274', 'serial': 'ba022fd7-964b-4325-9d01-74fb60f45274'}, 'volume_type': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:22:12 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:22:12 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:22:12 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:22:12 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:22:12 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:22:12 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:22:12 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:22:12 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:22:12 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:22:12 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:22:12 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:22:12 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:22:12 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:22:12 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:22:12 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:22:12 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:22:12 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:22:12 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:22:12 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 02:22:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'xbIDCRQN', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:22:12 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:22:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:22:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:22:12 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:22:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:22:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:22:12 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 02:22:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:22:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 -p 172.16.0.220:3260" returned: 21 in 0.024s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:22:12 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[2b0daa9a-87e1-4646-8fc9-8bd92215e547]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:22:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Aug 09 02:22:12 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:22:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 -p 172.16.0.220:3260 --interface default --op new {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:22:12 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.024s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:22:12 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[27b4cd95-15d2-4228-9b29-efd92e562013]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274] added\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:22:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274] added
Aug 09 02:22:12 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:22:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:22:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:22:13 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[bb2d5a4c-748b-45c7-9aab-8d989cfa13bc]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:22:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:22:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:22:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:22:13 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[7d603212-2979-48a1-a8e0-a4c5538829f8]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:22:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:22:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v xbIDCRQN {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:22:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v xbIDCRQN" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:22:13 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[7006e710-37cb-4877-933d-959e246136aa]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:22:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'xbIDCRQN'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:22:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:22:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:22:13 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[6779eca1-d0f3-40de-b36f-a889d3c18a07]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:22:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:22:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:22:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:22:13 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[0a6b9a4f-a8c4-4830-b27e-1a85781f7ce1]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f (non-flash)\ntcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:22:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f (non-flash)
Aug 09 02:22:13 user nova-compute[72351]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497 (non-flash)
Aug 09 02:22:13 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:22:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f (non-flash)
Aug 09 02:22:13 user nova-compute[72351]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497 (non-flash)
Aug 09 02:22:13 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:22:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 -p 172.16.0.220:3260 --login {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:22:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 -p 172.16.0.220:3260 --login" returned: 0 in 0.062s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:22:13 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[c04a2891-d3bf-4e84-a032-492e0e04cf4c]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:22:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274, portal: 172.16.0.220,3260]
Aug 09 02:22:13 user nova-compute[72351]: Login to [iface: default, target: iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274, portal: 172.16.0.220,3260] successful.
Aug 09 02:22:13 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:22:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:22:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.044s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:22:13 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[5d768c0b-5653-42fe-b079-d1ca37a4861a]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:22:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:22:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:22:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:22:13 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ad2099a2-722c-426c-a0fd-8004c21d650e]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f (non-flash)\ntcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497 (non-flash)\ntcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:22:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f (non-flash)
Aug 09 02:22:13 user nova-compute[72351]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497 (non-flash)
Aug 09 02:22:13 user nova-compute[72351]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)
Aug 09 02:22:13 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:22:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f (non-flash)
Aug 09 02:22:13 user nova-compute[72351]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497 (non-flash)
Aug 09 02:22:13 user nova-compute[72351]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)
Aug 09 02:22:13 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:22:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.322s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:22:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 02:22:13 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] HCTL ('35', '-', '-', 0) found on session 17 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 02:22:13 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Scanning host 35 c: -, t: -, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 02:22:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host35/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:22:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host35/scan" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:22:13 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[6a8f98e5-b587-4a1b-94b6-331682e0a1ae]: (4, ('- - 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:22:13 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Searching for a device in session 17 and hctl ['35', '*', '*', 0] yield: sdd {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:22:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Connected to sdd using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'xbIDCRQN', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 02:22:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-8787720e-dfa6-4321-8d97-6cc0ee95878e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Acquiring lock "822e25c4-8dd1-4e21-b0ed-f340d59c6c98" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:22:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-8787720e-dfa6-4321-8d97-6cc0ee95878e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Lock "822e25c4-8dd1-4e21-b0ed-f340d59c6c98" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:22:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-8787720e-dfa6-4321-8d97-6cc0ee95878e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Acquiring lock "822e25c4-8dd1-4e21-b0ed-f340d59c6c98-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:22:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-8787720e-dfa6-4321-8d97-6cc0ee95878e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Lock "822e25c4-8dd1-4e21-b0ed-f340d59c6c98-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:22:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-8787720e-dfa6-4321-8d97-6cc0ee95878e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Lock "822e25c4-8dd1-4e21-b0ed-f340d59c6c98-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:22:13 user nova-compute[72351]: INFO nova.compute.manager [None req-8787720e-dfa6-4321-8d97-6cc0ee95878e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Terminating instance
Aug 09 02:22:13 user nova-compute[72351]: DEBUG nova.compute.manager [None req-8787720e-dfa6-4321-8d97-6cc0ee95878e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Start destroying the instance on the hypervisor. {{(pid=72351) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3124}}
Aug 09 02:22:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:13 user nova-compute[72351]: DEBUG nova.compute.manager [req-58950d03-7f2d-4e1e-a312-48304f910b85 req-4868e461-7191-4836-8b00-7ee44287a9f2 service nova] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Received event network-vif-unplugged-6bfee618-fc3b-43ba-b96f-c75db4b7a4e9 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:22:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-58950d03-7f2d-4e1e-a312-48304f910b85 req-4868e461-7191-4836-8b00-7ee44287a9f2 service nova] Acquiring lock "822e25c4-8dd1-4e21-b0ed-f340d59c6c98-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:22:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-58950d03-7f2d-4e1e-a312-48304f910b85 req-4868e461-7191-4836-8b00-7ee44287a9f2 service nova] Lock "822e25c4-8dd1-4e21-b0ed-f340d59c6c98-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:22:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-58950d03-7f2d-4e1e-a312-48304f910b85 req-4868e461-7191-4836-8b00-7ee44287a9f2 service nova] Lock "822e25c4-8dd1-4e21-b0ed-f340d59c6c98-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:22:13 user nova-compute[72351]: DEBUG nova.compute.manager [req-58950d03-7f2d-4e1e-a312-48304f910b85 req-4868e461-7191-4836-8b00-7ee44287a9f2 service nova] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] No waiting events found dispatching network-vif-unplugged-6bfee618-fc3b-43ba-b96f-c75db4b7a4e9 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:22:13 user nova-compute[72351]: DEBUG nova.compute.manager [req-58950d03-7f2d-4e1e-a312-48304f910b85 req-4868e461-7191-4836-8b00-7ee44287a9f2 service nova] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Received event network-vif-unplugged-6bfee618-fc3b-43ba-b96f-c75db4b7a4e9 for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:22:14 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Instance destroyed successfully.
Aug 09 02:22:14 user nova-compute[72351]: DEBUG nova.objects.instance [None req-8787720e-dfa6-4321-8d97-6cc0ee95878e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Lazy-loading 'resources' on Instance uuid 822e25c4-8dd1-4e21-b0ed-f340d59c6c98 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:22:14 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-8787720e-dfa6-4321-8d97-6cc0ee95878e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:19:01Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestServerVolumeAttachScenarioOldVersion-server-1011938981',display_name='tempest-TestServerVolumeAttachScenarioOldVersion-server-1011938981',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testservervolumeattachscenariooldversion-server-1011938',id=11,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGPTAtBLvv+eQnb1GxApCCwfO/b9dFAlv4Z9d8CIpZZkoHUa1BvykzJdDs1KfKyHfsGzGGa6sqh+JXq3NyAXht/DjjCN8xgzJjWBqtfbK66i+iGwapUmiS6V9Np3Mgd9og==',key_name='tempest-keypair-1739372982',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:19:11Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='5fd127f53fc54fdbbf8ae524f3578f91',ramdisk_id='',reservation_id='r-cat7ueqw',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TestServerVolumeAttachScenarioOldVersion-59067075',owner_user_name='tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:19:12Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='761c64dbddf443dbb05b2c1211b9a91d',uuid=822e25c4-8dd1-4e21-b0ed-f340d59c6c98,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "6bfee618-fc3b-43ba-b96f-c75db4b7a4e9", "address": "fa:16:3e:e4:67:dd", "network": {"id": "66067153-d668-48d7-9851-b33abf90f2e1", "bridge": "br-int", "label": "tempest-TestServerVolumeAttachScenarioOldVersion-1194345640-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.228", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5fd127f53fc54fdbbf8ae524f3578f91", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6bfee618-fc", "ovs_interfaceid": "6bfee618-fc3b-43ba-b96f-c75db4b7a4e9", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:22:14 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-8787720e-dfa6-4321-8d97-6cc0ee95878e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Converting VIF {"id": "6bfee618-fc3b-43ba-b96f-c75db4b7a4e9", "address": "fa:16:3e:e4:67:dd", "network": {"id": "66067153-d668-48d7-9851-b33abf90f2e1", "bridge": "br-int", "label": "tempest-TestServerVolumeAttachScenarioOldVersion-1194345640-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.228", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5fd127f53fc54fdbbf8ae524f3578f91", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6bfee618-fc", "ovs_interfaceid": "6bfee618-fc3b-43ba-b96f-c75db4b7a4e9", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:22:14 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-8787720e-dfa6-4321-8d97-6cc0ee95878e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:e4:67:dd,bridge_name='br-int',has_traffic_filtering=True,id=6bfee618-fc3b-43ba-b96f-c75db4b7a4e9,network=Network(66067153-d668-48d7-9851-b33abf90f2e1),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6bfee618-fc') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:22:14 user nova-compute[72351]: DEBUG os_vif [None req-8787720e-dfa6-4321-8d97-6cc0ee95878e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:e4:67:dd,bridge_name='br-int',has_traffic_filtering=True,id=6bfee618-fc3b-43ba-b96f-c75db4b7a4e9,network=Network(66067153-d668-48d7-9851-b33abf90f2e1),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6bfee618-fc') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:22:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap6bfee618-fc, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:22:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:22:14 user nova-compute[72351]: INFO os_vif [None req-8787720e-dfa6-4321-8d97-6cc0ee95878e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:e4:67:dd,bridge_name='br-int',has_traffic_filtering=True,id=6bfee618-fc3b-43ba-b96f-c75db4b7a4e9,network=Network(66067153-d668-48d7-9851-b33abf90f2e1),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6bfee618-fc')
Aug 09 02:22:14 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-8787720e-dfa6-4321-8d97-6cc0ee95878e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Deleting instance files /opt/stack/data/nova/instances/822e25c4-8dd1-4e21-b0ed-f340d59c6c98_del
Aug 09 02:22:14 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-8787720e-dfa6-4321-8d97-6cc0ee95878e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Deletion of /opt/stack/data/nova/instances/822e25c4-8dd1-4e21-b0ed-f340d59c6c98_del complete
Aug 09 02:22:14 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.358s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:22:14 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] <== connect_volume: return (1360ms) {'type': 'block', 'scsi_wwn': '26164653131363530', 'path': '/dev/sdd'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:22:14 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26164653131363530', 'path': '/dev/sdd'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 02:22:14 user nova-compute[72351]: INFO nova.compute.manager [None req-8787720e-dfa6-4321-8d97-6cc0ee95878e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Took 0.96 seconds to destroy the instance on the hypervisor.
Aug 09 02:22:14 user nova-compute[72351]: DEBUG oslo.service.loopingcall [None req-8787720e-dfa6-4321-8d97-6cc0ee95878e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72351) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Aug 09 02:22:14 user nova-compute[72351]: DEBUG nova.compute.manager [-] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Deallocating network for instance {{(pid=72351) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2263}}
Aug 09 02:22:14 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] deallocate_for_instance() {{(pid=72351) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1795}}
Aug 09 02:22:14 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:21:56Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-volume-backed-server-543725645',display_name='tempest-TestVolumeBootPattern-volume-backed-server-543725645',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testvolumebootpattern-volume-backed-server-543725645',id=18,image_ref='',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBG11cGkjeY18Po8k08DMyUzQtsrtCaWAq6iJyHMr9Whc2GLjfVhsEnL4e/8YHs5tlZeSHiwOT/FcEFipB83gNXXJkM/cq3cq4F+DuxaxT/aGoMoUGxsL8LTol/71KOxcMQ==',key_name='tempest-keypair-1791716351',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='a4c6bbfd8d904ba1ac69ba5da4564513',ramdisk_id='',reservation_id='r-2d0wwhw3',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestVolumeBootPattern-350743460',owner_user_name='tempest-TestVolumeBootPattern-350743460-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:21:58Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='07d8d5a634274df19c2db769d2d27b29',uuid=1d772c10-1c8a-4101-9a61-f8a60bb19c66,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7", "address": "fa:16:3e:75:70:ff", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap55e3cfd3-c6", "ovs_interfaceid": "55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:22:14 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Converting VIF {"id": "55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7", "address": "fa:16:3e:75:70:ff", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap55e3cfd3-c6", "ovs_interfaceid": "55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:22:14 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:75:70:ff,bridge_name='br-int',has_traffic_filtering=True,id=55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7,network=Network(5ef428cf-e408-490c-8f60-70934f760f4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap55e3cfd3-c6') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:22:14 user nova-compute[72351]: DEBUG nova.objects.instance [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lazy-loading 'pci_devices' on Instance uuid 1d772c10-1c8a-4101-9a61-f8a60bb19c66 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:22:14 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:22:14 user nova-compute[72351]:   <uuid>1d772c10-1c8a-4101-9a61-f8a60bb19c66</uuid>
Aug 09 02:22:14 user nova-compute[72351]:   <name>instance-00000012</name>
Aug 09 02:22:14 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:22:14 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:22:14 user nova-compute[72351]:   <metadata>
Aug 09 02:22:14 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:22:14 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:22:14 user nova-compute[72351]:       <nova:name>tempest-TestVolumeBootPattern-volume-backed-server-543725645</nova:name>
Aug 09 02:22:14 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:22:12</nova:creationTime>
Aug 09 02:22:14 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:22:14 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:22:14 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:22:14 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:22:14 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:22:14 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:22:14 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:22:14 user nova-compute[72351]:       <nova:owner>
Aug 09 02:22:14 user nova-compute[72351]:         <nova:user uuid="07d8d5a634274df19c2db769d2d27b29">tempest-TestVolumeBootPattern-350743460-project-member</nova:user>
Aug 09 02:22:14 user nova-compute[72351]:         <nova:project uuid="a4c6bbfd8d904ba1ac69ba5da4564513">tempest-TestVolumeBootPattern-350743460</nova:project>
Aug 09 02:22:14 user nova-compute[72351]:       </nova:owner>
Aug 09 02:22:14 user nova-compute[72351]:       <nova:ports>
Aug 09 02:22:14 user nova-compute[72351]:         <nova:port uuid="55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7">
Aug 09 02:22:14 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.13" ipVersion="4"/>
Aug 09 02:22:14 user nova-compute[72351]:         </nova:port>
Aug 09 02:22:14 user nova-compute[72351]:       </nova:ports>
Aug 09 02:22:14 user nova-compute[72351]:     </nova:instance>
Aug 09 02:22:14 user nova-compute[72351]:   </metadata>
Aug 09 02:22:14 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:22:14 user nova-compute[72351]:     <system>
Aug 09 02:22:14 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:22:14 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:22:14 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:22:14 user nova-compute[72351]:       <entry name="serial">1d772c10-1c8a-4101-9a61-f8a60bb19c66</entry>
Aug 09 02:22:14 user nova-compute[72351]:       <entry name="uuid">1d772c10-1c8a-4101-9a61-f8a60bb19c66</entry>
Aug 09 02:22:14 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:22:14 user nova-compute[72351]:     </system>
Aug 09 02:22:14 user nova-compute[72351]:   </sysinfo>
Aug 09 02:22:14 user nova-compute[72351]:   <os>
Aug 09 02:22:14 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:22:14 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:22:14 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:22:14 user nova-compute[72351]:   </os>
Aug 09 02:22:14 user nova-compute[72351]:   <features>
Aug 09 02:22:14 user nova-compute[72351]:     <acpi/>
Aug 09 02:22:14 user nova-compute[72351]:     <apic/>
Aug 09 02:22:14 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:22:14 user nova-compute[72351]:   </features>
Aug 09 02:22:14 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:22:14 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:22:14 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:22:14 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:22:14 user nova-compute[72351]:   </clock>
Aug 09 02:22:14 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:22:14 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:22:14 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:22:14 user nova-compute[72351]:   </cpu>
Aug 09 02:22:14 user nova-compute[72351]:   <devices>
Aug 09 02:22:14 user nova-compute[72351]:     <disk type="block" device="disk">
Aug 09 02:22:14 user nova-compute[72351]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:22:14 user nova-compute[72351]:       <source dev="/dev/sdd"/>
Aug 09 02:22:14 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:22:14 user nova-compute[72351]:       <serial>ba022fd7-964b-4325-9d01-74fb60f45274</serial>
Aug 09 02:22:14 user nova-compute[72351]:     </disk>
Aug 09 02:22:14 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:22:14 user nova-compute[72351]:       <mac address="fa:16:3e:75:70:ff"/>
Aug 09 02:22:14 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:22:14 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:22:14 user nova-compute[72351]:       <target dev="tap55e3cfd3-c6"/>
Aug 09 02:22:14 user nova-compute[72351]:     </interface>
Aug 09 02:22:14 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:22:14 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/1d772c10-1c8a-4101-9a61-f8a60bb19c66/console.log" append="off"/>
Aug 09 02:22:14 user nova-compute[72351]:     </serial>
Aug 09 02:22:14 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:22:14 user nova-compute[72351]:     <video>
Aug 09 02:22:14 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:22:14 user nova-compute[72351]:     </video>
Aug 09 02:22:14 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:22:14 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:22:14 user nova-compute[72351]:     </rng>
Aug 09 02:22:14 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:22:14 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:22:14 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:22:14 user nova-compute[72351]:     </memballoon>
Aug 09 02:22:14 user nova-compute[72351]:   </devices>
Aug 09 02:22:14 user nova-compute[72351]: </domain>
Aug 09 02:22:14 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:22:14 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:21:56Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-volume-backed-server-543725645',display_name='tempest-TestVolumeBootPattern-volume-backed-server-543725645',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testvolumebootpattern-volume-backed-server-543725645',id=18,image_ref='',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBG11cGkjeY18Po8k08DMyUzQtsrtCaWAq6iJyHMr9Whc2GLjfVhsEnL4e/8YHs5tlZeSHiwOT/FcEFipB83gNXXJkM/cq3cq4F+DuxaxT/aGoMoUGxsL8LTol/71KOxcMQ==',key_name='tempest-keypair-1791716351',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='a4c6bbfd8d904ba1ac69ba5da4564513',ramdisk_id='',reservation_id='r-2d0wwhw3',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestVolumeBootPattern-350743460',owner_user_name='tempest-TestVolumeBootPattern-350743460-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:21:58Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='07d8d5a634274df19c2db769d2d27b29',uuid=1d772c10-1c8a-4101-9a61-f8a60bb19c66,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7", "address": "fa:16:3e:75:70:ff", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap55e3cfd3-c6", "ovs_interfaceid": "55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:22:14 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Converting VIF {"id": "55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7", "address": "fa:16:3e:75:70:ff", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap55e3cfd3-c6", "ovs_interfaceid": "55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:22:14 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:75:70:ff,bridge_name='br-int',has_traffic_filtering=True,id=55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7,network=Network(5ef428cf-e408-490c-8f60-70934f760f4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap55e3cfd3-c6') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:22:14 user nova-compute[72351]: DEBUG os_vif [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:75:70:ff,bridge_name='br-int',has_traffic_filtering=True,id=55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7,network=Network(5ef428cf-e408-490c-8f60-70934f760f4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap55e3cfd3-c6') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:22:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:22:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:22:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap55e3cfd3-c6, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:22:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap55e3cfd3-c6, col_values=(('external_ids', {'iface-id': '55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:75:70:ff', 'vm-uuid': '1d772c10-1c8a-4101-9a61-f8a60bb19c66'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:22:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:22:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:14 user nova-compute[72351]: INFO os_vif [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:75:70:ff,bridge_name='br-int',has_traffic_filtering=True,id=55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7,network=Network(5ef428cf-e408-490c-8f60-70934f760f4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap55e3cfd3-c6')
Aug 09 02:22:14 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:22:14 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] No VIF found with MAC fa:16:3e:75:70:ff, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:22:15 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:15 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:15 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:15 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:15 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:22:16 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Took 1.72 seconds to deallocate network for instance.
Aug 09 02:22:16 user nova-compute[72351]: DEBUG nova.compute.manager [req-3bb7478b-9358-410f-b8e0-60e615c8e6e5 req-674bdb77-ef54-4dc4-8069-cbaf335081af service nova] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Received event network-vif-plugged-6bfee618-fc3b-43ba-b96f-c75db4b7a4e9 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:22:16 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-3bb7478b-9358-410f-b8e0-60e615c8e6e5 req-674bdb77-ef54-4dc4-8069-cbaf335081af service nova] Acquiring lock "822e25c4-8dd1-4e21-b0ed-f340d59c6c98-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:22:16 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-3bb7478b-9358-410f-b8e0-60e615c8e6e5 req-674bdb77-ef54-4dc4-8069-cbaf335081af service nova] Lock "822e25c4-8dd1-4e21-b0ed-f340d59c6c98-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:22:16 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-3bb7478b-9358-410f-b8e0-60e615c8e6e5 req-674bdb77-ef54-4dc4-8069-cbaf335081af service nova] Lock "822e25c4-8dd1-4e21-b0ed-f340d59c6c98-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:22:16 user nova-compute[72351]: DEBUG nova.compute.manager [req-3bb7478b-9358-410f-b8e0-60e615c8e6e5 req-674bdb77-ef54-4dc4-8069-cbaf335081af service nova] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] No waiting events found dispatching network-vif-plugged-6bfee618-fc3b-43ba-b96f-c75db4b7a4e9 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:22:16 user nova-compute[72351]: WARNING nova.compute.manager [req-3bb7478b-9358-410f-b8e0-60e615c8e6e5 req-674bdb77-ef54-4dc4-8069-cbaf335081af service nova] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Received unexpected event network-vif-plugged-6bfee618-fc3b-43ba-b96f-c75db4b7a4e9 for instance with vm_state active and task_state deleting.
Aug 09 02:22:16 user nova-compute[72351]: DEBUG nova.compute.manager [req-3bb7478b-9358-410f-b8e0-60e615c8e6e5 req-674bdb77-ef54-4dc4-8069-cbaf335081af service nova] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Received event network-vif-deleted-6bfee618-fc3b-43ba-b96f-c75db4b7a4e9 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:22:16 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-8787720e-dfa6-4321-8d97-6cc0ee95878e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:22:16 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-8787720e-dfa6-4321-8d97-6cc0ee95878e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:22:16 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-8787720e-dfa6-4321-8d97-6cc0ee95878e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:22:16 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:16 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:16 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-8787720e-dfa6-4321-8d97-6cc0ee95878e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:22:16 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:16 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:16 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-8787720e-dfa6-4321-8d97-6cc0ee95878e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.492s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:22:16 user nova-compute[72351]: INFO nova.scheduler.client.report [None req-8787720e-dfa6-4321-8d97-6cc0ee95878e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Deleted allocations for instance 822e25c4-8dd1-4e21-b0ed-f340d59c6c98
Aug 09 02:22:16 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:16 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-8787720e-dfa6-4321-8d97-6cc0ee95878e tempest-TestServerVolumeAttachScenarioOldVersion-59067075 tempest-TestServerVolumeAttachScenarioOldVersion-59067075-project-member] Lock "822e25c4-8dd1-4e21-b0ed-f340d59c6c98" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 3.578s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:22:16 user nova-compute[72351]: DEBUG nova.compute.manager [req-e5dba55d-5499-4c48-a419-9a270a26d941 req-35a39a40-95e7-4922-b313-335073033bad service nova] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Received event network-vif-plugged-55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:22:16 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-e5dba55d-5499-4c48-a419-9a270a26d941 req-35a39a40-95e7-4922-b313-335073033bad service nova] Acquiring lock "1d772c10-1c8a-4101-9a61-f8a60bb19c66-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:22:16 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-e5dba55d-5499-4c48-a419-9a270a26d941 req-35a39a40-95e7-4922-b313-335073033bad service nova] Lock "1d772c10-1c8a-4101-9a61-f8a60bb19c66-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:22:16 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-e5dba55d-5499-4c48-a419-9a270a26d941 req-35a39a40-95e7-4922-b313-335073033bad service nova] Lock "1d772c10-1c8a-4101-9a61-f8a60bb19c66-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:22:16 user nova-compute[72351]: DEBUG nova.compute.manager [req-e5dba55d-5499-4c48-a419-9a270a26d941 req-35a39a40-95e7-4922-b313-335073033bad service nova] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] No waiting events found dispatching network-vif-plugged-55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:22:16 user nova-compute[72351]: WARNING nova.compute.manager [req-e5dba55d-5499-4c48-a419-9a270a26d941 req-35a39a40-95e7-4922-b313-335073033bad service nova] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Received unexpected event network-vif-plugged-55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7 for instance with vm_state building and task_state spawning.
Aug 09 02:22:17 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:18 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547738.2907102, 1d772c10-1c8a-4101-9a61-f8a60bb19c66 => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:22:18 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] VM Resumed (Lifecycle Event)
Aug 09 02:22:18 user nova-compute[72351]: DEBUG nova.compute.manager [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:22:18 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Guest created on hypervisor {{(pid=72351) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4411}}
Aug 09 02:22:18 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Instance spawned successfully.
Aug 09 02:22:18 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:894}}
Aug 09 02:22:18 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:22:18 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:22:18 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Found default for hw_cdrom_bus of ide {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:22:18 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Found default for hw_disk_bus of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:22:18 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Found default for hw_input_bus of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:22:18 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Found default for hw_pointer_model of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:22:18 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Found default for hw_video_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:22:18 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Found default for hw_vif_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:22:18 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:22:18 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547738.2961624, 1d772c10-1c8a-4101-9a61-f8a60bb19c66 => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:22:18 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] VM Started (Lifecycle Event)
Aug 09 02:22:18 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:22:18 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:22:18 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:22:18 user nova-compute[72351]: INFO nova.compute.manager [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Took 5.64 seconds to spawn the instance on the hypervisor.
Aug 09 02:22:18 user nova-compute[72351]: DEBUG nova.compute.manager [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:22:18 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f57a1810-3229-4ad7-9ab0-e492ad53b5d4 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Acquiring lock "32a28ecb-af4b-4c6e-8f2d-a51857134b34" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:22:18 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f57a1810-3229-4ad7-9ab0-e492ad53b5d4 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Lock "32a28ecb-af4b-4c6e-8f2d-a51857134b34" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:22:18 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f57a1810-3229-4ad7-9ab0-e492ad53b5d4 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Acquiring lock "32a28ecb-af4b-4c6e-8f2d-a51857134b34-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:22:18 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f57a1810-3229-4ad7-9ab0-e492ad53b5d4 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Lock "32a28ecb-af4b-4c6e-8f2d-a51857134b34-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:22:18 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f57a1810-3229-4ad7-9ab0-e492ad53b5d4 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Lock "32a28ecb-af4b-4c6e-8f2d-a51857134b34-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:22:18 user nova-compute[72351]: INFO nova.compute.manager [None req-f57a1810-3229-4ad7-9ab0-e492ad53b5d4 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] Terminating instance
Aug 09 02:22:18 user nova-compute[72351]: DEBUG nova.compute.manager [None req-f57a1810-3229-4ad7-9ab0-e492ad53b5d4 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] Start destroying the instance on the hypervisor. {{(pid=72351) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3124}}
Aug 09 02:22:18 user nova-compute[72351]: INFO nova.compute.manager [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Took 20.87 seconds to build instance.
Aug 09 02:22:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:18 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-08588c5b-3d5e-4524-bb0b-4fad9ce9e30b tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "1d772c10-1c8a-4101-9a61-f8a60bb19c66" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 21.014s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:22:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:18 user nova-compute[72351]: DEBUG nova.compute.manager [req-788fa48f-6129-4d5f-b59f-43a2a6834763 req-d9e51c04-96d4-4125-9ac4-19290bf0fd70 service nova] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] Received event network-vif-unplugged-4bd2f461-68b0-4882-98d6-2bb99a324210 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:22:18 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-788fa48f-6129-4d5f-b59f-43a2a6834763 req-d9e51c04-96d4-4125-9ac4-19290bf0fd70 service nova] Acquiring lock "32a28ecb-af4b-4c6e-8f2d-a51857134b34-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:22:18 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-788fa48f-6129-4d5f-b59f-43a2a6834763 req-d9e51c04-96d4-4125-9ac4-19290bf0fd70 service nova] Lock "32a28ecb-af4b-4c6e-8f2d-a51857134b34-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:22:18 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-788fa48f-6129-4d5f-b59f-43a2a6834763 req-d9e51c04-96d4-4125-9ac4-19290bf0fd70 service nova] Lock "32a28ecb-af4b-4c6e-8f2d-a51857134b34-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:22:18 user nova-compute[72351]: DEBUG nova.compute.manager [req-788fa48f-6129-4d5f-b59f-43a2a6834763 req-d9e51c04-96d4-4125-9ac4-19290bf0fd70 service nova] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] No waiting events found dispatching network-vif-unplugged-4bd2f461-68b0-4882-98d6-2bb99a324210 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:22:18 user nova-compute[72351]: DEBUG nova.compute.manager [req-788fa48f-6129-4d5f-b59f-43a2a6834763 req-d9e51c04-96d4-4125-9ac4-19290bf0fd70 service nova] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] Received event network-vif-unplugged-4bd2f461-68b0-4882-98d6-2bb99a324210 for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:22:19 user nova-compute[72351]: DEBUG nova.compute.manager [req-2570fb51-b84b-4308-a751-7e2f6367dba1 req-6b86b114-ae58-4a2d-a844-0c5f896b7b7e service nova] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Received event network-vif-plugged-55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:22:19 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-2570fb51-b84b-4308-a751-7e2f6367dba1 req-6b86b114-ae58-4a2d-a844-0c5f896b7b7e service nova] Acquiring lock "1d772c10-1c8a-4101-9a61-f8a60bb19c66-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:22:19 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-2570fb51-b84b-4308-a751-7e2f6367dba1 req-6b86b114-ae58-4a2d-a844-0c5f896b7b7e service nova] Lock "1d772c10-1c8a-4101-9a61-f8a60bb19c66-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:22:19 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-2570fb51-b84b-4308-a751-7e2f6367dba1 req-6b86b114-ae58-4a2d-a844-0c5f896b7b7e service nova] Lock "1d772c10-1c8a-4101-9a61-f8a60bb19c66-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:22:19 user nova-compute[72351]: DEBUG nova.compute.manager [req-2570fb51-b84b-4308-a751-7e2f6367dba1 req-6b86b114-ae58-4a2d-a844-0c5f896b7b7e service nova] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] No waiting events found dispatching network-vif-plugged-55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:22:19 user nova-compute[72351]: WARNING nova.compute.manager [req-2570fb51-b84b-4308-a751-7e2f6367dba1 req-6b86b114-ae58-4a2d-a844-0c5f896b7b7e service nova] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Received unexpected event network-vif-plugged-55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7 for instance with vm_state active and task_state None.
Aug 09 02:22:19 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] Instance destroyed successfully.
Aug 09 02:22:19 user nova-compute[72351]: DEBUG nova.objects.instance [None req-f57a1810-3229-4ad7-9ab0-e492ad53b5d4 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Lazy-loading 'resources' on Instance uuid 32a28ecb-af4b-4c6e-8f2d-a51857134b34 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:22:19 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:19 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-f57a1810-3229-4ad7-9ab0-e492ad53b5d4 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:20:27Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-VolumesActionsTest-instance-434546642',display_name='tempest-VolumesActionsTest-instance-434546642',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-volumesactionstest-instance-434546642',id=14,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:20:35Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='5209610aaaa34f7a8ffb7757dbfac85d',ramdisk_id='',reservation_id='r-4al0zprq',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-VolumesActionsTest-1343359891',owner_user_name='tempest-VolumesActionsTest-1343359891-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:20:36Z,user_data=None,user_id='00a45ab3cb2d43028418af1ae209da51',uuid=32a28ecb-af4b-4c6e-8f2d-a51857134b34,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "4bd2f461-68b0-4882-98d6-2bb99a324210", "address": "fa:16:3e:15:97:f8", "network": {"id": "fc48221d-cf2e-4871-9741-c102904e10a6", "bridge": "br-int", "label": "tempest-VolumesActionsTest-857807030-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5209610aaaa34f7a8ffb7757dbfac85d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4bd2f461-68", "ovs_interfaceid": "4bd2f461-68b0-4882-98d6-2bb99a324210", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:22:19 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-f57a1810-3229-4ad7-9ab0-e492ad53b5d4 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Converting VIF {"id": "4bd2f461-68b0-4882-98d6-2bb99a324210", "address": "fa:16:3e:15:97:f8", "network": {"id": "fc48221d-cf2e-4871-9741-c102904e10a6", "bridge": "br-int", "label": "tempest-VolumesActionsTest-857807030-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5209610aaaa34f7a8ffb7757dbfac85d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4bd2f461-68", "ovs_interfaceid": "4bd2f461-68b0-4882-98d6-2bb99a324210", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:22:19 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-f57a1810-3229-4ad7-9ab0-e492ad53b5d4 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:15:97:f8,bridge_name='br-int',has_traffic_filtering=True,id=4bd2f461-68b0-4882-98d6-2bb99a324210,network=Network(fc48221d-cf2e-4871-9741-c102904e10a6),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap4bd2f461-68') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:22:19 user nova-compute[72351]: DEBUG os_vif [None req-f57a1810-3229-4ad7-9ab0-e492ad53b5d4 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:15:97:f8,bridge_name='br-int',has_traffic_filtering=True,id=4bd2f461-68b0-4882-98d6-2bb99a324210,network=Network(fc48221d-cf2e-4871-9741-c102904e10a6),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap4bd2f461-68') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:22:19 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:19 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap4bd2f461-68, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:22:19 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:22:19 user nova-compute[72351]: INFO os_vif [None req-f57a1810-3229-4ad7-9ab0-e492ad53b5d4 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:15:97:f8,bridge_name='br-int',has_traffic_filtering=True,id=4bd2f461-68b0-4882-98d6-2bb99a324210,network=Network(fc48221d-cf2e-4871-9741-c102904e10a6),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap4bd2f461-68')
Aug 09 02:22:19 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-f57a1810-3229-4ad7-9ab0-e492ad53b5d4 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] Deleting instance files /opt/stack/data/nova/instances/32a28ecb-af4b-4c6e-8f2d-a51857134b34_del
Aug 09 02:22:19 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-f57a1810-3229-4ad7-9ab0-e492ad53b5d4 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] Deletion of /opt/stack/data/nova/instances/32a28ecb-af4b-4c6e-8f2d-a51857134b34_del complete
Aug 09 02:22:19 user nova-compute[72351]: INFO nova.compute.manager [None req-f57a1810-3229-4ad7-9ab0-e492ad53b5d4 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] Took 0.94 seconds to destroy the instance on the hypervisor.
Aug 09 02:22:19 user nova-compute[72351]: DEBUG oslo.service.loopingcall [None req-f57a1810-3229-4ad7-9ab0-e492ad53b5d4 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72351) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Aug 09 02:22:19 user nova-compute[72351]: DEBUG nova.compute.manager [-] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] Deallocating network for instance {{(pid=72351) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2263}}
Aug 09 02:22:19 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] deallocate_for_instance() {{(pid=72351) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1795}}
Aug 09 02:22:19 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:20 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:22:20 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] Took 1.00 seconds to deallocate network for instance.
Aug 09 02:22:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f57a1810-3229-4ad7-9ab0-e492ad53b5d4 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:22:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f57a1810-3229-4ad7-9ab0-e492ad53b5d4 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:22:20 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-f57a1810-3229-4ad7-9ab0-e492ad53b5d4 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:22:20 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-f57a1810-3229-4ad7-9ab0-e492ad53b5d4 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:22:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f57a1810-3229-4ad7-9ab0-e492ad53b5d4 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.402s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:22:21 user nova-compute[72351]: INFO nova.scheduler.client.report [None req-f57a1810-3229-4ad7-9ab0-e492ad53b5d4 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Deleted allocations for instance 32a28ecb-af4b-4c6e-8f2d-a51857134b34
Aug 09 02:22:21 user nova-compute[72351]: DEBUG nova.compute.manager [req-be74a799-e52b-43da-aeec-a574461a4dc5 req-c4b202e9-16ab-4e1d-90e0-5e09d1061629 service nova] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] Received event network-vif-plugged-4bd2f461-68b0-4882-98d6-2bb99a324210 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:22:21 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-be74a799-e52b-43da-aeec-a574461a4dc5 req-c4b202e9-16ab-4e1d-90e0-5e09d1061629 service nova] Acquiring lock "32a28ecb-af4b-4c6e-8f2d-a51857134b34-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:22:21 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-be74a799-e52b-43da-aeec-a574461a4dc5 req-c4b202e9-16ab-4e1d-90e0-5e09d1061629 service nova] Lock "32a28ecb-af4b-4c6e-8f2d-a51857134b34-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:22:21 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-be74a799-e52b-43da-aeec-a574461a4dc5 req-c4b202e9-16ab-4e1d-90e0-5e09d1061629 service nova] Lock "32a28ecb-af4b-4c6e-8f2d-a51857134b34-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:22:21 user nova-compute[72351]: DEBUG nova.compute.manager [req-be74a799-e52b-43da-aeec-a574461a4dc5 req-c4b202e9-16ab-4e1d-90e0-5e09d1061629 service nova] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] No waiting events found dispatching network-vif-plugged-4bd2f461-68b0-4882-98d6-2bb99a324210 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:22:21 user nova-compute[72351]: WARNING nova.compute.manager [req-be74a799-e52b-43da-aeec-a574461a4dc5 req-c4b202e9-16ab-4e1d-90e0-5e09d1061629 service nova] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] Received unexpected event network-vif-plugged-4bd2f461-68b0-4882-98d6-2bb99a324210 for instance with vm_state deleted and task_state None.
Aug 09 02:22:21 user nova-compute[72351]: DEBUG nova.compute.manager [req-be74a799-e52b-43da-aeec-a574461a4dc5 req-c4b202e9-16ab-4e1d-90e0-5e09d1061629 service nova] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] Received event network-vif-deleted-4bd2f461-68b0-4882-98d6-2bb99a324210 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:22:21 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f57a1810-3229-4ad7-9ab0-e492ad53b5d4 tempest-VolumesActionsTest-1343359891 tempest-VolumesActionsTest-1343359891-project-member] Lock "32a28ecb-af4b-4c6e-8f2d-a51857134b34" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.594s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:22:21 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Acquiring lock "585c81a5-ac24-4d12-a8a6-1e7b12a2d53a" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:22:21 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "585c81a5-ac24-4d12-a8a6-1e7b12a2d53a" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:22:21 user nova-compute[72351]: DEBUG nova.compute.manager [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Starting instance... {{(pid=72351) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2406}}
Aug 09 02:22:21 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:22:21 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:22:21 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72351) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Aug 09 02:22:21 user nova-compute[72351]: INFO nova.compute.claims [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Claim successful on node user
Aug 09 02:22:21 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:22:22 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:22:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.493s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:22:22 user nova-compute[72351]: DEBUG nova.compute.manager [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Start building networks asynchronously for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2803}}
Aug 09 02:22:22 user nova-compute[72351]: DEBUG nova.compute.manager [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Allocating IP information in the background. {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1956}}
Aug 09 02:22:22 user nova-compute[72351]: DEBUG nova.network.neutron [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] allocate_for_instance() {{(pid=72351) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Aug 09 02:22:22 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Aug 09 02:22:22 user nova-compute[72351]: DEBUG nova.compute.manager [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Start building block device mappings for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2838}}
Aug 09 02:22:22 user nova-compute[72351]: DEBUG nova.policy [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '5dca34fdadbb4386a1ff1cb255265d47', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '69d584c230b94ff8aae722eb2f364c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72351) authorize /opt/stack/nova/nova/policy.py:203}}
Aug 09 02:22:22 user nova-compute[72351]: DEBUG nova.compute.manager [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Start spawning the instance on the hypervisor. {{(pid=72351) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2612}}
Aug 09 02:22:22 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Creating instance directory {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4717}}
Aug 09 02:22:22 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Creating image(s)
Aug 09 02:22:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Acquiring lock "/opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:22:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "/opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:22:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "/opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:22:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:22:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.171s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:22:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Acquiring lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:22:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:22:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:22:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.159s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:22:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk 1073741824 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:22:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk 1073741824" returned: 0 in 0.058s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:22:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.224s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:22:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:22:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.175s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:22:22 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Checking if we can resize image /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk. size=1073741824 {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Aug 09 02:22:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:22:22 user nova-compute[72351]: DEBUG nova.network.neutron [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Successfully created port: e521fae4-093d-4c92-a32d-684c1a5137f5 {{(pid=72351) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Aug 09 02:22:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json" returned: 0 in 0.147s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:22:23 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Cannot resize image /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk to a smaller size. {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Aug 09 02:22:23 user nova-compute[72351]: DEBUG nova.objects.instance [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lazy-loading 'migration_context' on Instance uuid 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:22:23 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Created local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4851}}
Aug 09 02:22:23 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Ensure instance console log exists: /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/console.log {{(pid=72351) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4603}}
Aug 09 02:22:23 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:22:23 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:22:23 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:22:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:24 user nova-compute[72351]: DEBUG nova.network.neutron [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Successfully updated port: e521fae4-093d-4c92-a32d-684c1a5137f5 {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:22:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Acquiring lock "refresh_cache-585c81a5-ac24-4d12-a8a6-1e7b12a2d53a" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:22:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Acquired lock "refresh_cache-585c81a5-ac24-4d12-a8a6-1e7b12a2d53a" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:22:24 user nova-compute[72351]: DEBUG nova.network.neutron [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:22:24 user nova-compute[72351]: DEBUG nova.compute.manager [req-eb69b3a7-21d8-469a-bc8d-64d36f4b5111 req-a0992392-f4ad-463c-815b-1d72d70f6457 service nova] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Received event network-changed-e521fae4-093d-4c92-a32d-684c1a5137f5 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:22:24 user nova-compute[72351]: DEBUG nova.compute.manager [req-eb69b3a7-21d8-469a-bc8d-64d36f4b5111 req-a0992392-f4ad-463c-815b-1d72d70f6457 service nova] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Refreshing instance network info cache due to event network-changed-e521fae4-093d-4c92-a32d-684c1a5137f5. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:22:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-eb69b3a7-21d8-469a-bc8d-64d36f4b5111 req-a0992392-f4ad-463c-815b-1d72d70f6457 service nova] Acquiring lock "refresh_cache-585c81a5-ac24-4d12-a8a6-1e7b12a2d53a" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:22:24 user nova-compute[72351]: DEBUG nova.network.neutron [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:22:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6d6e4099-6c7b-4836-9cb6-34337a126618 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Acquiring lock "a1d1f738-0edf-4184-aa56-020c07830492" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:22:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6d6e4099-6c7b-4836-9cb6-34337a126618 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Lock "a1d1f738-0edf-4184-aa56-020c07830492" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:22:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6d6e4099-6c7b-4836-9cb6-34337a126618 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Acquiring lock "a1d1f738-0edf-4184-aa56-020c07830492-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:22:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6d6e4099-6c7b-4836-9cb6-34337a126618 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Lock "a1d1f738-0edf-4184-aa56-020c07830492-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:22:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6d6e4099-6c7b-4836-9cb6-34337a126618 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Lock "a1d1f738-0edf-4184-aa56-020c07830492-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:22:24 user nova-compute[72351]: INFO nova.compute.manager [None req-6d6e4099-6c7b-4836-9cb6-34337a126618 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: a1d1f738-0edf-4184-aa56-020c07830492] Terminating instance
Aug 09 02:22:24 user nova-compute[72351]: DEBUG nova.compute.manager [None req-6d6e4099-6c7b-4836-9cb6-34337a126618 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: a1d1f738-0edf-4184-aa56-020c07830492] Start destroying the instance on the hypervisor. {{(pid=72351) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3124}}
Aug 09 02:22:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:25 user nova-compute[72351]: DEBUG nova.network.neutron [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Updating instance_info_cache with network_info: [{"id": "e521fae4-093d-4c92-a32d-684c1a5137f5", "address": "fa:16:3e:e5:b0:10", "network": {"id": "b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-210689817-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "69d584c230b94ff8aae722eb2f364c5c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape521fae4-09", "ovs_interfaceid": "e521fae4-093d-4c92-a32d-684c1a5137f5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:22:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Releasing lock "refresh_cache-585c81a5-ac24-4d12-a8a6-1e7b12a2d53a" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:22:25 user nova-compute[72351]: DEBUG nova.compute.manager [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Instance network_info: |[{"id": "e521fae4-093d-4c92-a32d-684c1a5137f5", "address": "fa:16:3e:e5:b0:10", "network": {"id": "b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-210689817-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "69d584c230b94ff8aae722eb2f364c5c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape521fae4-09", "ovs_interfaceid": "e521fae4-093d-4c92-a32d-684c1a5137f5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1971}}
Aug 09 02:22:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-eb69b3a7-21d8-469a-bc8d-64d36f4b5111 req-a0992392-f4ad-463c-815b-1d72d70f6457 service nova] Acquired lock "refresh_cache-585c81a5-ac24-4d12-a8a6-1e7b12a2d53a" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:22:25 user nova-compute[72351]: DEBUG nova.network.neutron [req-eb69b3a7-21d8-469a-bc8d-64d36f4b5111 req-a0992392-f4ad-463c-815b-1d72d70f6457 service nova] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Refreshing network info cache for port e521fae4-093d-4c92-a32d-684c1a5137f5 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:22:25 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Start _get_guest_xml network_info=[{"id": "e521fae4-093d-4c92-a32d-684c1a5137f5", "address": "fa:16:3e:e5:b0:10", "network": {"id": "b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-210689817-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "69d584c230b94ff8aae722eb2f364c5c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape521fae4-09", "ovs_interfaceid": "e521fae4-093d-4c92-a32d-684c1a5137f5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'size': 0, 'device_type': 'disk', 'encryption_secret_uuid': None, 'guest_format': None, 'encryption_options': None, 'disk_bus': 'virtio', 'encrypted': False, 'encryption_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'image_id': '18ccda7f-31d3-4978-ab9d-4ed2f74eff94'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:22:25 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:22:25 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:22:25 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:22:25 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:22:25 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:22:25 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:22:25 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:22:25 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:22:25 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:22:25 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:22:25 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:22:25 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:22:25 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:22:25 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:22:25 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:22:25 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:22:25 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:22:25 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:22:25 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:22:20Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerActionsTestOtherA-server-446242587',display_name='tempest-ServerActionsTestOtherA-server-446242587',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serveractionstestothera-server-446242587',id=19,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBB4HNXglactRKXsG6uozX3XR7A39DBllYpgpCz+5+1RhfLOeXdmVkNfDkIG7vQrgoVNKWK/MwqtjPDLEVURcnWQfOuv4ySYdfEPj4hs6FpDqfJbAa5qX0FAbZQuEg69FVQ==',key_name='tempest-keypair-976839412',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='69d584c230b94ff8aae722eb2f364c5c',ramdisk_id='',reservation_id='r-x0v0gmsw',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerActionsTestOtherA-1303549305',owner_user_name='tempest-ServerActionsTestOtherA-1303549305-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:22:22Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='5dca34fdadbb4386a1ff1cb255265d47',uuid=585c81a5-ac24-4d12-a8a6-1e7b12a2d53a,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "e521fae4-093d-4c92-a32d-684c1a5137f5", "address": "fa:16:3e:e5:b0:10", "network": {"id": "b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-210689817-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "69d584c230b94ff8aae722eb2f364c5c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape521fae4-09", "ovs_interfaceid": "e521fae4-093d-4c92-a32d-684c1a5137f5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:22:25 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Converting VIF {"id": "e521fae4-093d-4c92-a32d-684c1a5137f5", "address": "fa:16:3e:e5:b0:10", "network": {"id": "b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-210689817-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "69d584c230b94ff8aae722eb2f364c5c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape521fae4-09", "ovs_interfaceid": "e521fae4-093d-4c92-a32d-684c1a5137f5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:22:25 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:e5:b0:10,bridge_name='br-int',has_traffic_filtering=True,id=e521fae4-093d-4c92-a32d-684c1a5137f5,network=Network(b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape521fae4-09') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:22:25 user nova-compute[72351]: DEBUG nova.objects.instance [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lazy-loading 'pci_devices' on Instance uuid 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:22:25 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:22:25 user nova-compute[72351]:   <uuid>585c81a5-ac24-4d12-a8a6-1e7b12a2d53a</uuid>
Aug 09 02:22:25 user nova-compute[72351]:   <name>instance-00000013</name>
Aug 09 02:22:25 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:22:25 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:22:25 user nova-compute[72351]:   <metadata>
Aug 09 02:22:25 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:22:25 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:22:25 user nova-compute[72351]:       <nova:name>tempest-ServerActionsTestOtherA-server-446242587</nova:name>
Aug 09 02:22:25 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:22:25</nova:creationTime>
Aug 09 02:22:25 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:22:25 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:22:25 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:22:25 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:22:25 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:22:25 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:22:25 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:22:25 user nova-compute[72351]:       <nova:owner>
Aug 09 02:22:25 user nova-compute[72351]:         <nova:user uuid="5dca34fdadbb4386a1ff1cb255265d47">tempest-ServerActionsTestOtherA-1303549305-project-member</nova:user>
Aug 09 02:22:25 user nova-compute[72351]:         <nova:project uuid="69d584c230b94ff8aae722eb2f364c5c">tempest-ServerActionsTestOtherA-1303549305</nova:project>
Aug 09 02:22:25 user nova-compute[72351]:       </nova:owner>
Aug 09 02:22:25 user nova-compute[72351]:       <nova:root type="image" uuid="18ccda7f-31d3-4978-ab9d-4ed2f74eff94"/>
Aug 09 02:22:25 user nova-compute[72351]:       <nova:ports>
Aug 09 02:22:25 user nova-compute[72351]:         <nova:port uuid="e521fae4-093d-4c92-a32d-684c1a5137f5">
Aug 09 02:22:25 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.4" ipVersion="4"/>
Aug 09 02:22:25 user nova-compute[72351]:         </nova:port>
Aug 09 02:22:25 user nova-compute[72351]:       </nova:ports>
Aug 09 02:22:25 user nova-compute[72351]:     </nova:instance>
Aug 09 02:22:25 user nova-compute[72351]:   </metadata>
Aug 09 02:22:25 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:22:25 user nova-compute[72351]:     <system>
Aug 09 02:22:25 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:22:25 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:22:25 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:22:25 user nova-compute[72351]:       <entry name="serial">585c81a5-ac24-4d12-a8a6-1e7b12a2d53a</entry>
Aug 09 02:22:25 user nova-compute[72351]:       <entry name="uuid">585c81a5-ac24-4d12-a8a6-1e7b12a2d53a</entry>
Aug 09 02:22:25 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:22:25 user nova-compute[72351]:     </system>
Aug 09 02:22:25 user nova-compute[72351]:   </sysinfo>
Aug 09 02:22:25 user nova-compute[72351]:   <os>
Aug 09 02:22:25 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:22:25 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:22:25 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:22:25 user nova-compute[72351]:   </os>
Aug 09 02:22:25 user nova-compute[72351]:   <features>
Aug 09 02:22:25 user nova-compute[72351]:     <acpi/>
Aug 09 02:22:25 user nova-compute[72351]:     <apic/>
Aug 09 02:22:25 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:22:25 user nova-compute[72351]:   </features>
Aug 09 02:22:25 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:22:25 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:22:25 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:22:25 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:22:25 user nova-compute[72351]:   </clock>
Aug 09 02:22:25 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:22:25 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:22:25 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:22:25 user nova-compute[72351]:   </cpu>
Aug 09 02:22:25 user nova-compute[72351]:   <devices>
Aug 09 02:22:25 user nova-compute[72351]:     <disk type="file" device="disk">
Aug 09 02:22:25 user nova-compute[72351]:       <driver name="qemu" type="qcow2" cache="none"/>
Aug 09 02:22:25 user nova-compute[72351]:       <source file="/opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk"/>
Aug 09 02:22:25 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:22:25 user nova-compute[72351]:     </disk>
Aug 09 02:22:25 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:22:25 user nova-compute[72351]:       <mac address="fa:16:3e:e5:b0:10"/>
Aug 09 02:22:25 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:22:25 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:22:25 user nova-compute[72351]:       <target dev="tape521fae4-09"/>
Aug 09 02:22:25 user nova-compute[72351]:     </interface>
Aug 09 02:22:25 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:22:25 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/console.log" append="off"/>
Aug 09 02:22:25 user nova-compute[72351]:     </serial>
Aug 09 02:22:25 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:22:25 user nova-compute[72351]:     <video>
Aug 09 02:22:25 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:22:25 user nova-compute[72351]:     </video>
Aug 09 02:22:25 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:22:25 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:22:25 user nova-compute[72351]:     </rng>
Aug 09 02:22:25 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:22:25 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:22:25 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:22:25 user nova-compute[72351]:     </memballoon>
Aug 09 02:22:25 user nova-compute[72351]:   </devices>
Aug 09 02:22:25 user nova-compute[72351]: </domain>
Aug 09 02:22:25 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:22:25 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:22:20Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerActionsTestOtherA-server-446242587',display_name='tempest-ServerActionsTestOtherA-server-446242587',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serveractionstestothera-server-446242587',id=19,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBB4HNXglactRKXsG6uozX3XR7A39DBllYpgpCz+5+1RhfLOeXdmVkNfDkIG7vQrgoVNKWK/MwqtjPDLEVURcnWQfOuv4ySYdfEPj4hs6FpDqfJbAa5qX0FAbZQuEg69FVQ==',key_name='tempest-keypair-976839412',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='69d584c230b94ff8aae722eb2f364c5c',ramdisk_id='',reservation_id='r-x0v0gmsw',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerActionsTestOtherA-1303549305',owner_user_name='tempest-ServerActionsTestOtherA-1303549305-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:22:22Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='5dca34fdadbb4386a1ff1cb255265d47',uuid=585c81a5-ac24-4d12-a8a6-1e7b12a2d53a,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "e521fae4-093d-4c92-a32d-684c1a5137f5", "address": "fa:16:3e:e5:b0:10", "network": {"id": "b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-210689817-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "69d584c230b94ff8aae722eb2f364c5c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape521fae4-09", "ovs_interfaceid": "e521fae4-093d-4c92-a32d-684c1a5137f5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:22:25 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Converting VIF {"id": "e521fae4-093d-4c92-a32d-684c1a5137f5", "address": "fa:16:3e:e5:b0:10", "network": {"id": "b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-210689817-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "69d584c230b94ff8aae722eb2f364c5c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape521fae4-09", "ovs_interfaceid": "e521fae4-093d-4c92-a32d-684c1a5137f5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:22:25 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:e5:b0:10,bridge_name='br-int',has_traffic_filtering=True,id=e521fae4-093d-4c92-a32d-684c1a5137f5,network=Network(b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape521fae4-09') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:22:25 user nova-compute[72351]: DEBUG os_vif [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:e5:b0:10,bridge_name='br-int',has_traffic_filtering=True,id=e521fae4-093d-4c92-a32d-684c1a5137f5,network=Network(b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape521fae4-09') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:22:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:22:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:22:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tape521fae4-09, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:22:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tape521fae4-09, col_values=(('external_ids', {'iface-id': 'e521fae4-093d-4c92-a32d-684c1a5137f5', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:e5:b0:10', 'vm-uuid': '585c81a5-ac24-4d12-a8a6-1e7b12a2d53a'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:22:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:22:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:25 user nova-compute[72351]: INFO os_vif [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:e5:b0:10,bridge_name='br-int',has_traffic_filtering=True,id=e521fae4-093d-4c92-a32d-684c1a5137f5,network=Network(b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape521fae4-09')
Aug 09 02:22:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:25 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:22:25 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] No VIF found with MAC fa:16:3e:e5:b0:10, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:22:25 user nova-compute[72351]: DEBUG nova.compute.manager [req-3193d2f1-c348-4ab7-b7d0-4f0795fdb0fc req-35d52e8f-2d03-448a-87f7-960f22e373ef service nova] [instance: a1d1f738-0edf-4184-aa56-020c07830492] Received event network-vif-unplugged-3e179747-c782-4b72-90d0-5b3bb68c3101 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:22:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-3193d2f1-c348-4ab7-b7d0-4f0795fdb0fc req-35d52e8f-2d03-448a-87f7-960f22e373ef service nova] Acquiring lock "a1d1f738-0edf-4184-aa56-020c07830492-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:22:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-3193d2f1-c348-4ab7-b7d0-4f0795fdb0fc req-35d52e8f-2d03-448a-87f7-960f22e373ef service nova] Lock "a1d1f738-0edf-4184-aa56-020c07830492-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:22:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-3193d2f1-c348-4ab7-b7d0-4f0795fdb0fc req-35d52e8f-2d03-448a-87f7-960f22e373ef service nova] Lock "a1d1f738-0edf-4184-aa56-020c07830492-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:22:25 user nova-compute[72351]: DEBUG nova.compute.manager [req-3193d2f1-c348-4ab7-b7d0-4f0795fdb0fc req-35d52e8f-2d03-448a-87f7-960f22e373ef service nova] [instance: a1d1f738-0edf-4184-aa56-020c07830492] No waiting events found dispatching network-vif-unplugged-3e179747-c782-4b72-90d0-5b3bb68c3101 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:22:25 user nova-compute[72351]: DEBUG nova.compute.manager [req-3193d2f1-c348-4ab7-b7d0-4f0795fdb0fc req-35d52e8f-2d03-448a-87f7-960f22e373ef service nova] [instance: a1d1f738-0edf-4184-aa56-020c07830492] Received event network-vif-unplugged-3e179747-c782-4b72-90d0-5b3bb68c3101 for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:22:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:25 user nova-compute[72351]: DEBUG nova.network.neutron [req-eb69b3a7-21d8-469a-bc8d-64d36f4b5111 req-a0992392-f4ad-463c-815b-1d72d70f6457 service nova] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Updated VIF entry in instance network info cache for port e521fae4-093d-4c92-a32d-684c1a5137f5. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:22:25 user nova-compute[72351]: DEBUG nova.network.neutron [req-eb69b3a7-21d8-469a-bc8d-64d36f4b5111 req-a0992392-f4ad-463c-815b-1d72d70f6457 service nova] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Updating instance_info_cache with network_info: [{"id": "e521fae4-093d-4c92-a32d-684c1a5137f5", "address": "fa:16:3e:e5:b0:10", "network": {"id": "b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-210689817-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "69d584c230b94ff8aae722eb2f364c5c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape521fae4-09", "ovs_interfaceid": "e521fae4-093d-4c92-a32d-684c1a5137f5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:22:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-eb69b3a7-21d8-469a-bc8d-64d36f4b5111 req-a0992392-f4ad-463c-815b-1d72d70f6457 service nova] Releasing lock "refresh_cache-585c81a5-ac24-4d12-a8a6-1e7b12a2d53a" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:22:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:25 user nova-compute[72351]: DEBUG nova.compute.manager [req-a2b7ba4c-8430-4f6f-be9e-c8ea3d617700 req-93e2d23a-cd1b-4b66-bca2-86d4975f5f93 service nova] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Received event network-changed-74c98543-ce04-43d8-a58c-267e372db698 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:22:25 user nova-compute[72351]: DEBUG nova.compute.manager [req-a2b7ba4c-8430-4f6f-be9e-c8ea3d617700 req-93e2d23a-cd1b-4b66-bca2-86d4975f5f93 service nova] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Refreshing instance network info cache due to event network-changed-74c98543-ce04-43d8-a58c-267e372db698. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:22:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a2b7ba4c-8430-4f6f-be9e-c8ea3d617700 req-93e2d23a-cd1b-4b66-bca2-86d4975f5f93 service nova] Acquiring lock "refresh_cache-97b13ccd-15b4-44bf-b865-71b4da861168" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:22:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a2b7ba4c-8430-4f6f-be9e-c8ea3d617700 req-93e2d23a-cd1b-4b66-bca2-86d4975f5f93 service nova] Acquired lock "refresh_cache-97b13ccd-15b4-44bf-b865-71b4da861168" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:22:25 user nova-compute[72351]: DEBUG nova.network.neutron [req-a2b7ba4c-8430-4f6f-be9e-c8ea3d617700 req-93e2d23a-cd1b-4b66-bca2-86d4975f5f93 service nova] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Refreshing network info cache for port 74c98543-ce04-43d8-a58c-267e372db698 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:22:26 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:26 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:26 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: a1d1f738-0edf-4184-aa56-020c07830492] Instance destroyed successfully.
Aug 09 02:22:26 user nova-compute[72351]: DEBUG nova.objects.instance [None req-6d6e4099-6c7b-4836-9cb6-34337a126618 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Lazy-loading 'resources' on Instance uuid a1d1f738-0edf-4184-aa56-020c07830492 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:22:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-6d6e4099-6c7b-4836-9cb6-34337a126618 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:20:23Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-VolumesAdminNegativeTest-server-1398011561',display_name='tempest-VolumesAdminNegativeTest-server-1398011561',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-volumesadminnegativetest-server-1398011561',id=13,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:20:32Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='bc5e05fbd7f44f85bc3b1cc169dd03a9',ramdisk_id='',reservation_id='r-vh2c5s83',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-VolumesAdminNegativeTest-35684003',owner_user_name='tempest-VolumesAdminNegativeTest-35684003-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:20:32Z,user_data=None,user_id='fb9f73926e0d42f586e7c0055764d611',uuid=a1d1f738-0edf-4184-aa56-020c07830492,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "3e179747-c782-4b72-90d0-5b3bb68c3101", "address": "fa:16:3e:54:6f:e3", "network": {"id": "21efbfa4-f3a2-4317-89fc-3cc7b60fde97", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1919568535-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bc5e05fbd7f44f85bc3b1cc169dd03a9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3e179747-c7", "ovs_interfaceid": "3e179747-c782-4b72-90d0-5b3bb68c3101", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:22:26 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-6d6e4099-6c7b-4836-9cb6-34337a126618 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Converting VIF {"id": "3e179747-c782-4b72-90d0-5b3bb68c3101", "address": "fa:16:3e:54:6f:e3", "network": {"id": "21efbfa4-f3a2-4317-89fc-3cc7b60fde97", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1919568535-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bc5e05fbd7f44f85bc3b1cc169dd03a9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3e179747-c7", "ovs_interfaceid": "3e179747-c782-4b72-90d0-5b3bb68c3101", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:22:26 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-6d6e4099-6c7b-4836-9cb6-34337a126618 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:54:6f:e3,bridge_name='br-int',has_traffic_filtering=True,id=3e179747-c782-4b72-90d0-5b3bb68c3101,network=Network(21efbfa4-f3a2-4317-89fc-3cc7b60fde97),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3e179747-c7') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:22:26 user nova-compute[72351]: DEBUG os_vif [None req-6d6e4099-6c7b-4836-9cb6-34337a126618 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:54:6f:e3,bridge_name='br-int',has_traffic_filtering=True,id=3e179747-c782-4b72-90d0-5b3bb68c3101,network=Network(21efbfa4-f3a2-4317-89fc-3cc7b60fde97),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3e179747-c7') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:22:26 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:26 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap3e179747-c7, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:22:26 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:26 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:22:26 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:26 user nova-compute[72351]: INFO os_vif [None req-6d6e4099-6c7b-4836-9cb6-34337a126618 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:54:6f:e3,bridge_name='br-int',has_traffic_filtering=True,id=3e179747-c782-4b72-90d0-5b3bb68c3101,network=Network(21efbfa4-f3a2-4317-89fc-3cc7b60fde97),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3e179747-c7')
Aug 09 02:22:26 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-6d6e4099-6c7b-4836-9cb6-34337a126618 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: a1d1f738-0edf-4184-aa56-020c07830492] Deleting instance files /opt/stack/data/nova/instances/a1d1f738-0edf-4184-aa56-020c07830492_del
Aug 09 02:22:26 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-6d6e4099-6c7b-4836-9cb6-34337a126618 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: a1d1f738-0edf-4184-aa56-020c07830492] Deletion of /opt/stack/data/nova/instances/a1d1f738-0edf-4184-aa56-020c07830492_del complete
Aug 09 02:22:26 user nova-compute[72351]: INFO nova.compute.manager [None req-6d6e4099-6c7b-4836-9cb6-34337a126618 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: a1d1f738-0edf-4184-aa56-020c07830492] Took 1.35 seconds to destroy the instance on the hypervisor.
Aug 09 02:22:26 user nova-compute[72351]: DEBUG oslo.service.loopingcall [None req-6d6e4099-6c7b-4836-9cb6-34337a126618 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72351) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Aug 09 02:22:26 user nova-compute[72351]: DEBUG nova.compute.manager [-] [instance: a1d1f738-0edf-4184-aa56-020c07830492] Deallocating network for instance {{(pid=72351) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2263}}
Aug 09 02:22:26 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: a1d1f738-0edf-4184-aa56-020c07830492] deallocate_for_instance() {{(pid=72351) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1795}}
Aug 09 02:22:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e32c42fd-1856-4aee-996f-0ac20a7aaa2a tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Acquiring lock "97b13ccd-15b4-44bf-b865-71b4da861168" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:22:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e32c42fd-1856-4aee-996f-0ac20a7aaa2a tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "97b13ccd-15b4-44bf-b865-71b4da861168" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:22:26 user nova-compute[72351]: INFO nova.compute.manager [None req-e32c42fd-1856-4aee-996f-0ac20a7aaa2a tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Detaching volume cda288d2-cdeb-4b8f-abbf-acaa3fab484f
Aug 09 02:22:26 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:26 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:26 user nova-compute[72351]: INFO nova.virt.block_device [None req-e32c42fd-1856-4aee-996f-0ac20a7aaa2a tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Attempting to driver detach volume cda288d2-cdeb-4b8f-abbf-acaa3fab484f from mountpoint /dev/vdb
Aug 09 02:22:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-e32c42fd-1856-4aee-996f-0ac20a7aaa2a tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Attempting to detach device vdb from instance 97b13ccd-15b4-44bf-b865-71b4da861168 from the persistent domain config. {{(pid=72351) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2477}}
Aug 09 02:22:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-e32c42fd-1856-4aee-996f-0ac20a7aaa2a tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 02:22:26 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:22:26 user nova-compute[72351]:   <source dev="/dev/sdb"/>
Aug 09 02:22:26 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:22:26 user nova-compute[72351]:   <serial>cda288d2-cdeb-4b8f-abbf-acaa3fab484f</serial>
Aug 09 02:22:26 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x05" function="0x0"/>
Aug 09 02:22:26 user nova-compute[72351]: </disk>
Aug 09 02:22:26 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 02:22:26 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-e32c42fd-1856-4aee-996f-0ac20a7aaa2a tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Successfully detached device vdb from instance 97b13ccd-15b4-44bf-b865-71b4da861168 from the persistent domain config.
Aug 09 02:22:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-e32c42fd-1856-4aee-996f-0ac20a7aaa2a tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 97b13ccd-15b4-44bf-b865-71b4da861168 from the live domain config. {{(pid=72351) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2513}}
Aug 09 02:22:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-e32c42fd-1856-4aee-996f-0ac20a7aaa2a tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 02:22:26 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:22:26 user nova-compute[72351]:   <source dev="/dev/sdb"/>
Aug 09 02:22:26 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:22:26 user nova-compute[72351]:   <serial>cda288d2-cdeb-4b8f-abbf-acaa3fab484f</serial>
Aug 09 02:22:26 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x05" function="0x0"/>
Aug 09 02:22:26 user nova-compute[72351]: </disk>
Aug 09 02:22:26 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 02:22:26 user nova-compute[72351]: DEBUG nova.compute.manager [req-1dea3d49-8cb0-4913-8f44-d876288eddce req-dcd87ce9-ff21-46f2-adc1-8cdc3e014cb8 service nova] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Received event network-changed-74c98543-ce04-43d8-a58c-267e372db698 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:22:26 user nova-compute[72351]: DEBUG nova.compute.manager [req-1dea3d49-8cb0-4913-8f44-d876288eddce req-dcd87ce9-ff21-46f2-adc1-8cdc3e014cb8 service nova] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Refreshing instance network info cache due to event network-changed-74c98543-ce04-43d8-a58c-267e372db698. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:22:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-1dea3d49-8cb0-4913-8f44-d876288eddce req-dcd87ce9-ff21-46f2-adc1-8cdc3e014cb8 service nova] Acquiring lock "refresh_cache-97b13ccd-15b4-44bf-b865-71b4da861168" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:22:27 user nova-compute[72351]: DEBUG nova.network.neutron [req-a2b7ba4c-8430-4f6f-be9e-c8ea3d617700 req-93e2d23a-cd1b-4b66-bca2-86d4975f5f93 service nova] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Updated VIF entry in instance network info cache for port 74c98543-ce04-43d8-a58c-267e372db698. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:22:27 user nova-compute[72351]: DEBUG nova.network.neutron [req-a2b7ba4c-8430-4f6f-be9e-c8ea3d617700 req-93e2d23a-cd1b-4b66-bca2-86d4975f5f93 service nova] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Updating instance_info_cache with network_info: [{"id": "74c98543-ce04-43d8-a58c-267e372db698", "address": "fa:16:3e:d4:4d:0a", "network": {"id": "7024d1f6-a400-4482-9364-78ae3b6adc80", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1603258958-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c2f67da658d94fabbab82ee6a6171c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap74c98543-ce", "ovs_interfaceid": "74c98543-ce04-43d8-a58c-267e372db698", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:22:27 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a2b7ba4c-8430-4f6f-be9e-c8ea3d617700 req-93e2d23a-cd1b-4b66-bca2-86d4975f5f93 service nova] Releasing lock "refresh_cache-97b13ccd-15b4-44bf-b865-71b4da861168" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:22:27 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-1dea3d49-8cb0-4913-8f44-d876288eddce req-dcd87ce9-ff21-46f2-adc1-8cdc3e014cb8 service nova] Acquired lock "refresh_cache-97b13ccd-15b4-44bf-b865-71b4da861168" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:22:27 user nova-compute[72351]: DEBUG nova.network.neutron [req-1dea3d49-8cb0-4913-8f44-d876288eddce req-dcd87ce9-ff21-46f2-adc1-8cdc3e014cb8 service nova] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Refreshing network info cache for port 74c98543-ce04-43d8-a58c-267e372db698 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:22:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:27 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Received event <DeviceRemovedEvent: 1691547747.4169497, 97b13ccd-15b4-44bf-b865-71b4da861168 => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=72351) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2360}}
Aug 09 02:22:27 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-e32c42fd-1856-4aee-996f-0ac20a7aaa2a tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 97b13ccd-15b4-44bf-b865-71b4da861168 {{(pid=72351) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2589}}
Aug 09 02:22:27 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-e32c42fd-1856-4aee-996f-0ac20a7aaa2a tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Successfully detached device vdb from instance 97b13ccd-15b4-44bf-b865-71b4da861168 from the live domain config.
Aug 09 02:22:27 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: a1d1f738-0edf-4184-aa56-020c07830492] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:22:27 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: a1d1f738-0edf-4184-aa56-020c07830492] Took 1.35 seconds to deallocate network for instance.
Aug 09 02:22:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:27 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6d6e4099-6c7b-4836-9cb6-34337a126618 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:22:27 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6d6e4099-6c7b-4836-9cb6-34337a126618 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:22:27 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-e32c42fd-1856-4aee-996f-0ac20a7aaa2a tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 02:22:27 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e32c42fd-1856-4aee-996f-0ac20a7aaa2a tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'cIMBpXdn', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb'}, None), 'kwargs': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:22:27 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-e32c42fd-1856-4aee-996f-0ac20a7aaa2a tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:22:27 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-e32c42fd-1856-4aee-996f-0ac20a7aaa2a tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:22:27 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-e32c42fd-1856-4aee-996f-0ac20a7aaa2a tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:22:27 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e32c42fd-1856-4aee-996f-0ac20a7aaa2a tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 02:22:27 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.032s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[39a79d66-4305-4395-8121-5a36f742fdd7]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.029s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[344660b5-ecda-4054-b417-a65359f9625e]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f (non-flash)\ntcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497 (non-flash)\ntcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e32c42fd-1856-4aee-996f-0ac20a7aaa2a tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f (non-flash)
Aug 09 02:22:28 user nova-compute[72351]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497 (non-flash)
Aug 09 02:22:28 user nova-compute[72351]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)
Aug 09 02:22:28 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e32c42fd-1856-4aee-996f-0ac20a7aaa2a tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f (non-flash)
Aug 09 02:22:28 user nova-compute[72351]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497 (non-flash)
Aug 09 02:22:28 user nova-compute[72351]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)
Aug 09 02:22:28 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e32c42fd-1856-4aee-996f-0ac20a7aaa2a tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fcfbc7452d0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f'): ({'sdb'}, set())}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-e32c42fd-1856-4aee-996f-0ac20a7aaa2a tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Removing single pathed devices sdb {{(pid=72351) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG nova.compute.manager [req-189fe290-3511-4efd-9269-5beed58544f0 req-42c59b51-555a-4267-9ffb-4ebab3888f05 service nova] [instance: a1d1f738-0edf-4184-aa56-020c07830492] Received event network-vif-plugged-3e179747-c782-4b72-90d0-5b3bb68c3101 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-189fe290-3511-4efd-9269-5beed58544f0 req-42c59b51-555a-4267-9ffb-4ebab3888f05 service nova] Acquiring lock "a1d1f738-0edf-4184-aa56-020c07830492-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-189fe290-3511-4efd-9269-5beed58544f0 req-42c59b51-555a-4267-9ffb-4ebab3888f05 service nova] Lock "a1d1f738-0edf-4184-aa56-020c07830492-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-189fe290-3511-4efd-9269-5beed58544f0 req-42c59b51-555a-4267-9ffb-4ebab3888f05 service nova] Lock "a1d1f738-0edf-4184-aa56-020c07830492-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG nova.compute.manager [req-189fe290-3511-4efd-9269-5beed58544f0 req-42c59b51-555a-4267-9ffb-4ebab3888f05 service nova] [instance: a1d1f738-0edf-4184-aa56-020c07830492] No waiting events found dispatching network-vif-plugged-3e179747-c782-4b72-90d0-5b3bb68c3101 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:22:28 user nova-compute[72351]: WARNING nova.compute.manager [req-189fe290-3511-4efd-9269-5beed58544f0 req-42c59b51-555a-4267-9ffb-4ebab3888f05 service nova] [instance: a1d1f738-0edf-4184-aa56-020c07830492] Received unexpected event network-vif-plugged-3e179747-c782-4b72-90d0-5b3bb68c3101 for instance with vm_state deleted and task_state None.
Aug 09 02:22:28 user nova-compute[72351]: DEBUG nova.compute.manager [req-189fe290-3511-4efd-9269-5beed58544f0 req-42c59b51-555a-4267-9ffb-4ebab3888f05 service nova] [instance: a1d1f738-0edf-4184-aa56-020c07830492] Received event network-vif-plugged-3e179747-c782-4b72-90d0-5b3bb68c3101 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-189fe290-3511-4efd-9269-5beed58544f0 req-42c59b51-555a-4267-9ffb-4ebab3888f05 service nova] Acquiring lock "a1d1f738-0edf-4184-aa56-020c07830492-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-189fe290-3511-4efd-9269-5beed58544f0 req-42c59b51-555a-4267-9ffb-4ebab3888f05 service nova] Lock "a1d1f738-0edf-4184-aa56-020c07830492-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-189fe290-3511-4efd-9269-5beed58544f0 req-42c59b51-555a-4267-9ffb-4ebab3888f05 service nova] Lock "a1d1f738-0edf-4184-aa56-020c07830492-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG nova.compute.manager [req-189fe290-3511-4efd-9269-5beed58544f0 req-42c59b51-555a-4267-9ffb-4ebab3888f05 service nova] [instance: a1d1f738-0edf-4184-aa56-020c07830492] No waiting events found dispatching network-vif-plugged-3e179747-c782-4b72-90d0-5b3bb68c3101 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:22:28 user nova-compute[72351]: WARNING nova.compute.manager [req-189fe290-3511-4efd-9269-5beed58544f0 req-42c59b51-555a-4267-9ffb-4ebab3888f05 service nova] [instance: a1d1f738-0edf-4184-aa56-020c07830492] Received unexpected event network-vif-plugged-3e179747-c782-4b72-90d0-5b3bb68c3101 for instance with vm_state deleted and task_state None.
Aug 09 02:22:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.028s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[8345eb33-7ac5-4db2-8c3b-709f8187ba16]: (4, ('path checker states:\nup                  3\n\npaths: 0\nbusy: False\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG nova.network.neutron [req-1dea3d49-8cb0-4913-8f44-d876288eddce req-dcd87ce9-ff21-46f2-adc1-8cdc3e014cb8 service nova] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Updated VIF entry in instance network info cache for port 74c98543-ce04-43d8-a58c-267e372db698. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG nova.network.neutron [req-1dea3d49-8cb0-4913-8f44-d876288eddce req-dcd87ce9-ff21-46f2-adc1-8cdc3e014cb8 service nova] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Updating instance_info_cache with network_info: [{"id": "74c98543-ce04-43d8-a58c-267e372db698", "address": "fa:16:3e:d4:4d:0a", "network": {"id": "7024d1f6-a400-4482-9364-78ae3b6adc80", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1603258958-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c2f67da658d94fabbab82ee6a6171c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap74c98543-ce", "ovs_interfaceid": "74c98543-ce04-43d8-a58c-267e372db698", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.025s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[c10d6415-a568-4818-91d5-07138c20fe4a]: (4, ('ok\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-e32c42fd-1856-4aee-996f-0ac20a7aaa2a tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Flushing IO for device /dev/sdb {{(pid=72351) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-1dea3d49-8cb0-4913-8f44-d876288eddce req-dcd87ce9-ff21-46f2-adc1-8cdc3e014cb8 service nova] Releasing lock "refresh_cache-97b13ccd-15b4-44bf-b865-71b4da861168" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[20121fb8-8827-45d3-ae4a-789371af6d56]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-e32c42fd-1856-4aee-996f-0ac20a7aaa2a tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=72351) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.175s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[16db6256-186f-4537-a521-fd0f638b6f55]: (4, ('1', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-e32c42fd-1856-4aee-996f-0ac20a7aaa2a tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-e32c42fd-1856-4aee-996f-0ac20a7aaa2a tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] SCSI volumes sdb have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e32c42fd-1856-4aee-996f-0ac20a7aaa2a tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f')] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.086s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[677b37b2-7829-49dc-b07f-8e75cb517628]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e32c42fd-1856-4aee-996f-0ac20a7aaa2a tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f -p 172.16.0.220:3260 --logout {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f -p 172.16.0.220:3260 --logout" returned: 0 in 0.092s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a95aed10-d220-4683-aef0-3e286904a181]: (4, ('Logging out of session [sid: 14, target: iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f, portal: 172.16.0.220,3260]\nLogout of [sid: 14, target: iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e32c42fd-1856-4aee-996f-0ac20a7aaa2a tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 14, target: iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f, portal: 172.16.0.220,3260]
Aug 09 02:22:28 user nova-compute[72351]: Logout of [sid: 14, target: iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f, portal: 172.16.0.220,3260] successful.
Aug 09 02:22:28 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f -p 172.16.0.220:3260 --op delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cda288d2-cdeb-4b8f-abbf-acaa3fab484f -p 172.16.0.220:3260 --op delete" returned: 0 in 0.151s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[05359c37-490d-4cdf-89e6-15fff708c03d]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e32c42fd-1856-4aee-996f-0ac20a7aaa2a tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-e32c42fd-1856-4aee-996f-0ac20a7aaa2a tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.878s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e32c42fd-1856-4aee-996f-0ac20a7aaa2a tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] <== disconnect_volume: return (880ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-e32c42fd-1856-4aee-996f-0ac20a7aaa2a tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-6d6e4099-6c7b-4836-9cb6-34337a126618 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-6d6e4099-6c7b-4836-9cb6-34337a126618 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG nova.compute.manager [req-c39563be-86cb-4307-968c-cb2bacbdb669 req-ca920515-0077-4674-a912-8d4dbe4d1d88 service nova] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Received event network-vif-plugged-e521fae4-093d-4c92-a32d-684c1a5137f5 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c39563be-86cb-4307-968c-cb2bacbdb669 req-ca920515-0077-4674-a912-8d4dbe4d1d88 service nova] Acquiring lock "585c81a5-ac24-4d12-a8a6-1e7b12a2d53a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c39563be-86cb-4307-968c-cb2bacbdb669 req-ca920515-0077-4674-a912-8d4dbe4d1d88 service nova] Lock "585c81a5-ac24-4d12-a8a6-1e7b12a2d53a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c39563be-86cb-4307-968c-cb2bacbdb669 req-ca920515-0077-4674-a912-8d4dbe4d1d88 service nova] Lock "585c81a5-ac24-4d12-a8a6-1e7b12a2d53a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG nova.compute.manager [req-c39563be-86cb-4307-968c-cb2bacbdb669 req-ca920515-0077-4674-a912-8d4dbe4d1d88 service nova] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] No waiting events found dispatching network-vif-plugged-e521fae4-093d-4c92-a32d-684c1a5137f5 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:22:28 user nova-compute[72351]: WARNING nova.compute.manager [req-c39563be-86cb-4307-968c-cb2bacbdb669 req-ca920515-0077-4674-a912-8d4dbe4d1d88 service nova] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Received unexpected event network-vif-plugged-e521fae4-093d-4c92-a32d-684c1a5137f5 for instance with vm_state building and task_state spawning.
Aug 09 02:22:28 user nova-compute[72351]: DEBUG nova.compute.manager [req-c39563be-86cb-4307-968c-cb2bacbdb669 req-ca920515-0077-4674-a912-8d4dbe4d1d88 service nova] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Received event network-vif-plugged-e521fae4-093d-4c92-a32d-684c1a5137f5 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c39563be-86cb-4307-968c-cb2bacbdb669 req-ca920515-0077-4674-a912-8d4dbe4d1d88 service nova] Acquiring lock "585c81a5-ac24-4d12-a8a6-1e7b12a2d53a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c39563be-86cb-4307-968c-cb2bacbdb669 req-ca920515-0077-4674-a912-8d4dbe4d1d88 service nova] Lock "585c81a5-ac24-4d12-a8a6-1e7b12a2d53a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c39563be-86cb-4307-968c-cb2bacbdb669 req-ca920515-0077-4674-a912-8d4dbe4d1d88 service nova] Lock "585c81a5-ac24-4d12-a8a6-1e7b12a2d53a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:22:28 user nova-compute[72351]: DEBUG nova.compute.manager [req-c39563be-86cb-4307-968c-cb2bacbdb669 req-ca920515-0077-4674-a912-8d4dbe4d1d88 service nova] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] No waiting events found dispatching network-vif-plugged-e521fae4-093d-4c92-a32d-684c1a5137f5 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:22:28 user nova-compute[72351]: WARNING nova.compute.manager [req-c39563be-86cb-4307-968c-cb2bacbdb669 req-ca920515-0077-4674-a912-8d4dbe4d1d88 service nova] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Received unexpected event network-vif-plugged-e521fae4-093d-4c92-a32d-684c1a5137f5 for instance with vm_state building and task_state spawning.
Aug 09 02:22:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6d6e4099-6c7b-4836-9cb6-34337a126618 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 1.020s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:22:28 user nova-compute[72351]: INFO nova.scheduler.client.report [None req-6d6e4099-6c7b-4836-9cb6-34337a126618 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Deleted allocations for instance a1d1f738-0edf-4184-aa56-020c07830492
Aug 09 02:22:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6d6e4099-6c7b-4836-9cb6-34337a126618 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Lock "a1d1f738-0edf-4184-aa56-020c07830492" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 3.980s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:22:29 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691547734.125553, 822e25c4-8dd1-4e21-b0ed-f340d59c6c98 => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:22:29 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] VM Stopped (Lifecycle Event)
Aug 09 02:22:29 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:29 user nova-compute[72351]: DEBUG nova.compute.manager [None req-81894209-84c9-4b2f-b869-051730105893 None None] [instance: 822e25c4-8dd1-4e21-b0ed-f340d59c6c98] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:22:29 user nova-compute[72351]: DEBUG nova.objects.instance [None req-e32c42fd-1856-4aee-996f-0ac20a7aaa2a tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lazy-loading 'flavor' on Instance uuid 97b13ccd-15b4-44bf-b865-71b4da861168 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:22:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e32c42fd-1856-4aee-996f-0ac20a7aaa2a tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "97b13ccd-15b4-44bf-b865-71b4da861168" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 3.217s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:22:29 user nova-compute[72351]: DEBUG nova.compute.manager [req-af44f598-a4d5-4b94-afb0-97c8e0782311 req-94d4950b-d7d9-4814-aa81-c8804822d475 service nova] [instance: a1d1f738-0edf-4184-aa56-020c07830492] Received event network-vif-deleted-3e179747-c782-4b72-90d0-5b3bb68c3101 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:22:30 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547750.3958976, 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:22:30 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] VM Resumed (Lifecycle Event)
Aug 09 02:22:30 user nova-compute[72351]: DEBUG nova.compute.manager [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:22:30 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Guest created on hypervisor {{(pid=72351) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4411}}
Aug 09 02:22:30 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Instance spawned successfully.
Aug 09 02:22:30 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:894}}
Aug 09 02:22:30 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:22:30 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:22:30 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Found default for hw_cdrom_bus of ide {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:22:30 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Found default for hw_disk_bus of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:22:30 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Found default for hw_input_bus of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:22:30 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Found default for hw_pointer_model of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:22:30 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Found default for hw_video_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:22:30 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Found default for hw_vif_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:22:30 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:22:30 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547750.3961935, 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:22:30 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] VM Started (Lifecycle Event)
Aug 09 02:22:30 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:22:30 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:22:30 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:22:30 user nova-compute[72351]: INFO nova.compute.manager [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Took 8.28 seconds to spawn the instance on the hypervisor.
Aug 09 02:22:30 user nova-compute[72351]: DEBUG nova.compute.manager [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:22:30 user nova-compute[72351]: INFO nova.compute.manager [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Took 9.20 seconds to build instance.
Aug 09 02:22:30 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-50954eee-69cb-482d-80bb-2962654946bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "585c81a5-ac24-4d12-a8a6-1e7b12a2d53a" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 9.308s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:22:31 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:32 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:32 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:34 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Acquiring lock "a7f3d076-864b-4b68-9207-214ce2c21444" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:22:34 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "a7f3d076-864b-4b68-9207-214ce2c21444" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:22:34 user nova-compute[72351]: DEBUG nova.compute.manager [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Starting instance... {{(pid=72351) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2406}}
Aug 09 02:22:34 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:22:34 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:22:34 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72351) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Aug 09 02:22:34 user nova-compute[72351]: INFO nova.compute.claims [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Claim successful on node user
Aug 09 02:22:34 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691547739.3553739, 32a28ecb-af4b-4c6e-8f2d-a51857134b34 => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:22:34 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] VM Stopped (Lifecycle Event)
Aug 09 02:22:34 user nova-compute[72351]: DEBUG nova.compute.manager [None req-341985b9-6a2d-4168-866f-defb3347c835 None None] [instance: 32a28ecb-af4b-4c6e-8f2d-a51857134b34] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:22:34 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:22:34 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:22:34 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.442s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:22:34 user nova-compute[72351]: DEBUG nova.compute.manager [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Start building networks asynchronously for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2803}}
Aug 09 02:22:34 user nova-compute[72351]: DEBUG nova.compute.manager [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Allocating IP information in the background. {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1956}}
Aug 09 02:22:34 user nova-compute[72351]: DEBUG nova.network.neutron [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] allocate_for_instance() {{(pid=72351) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Aug 09 02:22:34 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Aug 09 02:22:34 user nova-compute[72351]: DEBUG nova.compute.manager [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Start building block device mappings for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2838}}
Aug 09 02:22:34 user nova-compute[72351]: DEBUG nova.policy [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '73d05c48be5640c28455e8ba8b05ef31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '321111f935dd4e8aab4a083206ea9a7e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72351) authorize /opt/stack/nova/nova/policy.py:203}}
Aug 09 02:22:34 user nova-compute[72351]: DEBUG nova.compute.manager [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Start spawning the instance on the hypervisor. {{(pid=72351) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2612}}
Aug 09 02:22:34 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Creating instance directory {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4717}}
Aug 09 02:22:34 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Creating image(s)
Aug 09 02:22:34 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Acquiring lock "/opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:22:34 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "/opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:22:34 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "/opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.003s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:22:34 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:22:35 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.139s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:22:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Acquiring lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:22:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:22:35 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:22:35 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.162s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:22:35 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk 1073741824 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:22:35 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk 1073741824" returned: 0 in 0.050s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:22:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.219s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:22:35 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:22:35 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.135s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:22:35 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Checking if we can resize image /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk. size=1073741824 {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Aug 09 02:22:35 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:22:35 user nova-compute[72351]: DEBUG nova.network.neutron [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Successfully created port: 64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f {{(pid=72351) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Aug 09 02:22:35 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json" returned: 0 in 0.162s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:22:35 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Cannot resize image /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk to a smaller size. {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Aug 09 02:22:35 user nova-compute[72351]: DEBUG nova.objects.instance [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lazy-loading 'migration_context' on Instance uuid a7f3d076-864b-4b68-9207-214ce2c21444 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:22:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Created local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4851}}
Aug 09 02:22:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Ensure instance console log exists: /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/console.log {{(pid=72351) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4603}}
Aug 09 02:22:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:22:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:22:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:22:36 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:36 user nova-compute[72351]: DEBUG nova.network.neutron [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Successfully updated port: 64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:22:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Acquiring lock "refresh_cache-a7f3d076-864b-4b68-9207-214ce2c21444" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:22:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Acquired lock "refresh_cache-a7f3d076-864b-4b68-9207-214ce2c21444" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:22:36 user nova-compute[72351]: DEBUG nova.network.neutron [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:22:36 user nova-compute[72351]: DEBUG nova.compute.manager [req-7d61bcd5-ac69-4283-8fef-878c764cb654 req-2e9b0d08-6d29-4cf0-a51b-ed70428ccf2e service nova] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Received event network-changed-64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:22:36 user nova-compute[72351]: DEBUG nova.compute.manager [req-7d61bcd5-ac69-4283-8fef-878c764cb654 req-2e9b0d08-6d29-4cf0-a51b-ed70428ccf2e service nova] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Refreshing instance network info cache due to event network-changed-64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:22:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-7d61bcd5-ac69-4283-8fef-878c764cb654 req-2e9b0d08-6d29-4cf0-a51b-ed70428ccf2e service nova] Acquiring lock "refresh_cache-a7f3d076-864b-4b68-9207-214ce2c21444" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:22:36 user nova-compute[72351]: DEBUG nova.network.neutron [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:22:36 user nova-compute[72351]: DEBUG nova.network.neutron [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Updating instance_info_cache with network_info: [{"id": "64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f", "address": "fa:16:3e:ed:9c:88", "network": {"id": "9a452414-d61f-4045-a6cc-9be93c892c4c", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "321111f935dd4e8aab4a083206ea9a7e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap64bd80e2-6c", "ovs_interfaceid": "64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:22:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Releasing lock "refresh_cache-a7f3d076-864b-4b68-9207-214ce2c21444" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:22:36 user nova-compute[72351]: DEBUG nova.compute.manager [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Instance network_info: |[{"id": "64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f", "address": "fa:16:3e:ed:9c:88", "network": {"id": "9a452414-d61f-4045-a6cc-9be93c892c4c", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "321111f935dd4e8aab4a083206ea9a7e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap64bd80e2-6c", "ovs_interfaceid": "64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1971}}
Aug 09 02:22:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-7d61bcd5-ac69-4283-8fef-878c764cb654 req-2e9b0d08-6d29-4cf0-a51b-ed70428ccf2e service nova] Acquired lock "refresh_cache-a7f3d076-864b-4b68-9207-214ce2c21444" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:22:36 user nova-compute[72351]: DEBUG nova.network.neutron [req-7d61bcd5-ac69-4283-8fef-878c764cb654 req-2e9b0d08-6d29-4cf0-a51b-ed70428ccf2e service nova] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Refreshing network info cache for port 64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:22:36 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Start _get_guest_xml network_info=[{"id": "64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f", "address": "fa:16:3e:ed:9c:88", "network": {"id": "9a452414-d61f-4045-a6cc-9be93c892c4c", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "321111f935dd4e8aab4a083206ea9a7e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap64bd80e2-6c", "ovs_interfaceid": "64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'size': 0, 'device_type': 'disk', 'encryption_secret_uuid': None, 'guest_format': None, 'encryption_options': None, 'disk_bus': 'virtio', 'encrypted': False, 'encryption_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'image_id': '18ccda7f-31d3-4978-ab9d-4ed2f74eff94'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:22:36 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:22:36 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:22:36 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:22:36 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:22:36 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:22:36 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:22:36 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:22:36 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:22:36 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:22:36 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:22:36 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:22:36 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:22:36 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:22:36 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:22:36 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:22:36 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:22:36 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:22:36 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:22:36 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:22:33Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-578493690',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-578493690',id=20,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='321111f935dd4e8aab4a083206ea9a7e',ramdisk_id='',reservation_id='r-ovw0qy0n',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-2108329302',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:22:35Z,user_data=None,user_id='73d05c48be5640c28455e8ba8b05ef31',uuid=a7f3d076-864b-4b68-9207-214ce2c21444,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f", "address": "fa:16:3e:ed:9c:88", "network": {"id": "9a452414-d61f-4045-a6cc-9be93c892c4c", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "321111f935dd4e8aab4a083206ea9a7e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap64bd80e2-6c", "ovs_interfaceid": "64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:22:36 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Converting VIF {"id": "64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f", "address": "fa:16:3e:ed:9c:88", "network": {"id": "9a452414-d61f-4045-a6cc-9be93c892c4c", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "321111f935dd4e8aab4a083206ea9a7e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap64bd80e2-6c", "ovs_interfaceid": "64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:22:36 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:ed:9c:88,bridge_name='br-int',has_traffic_filtering=True,id=64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f,network=Network(9a452414-d61f-4045-a6cc-9be93c892c4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap64bd80e2-6c') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:22:36 user nova-compute[72351]: DEBUG nova.objects.instance [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lazy-loading 'pci_devices' on Instance uuid a7f3d076-864b-4b68-9207-214ce2c21444 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:22:36 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:22:36 user nova-compute[72351]:   <uuid>a7f3d076-864b-4b68-9207-214ce2c21444</uuid>
Aug 09 02:22:36 user nova-compute[72351]:   <name>instance-00000014</name>
Aug 09 02:22:36 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:22:36 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:22:36 user nova-compute[72351]:   <metadata>
Aug 09 02:22:36 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:22:36 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:22:36 user nova-compute[72351]:       <nova:name>tempest-ServerBootFromVolumeStableRescueTest-server-578493690</nova:name>
Aug 09 02:22:36 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:22:36</nova:creationTime>
Aug 09 02:22:36 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:22:36 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:22:36 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:22:36 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:22:36 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:22:36 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:22:36 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:22:36 user nova-compute[72351]:       <nova:owner>
Aug 09 02:22:36 user nova-compute[72351]:         <nova:user uuid="73d05c48be5640c28455e8ba8b05ef31">tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member</nova:user>
Aug 09 02:22:36 user nova-compute[72351]:         <nova:project uuid="321111f935dd4e8aab4a083206ea9a7e">tempest-ServerBootFromVolumeStableRescueTest-2108329302</nova:project>
Aug 09 02:22:36 user nova-compute[72351]:       </nova:owner>
Aug 09 02:22:36 user nova-compute[72351]:       <nova:root type="image" uuid="18ccda7f-31d3-4978-ab9d-4ed2f74eff94"/>
Aug 09 02:22:36 user nova-compute[72351]:       <nova:ports>
Aug 09 02:22:36 user nova-compute[72351]:         <nova:port uuid="64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f">
Aug 09 02:22:36 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.8" ipVersion="4"/>
Aug 09 02:22:36 user nova-compute[72351]:         </nova:port>
Aug 09 02:22:36 user nova-compute[72351]:       </nova:ports>
Aug 09 02:22:36 user nova-compute[72351]:     </nova:instance>
Aug 09 02:22:36 user nova-compute[72351]:   </metadata>
Aug 09 02:22:36 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:22:36 user nova-compute[72351]:     <system>
Aug 09 02:22:36 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:22:36 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:22:36 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:22:36 user nova-compute[72351]:       <entry name="serial">a7f3d076-864b-4b68-9207-214ce2c21444</entry>
Aug 09 02:22:36 user nova-compute[72351]:       <entry name="uuid">a7f3d076-864b-4b68-9207-214ce2c21444</entry>
Aug 09 02:22:36 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:22:36 user nova-compute[72351]:     </system>
Aug 09 02:22:36 user nova-compute[72351]:   </sysinfo>
Aug 09 02:22:36 user nova-compute[72351]:   <os>
Aug 09 02:22:36 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:22:36 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:22:36 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:22:36 user nova-compute[72351]:   </os>
Aug 09 02:22:36 user nova-compute[72351]:   <features>
Aug 09 02:22:36 user nova-compute[72351]:     <acpi/>
Aug 09 02:22:36 user nova-compute[72351]:     <apic/>
Aug 09 02:22:36 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:22:36 user nova-compute[72351]:   </features>
Aug 09 02:22:36 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:22:36 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:22:36 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:22:36 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:22:36 user nova-compute[72351]:   </clock>
Aug 09 02:22:36 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:22:36 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:22:36 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:22:36 user nova-compute[72351]:   </cpu>
Aug 09 02:22:36 user nova-compute[72351]:   <devices>
Aug 09 02:22:36 user nova-compute[72351]:     <disk type="file" device="disk">
Aug 09 02:22:36 user nova-compute[72351]:       <driver name="qemu" type="qcow2" cache="none"/>
Aug 09 02:22:36 user nova-compute[72351]:       <source file="/opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk"/>
Aug 09 02:22:36 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:22:36 user nova-compute[72351]:     </disk>
Aug 09 02:22:36 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:22:36 user nova-compute[72351]:       <mac address="fa:16:3e:ed:9c:88"/>
Aug 09 02:22:36 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:22:36 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:22:36 user nova-compute[72351]:       <target dev="tap64bd80e2-6c"/>
Aug 09 02:22:36 user nova-compute[72351]:     </interface>
Aug 09 02:22:36 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:22:36 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/console.log" append="off"/>
Aug 09 02:22:36 user nova-compute[72351]:     </serial>
Aug 09 02:22:36 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:22:36 user nova-compute[72351]:     <video>
Aug 09 02:22:36 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:22:36 user nova-compute[72351]:     </video>
Aug 09 02:22:36 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:22:36 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:22:36 user nova-compute[72351]:     </rng>
Aug 09 02:22:36 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:22:36 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:22:36 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:22:36 user nova-compute[72351]:     </memballoon>
Aug 09 02:22:36 user nova-compute[72351]:   </devices>
Aug 09 02:22:36 user nova-compute[72351]: </domain>
Aug 09 02:22:36 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:22:36 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:22:33Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-578493690',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-578493690',id=20,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='321111f935dd4e8aab4a083206ea9a7e',ramdisk_id='',reservation_id='r-ovw0qy0n',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-2108329302',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:22:35Z,user_data=None,user_id='73d05c48be5640c28455e8ba8b05ef31',uuid=a7f3d076-864b-4b68-9207-214ce2c21444,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f", "address": "fa:16:3e:ed:9c:88", "network": {"id": "9a452414-d61f-4045-a6cc-9be93c892c4c", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "321111f935dd4e8aab4a083206ea9a7e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap64bd80e2-6c", "ovs_interfaceid": "64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:22:36 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Converting VIF {"id": "64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f", "address": "fa:16:3e:ed:9c:88", "network": {"id": "9a452414-d61f-4045-a6cc-9be93c892c4c", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "321111f935dd4e8aab4a083206ea9a7e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap64bd80e2-6c", "ovs_interfaceid": "64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:22:36 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:ed:9c:88,bridge_name='br-int',has_traffic_filtering=True,id=64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f,network=Network(9a452414-d61f-4045-a6cc-9be93c892c4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap64bd80e2-6c') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:22:36 user nova-compute[72351]: DEBUG os_vif [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:ed:9c:88,bridge_name='br-int',has_traffic_filtering=True,id=64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f,network=Network(9a452414-d61f-4045-a6cc-9be93c892c4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap64bd80e2-6c') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:22:36 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:36 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:22:36 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:22:36 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:36 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap64bd80e2-6c, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:22:36 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap64bd80e2-6c, col_values=(('external_ids', {'iface-id': '64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:ed:9c:88', 'vm-uuid': 'a7f3d076-864b-4b68-9207-214ce2c21444'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:22:36 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:36 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:22:36 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:36 user nova-compute[72351]: INFO os_vif [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:ed:9c:88,bridge_name='br-int',has_traffic_filtering=True,id=64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f,network=Network(9a452414-d61f-4045-a6cc-9be93c892c4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap64bd80e2-6c')
Aug 09 02:22:37 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:22:37 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] No VIF found with MAC fa:16:3e:ed:9c:88, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:22:37 user nova-compute[72351]: DEBUG nova.network.neutron [req-7d61bcd5-ac69-4283-8fef-878c764cb654 req-2e9b0d08-6d29-4cf0-a51b-ed70428ccf2e service nova] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Updated VIF entry in instance network info cache for port 64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:22:37 user nova-compute[72351]: DEBUG nova.network.neutron [req-7d61bcd5-ac69-4283-8fef-878c764cb654 req-2e9b0d08-6d29-4cf0-a51b-ed70428ccf2e service nova] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Updating instance_info_cache with network_info: [{"id": "64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f", "address": "fa:16:3e:ed:9c:88", "network": {"id": "9a452414-d61f-4045-a6cc-9be93c892c4c", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "321111f935dd4e8aab4a083206ea9a7e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap64bd80e2-6c", "ovs_interfaceid": "64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:22:37 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-7d61bcd5-ac69-4283-8fef-878c764cb654 req-2e9b0d08-6d29-4cf0-a51b-ed70428ccf2e service nova] Releasing lock "refresh_cache-a7f3d076-864b-4b68-9207-214ce2c21444" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:22:37 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:38 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:38 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:38 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:39 user nova-compute[72351]: DEBUG nova.compute.manager [req-26f2d041-32a3-4634-8505-8a66631faa00 req-553e6e63-3973-4a8c-92ac-dca5f0c2397e service nova] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Received event network-vif-plugged-64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:22:39 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-26f2d041-32a3-4634-8505-8a66631faa00 req-553e6e63-3973-4a8c-92ac-dca5f0c2397e service nova] Acquiring lock "a7f3d076-864b-4b68-9207-214ce2c21444-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:22:39 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-26f2d041-32a3-4634-8505-8a66631faa00 req-553e6e63-3973-4a8c-92ac-dca5f0c2397e service nova] Lock "a7f3d076-864b-4b68-9207-214ce2c21444-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:22:39 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-26f2d041-32a3-4634-8505-8a66631faa00 req-553e6e63-3973-4a8c-92ac-dca5f0c2397e service nova] Lock "a7f3d076-864b-4b68-9207-214ce2c21444-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:22:39 user nova-compute[72351]: DEBUG nova.compute.manager [req-26f2d041-32a3-4634-8505-8a66631faa00 req-553e6e63-3973-4a8c-92ac-dca5f0c2397e service nova] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] No waiting events found dispatching network-vif-plugged-64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:22:39 user nova-compute[72351]: WARNING nova.compute.manager [req-26f2d041-32a3-4634-8505-8a66631faa00 req-553e6e63-3973-4a8c-92ac-dca5f0c2397e service nova] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Received unexpected event network-vif-plugged-64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f for instance with vm_state building and task_state spawning.
Aug 09 02:22:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:41 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691547746.0385451, a1d1f738-0edf-4184-aa56-020c07830492 => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:22:41 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: a1d1f738-0edf-4184-aa56-020c07830492] VM Stopped (Lifecycle Event)
Aug 09 02:22:41 user nova-compute[72351]: DEBUG nova.compute.manager [None req-5fa70a20-f83f-4bba-9396-3731f97853ed None None] [instance: a1d1f738-0edf-4184-aa56-020c07830492] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:22:41 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547761.1235905, a7f3d076-864b-4b68-9207-214ce2c21444 => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:22:41 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] VM Resumed (Lifecycle Event)
Aug 09 02:22:41 user nova-compute[72351]: DEBUG nova.compute.manager [req-83e11bdf-4b44-458c-ae7f-6d8c39b88197 req-bbbc4632-6d52-405a-b188-ea1b74b4e0a6 service nova] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Received event network-vif-plugged-64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:22:41 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-83e11bdf-4b44-458c-ae7f-6d8c39b88197 req-bbbc4632-6d52-405a-b188-ea1b74b4e0a6 service nova] Acquiring lock "a7f3d076-864b-4b68-9207-214ce2c21444-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:22:41 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-83e11bdf-4b44-458c-ae7f-6d8c39b88197 req-bbbc4632-6d52-405a-b188-ea1b74b4e0a6 service nova] Lock "a7f3d076-864b-4b68-9207-214ce2c21444-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:22:41 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-83e11bdf-4b44-458c-ae7f-6d8c39b88197 req-bbbc4632-6d52-405a-b188-ea1b74b4e0a6 service nova] Lock "a7f3d076-864b-4b68-9207-214ce2c21444-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:22:41 user nova-compute[72351]: DEBUG nova.compute.manager [req-83e11bdf-4b44-458c-ae7f-6d8c39b88197 req-bbbc4632-6d52-405a-b188-ea1b74b4e0a6 service nova] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] No waiting events found dispatching network-vif-plugged-64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:22:41 user nova-compute[72351]: WARNING nova.compute.manager [req-83e11bdf-4b44-458c-ae7f-6d8c39b88197 req-bbbc4632-6d52-405a-b188-ea1b74b4e0a6 service nova] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Received unexpected event network-vif-plugged-64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f for instance with vm_state building and task_state spawning.
Aug 09 02:22:41 user nova-compute[72351]: DEBUG nova.compute.manager [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:22:41 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Guest created on hypervisor {{(pid=72351) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4411}}
Aug 09 02:22:41 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Instance spawned successfully.
Aug 09 02:22:41 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:894}}
Aug 09 02:22:41 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:22:41 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:22:41 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Found default for hw_cdrom_bus of ide {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:22:41 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Found default for hw_disk_bus of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:22:41 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Found default for hw_input_bus of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:22:41 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Found default for hw_pointer_model of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:22:41 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Found default for hw_video_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:22:41 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Found default for hw_vif_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:22:41 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:22:41 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547761.126299, a7f3d076-864b-4b68-9207-214ce2c21444 => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:22:41 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] VM Started (Lifecycle Event)
Aug 09 02:22:41 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:22:41 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:22:41 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:22:41 user nova-compute[72351]: INFO nova.compute.manager [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Took 6.27 seconds to spawn the instance on the hypervisor.
Aug 09 02:22:41 user nova-compute[72351]: DEBUG nova.compute.manager [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:22:41 user nova-compute[72351]: INFO nova.compute.manager [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Took 7.08 seconds to build instance.
Aug 09 02:22:41 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ff0a014a-1095-4565-9685-1ace7ffb57aa tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "a7f3d076-864b-4b68-9207-214ce2c21444" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 7.181s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:22:41 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:42 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:46 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:47 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:49 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._run_pending_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:22:49 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Cleaning up deleted instances {{(pid=72351) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11129}}
Aug 09 02:22:49 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] There are 0 instances to clean {{(pid=72351) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11138}}
Aug 09 02:22:51 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:52 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:52 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:22:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:22:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:22:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:22:52 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Auditing locally available compute resources for user (node: user) {{(pid=72351) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Aug 09 02:22:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:22:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json" returned: 0 in 0.157s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:22:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:22:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:22:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:22:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json" returned: 0 in 0.151s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:22:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:22:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json" returned: 0 in 0.166s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:22:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:22:53 user nova-compute[72351]: INFO nova.compute.manager [None req-e8072ff4-149c-4d64-b563-4c97106e71c7 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Unrescuing
Aug 09 02:22:53 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e8072ff4-149c-4d64-b563-4c97106e71c7 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Acquiring lock "refresh_cache-2d78475c-107d-4256-b12e-dbe9c8f3b9b2" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:22:53 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e8072ff4-149c-4d64-b563-4c97106e71c7 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Acquired lock "refresh_cache-2d78475c-107d-4256-b12e-dbe9c8f3b9b2" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:22:53 user nova-compute[72351]: DEBUG nova.network.neutron [None req-e8072ff4-149c-4d64-b563-4c97106e71c7 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:22:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json" returned: 0 in 0.195s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:22:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:22:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:22:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:22:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2/disk --force-share --output=json" returned: 0 in 0.140s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:22:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:22:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2/disk --force-share --output=json" returned: 0 in 0.158s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:22:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdc (vdb) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:22:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2/disk.rescue --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:22:54 user nova-compute[72351]: DEBUG nova.network.neutron [None req-e8072ff4-149c-4d64-b563-4c97106e71c7 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Updating instance_info_cache with network_info: [{"id": "ff25eb15-1af0-4110-9d9a-ecd0a1d4627d", "address": "fa:16:3e:0a:89:6a", "network": {"id": "fd1fe4e0-5d09-43c1-8dbc-1f60cb7f38d7", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1014783449-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.229", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "747d5750a2174aaea71849c70167a816", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapff25eb15-1a", "ovs_interfaceid": "ff25eb15-1af0-4110-9d9a-ecd0a1d4627d", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:22:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e8072ff4-149c-4d64-b563-4c97106e71c7 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Releasing lock "refresh_cache-2d78475c-107d-4256-b12e-dbe9c8f3b9b2" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:22:54 user nova-compute[72351]: DEBUG nova.objects.instance [None req-e8072ff4-149c-4d64-b563-4c97106e71c7 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Lazy-loading 'flavor' on Instance uuid 2d78475c-107d-4256-b12e-dbe9c8f3b9b2 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:22:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2/disk.rescue --force-share --output=json" returned: 0 in 0.202s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:22:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2/disk.rescue --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:22:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2/disk.rescue --force-share --output=json" returned: 0 in 0.162s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:22:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:22:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:55 user nova-compute[72351]: DEBUG nova.compute.manager [req-3bbc9ed4-a217-44b0-ab47-1e146f6eb9bf req-8846f374-199d-41f4-b05b-d8dcf3612159 service nova] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Received event network-vif-unplugged-ff25eb15-1af0-4110-9d9a-ecd0a1d4627d {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:22:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-3bbc9ed4-a217-44b0-ab47-1e146f6eb9bf req-8846f374-199d-41f4-b05b-d8dcf3612159 service nova] Acquiring lock "2d78475c-107d-4256-b12e-dbe9c8f3b9b2-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:22:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-3bbc9ed4-a217-44b0-ab47-1e146f6eb9bf req-8846f374-199d-41f4-b05b-d8dcf3612159 service nova] Lock "2d78475c-107d-4256-b12e-dbe9c8f3b9b2-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:22:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-3bbc9ed4-a217-44b0-ab47-1e146f6eb9bf req-8846f374-199d-41f4-b05b-d8dcf3612159 service nova] Lock "2d78475c-107d-4256-b12e-dbe9c8f3b9b2-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:22:55 user nova-compute[72351]: DEBUG nova.compute.manager [req-3bbc9ed4-a217-44b0-ab47-1e146f6eb9bf req-8846f374-199d-41f4-b05b-d8dcf3612159 service nova] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] No waiting events found dispatching network-vif-unplugged-ff25eb15-1af0-4110-9d9a-ecd0a1d4627d {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:22:55 user nova-compute[72351]: WARNING nova.compute.manager [req-3bbc9ed4-a217-44b0-ab47-1e146f6eb9bf req-8846f374-199d-41f4-b05b-d8dcf3612159 service nova] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Received unexpected event network-vif-unplugged-ff25eb15-1af0-4110-9d9a-ecd0a1d4627d for instance with vm_state rescued and task_state unrescuing.
Aug 09 02:22:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json" returned: 0 in 0.213s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:22:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:22:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:22:55 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdd (vda) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:22:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/74728ca8-34d4-45e6-a1f2-21c0952ff835/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:22:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/74728ca8-34d4-45e6-a1f2-21c0952ff835/disk --force-share --output=json" returned: 0 in 0.150s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:22:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/74728ca8-34d4-45e6-a1f2-21c0952ff835/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:22:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/74728ca8-34d4-45e6-a1f2-21c0952ff835/disk --force-share --output=json" returned: 0 in 0.179s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:22:55 user nova-compute[72351]: DEBUG nova.compute.manager [req-ce753446-a1b9-4c33-accd-7bb5beafa1a7 req-37a1b90f-f6ea-4187-bf35-df9c633654a1 service nova] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Received event network-changed-ff25eb15-1af0-4110-9d9a-ecd0a1d4627d {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:22:55 user nova-compute[72351]: DEBUG nova.compute.manager [req-ce753446-a1b9-4c33-accd-7bb5beafa1a7 req-37a1b90f-f6ea-4187-bf35-df9c633654a1 service nova] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Refreshing instance network info cache due to event network-changed-ff25eb15-1af0-4110-9d9a-ecd0a1d4627d. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:22:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ce753446-a1b9-4c33-accd-7bb5beafa1a7 req-37a1b90f-f6ea-4187-bf35-df9c633654a1 service nova] Acquiring lock "refresh_cache-2d78475c-107d-4256-b12e-dbe9c8f3b9b2" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:22:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ce753446-a1b9-4c33-accd-7bb5beafa1a7 req-37a1b90f-f6ea-4187-bf35-df9c633654a1 service nova] Acquired lock "refresh_cache-2d78475c-107d-4256-b12e-dbe9c8f3b9b2" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:22:55 user nova-compute[72351]: DEBUG nova.network.neutron [req-ce753446-a1b9-4c33-accd-7bb5beafa1a7 req-37a1b90f-f6ea-4187-bf35-df9c633654a1 service nova] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Refreshing network info cache for port ff25eb15-1af0-4110-9d9a-ecd0a1d4627d {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:22:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/97b13ccd-15b4-44bf-b865-71b4da861168/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:22:55 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Instance destroyed successfully.
Aug 09 02:22:55 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/97b13ccd-15b4-44bf-b865-71b4da861168/disk --force-share --output=json" returned: 0 in 0.139s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:22:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/97b13ccd-15b4-44bf-b865-71b4da861168/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:22:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/97b13ccd-15b4-44bf-b865-71b4da861168/disk --force-share --output=json" returned: 0 in 0.142s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:22:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:22:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff/disk --force-share --output=json" returned: 0 in 0.163s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:22:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:22:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff/disk --force-share --output=json" returned: 0 in 0.152s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:22:56 user nova-compute[72351]: DEBUG nova.network.neutron [req-ce753446-a1b9-4c33-accd-7bb5beafa1a7 req-37a1b90f-f6ea-4187-bf35-df9c633654a1 service nova] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Updated VIF entry in instance network info cache for port ff25eb15-1af0-4110-9d9a-ecd0a1d4627d. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:22:56 user nova-compute[72351]: DEBUG nova.network.neutron [req-ce753446-a1b9-4c33-accd-7bb5beafa1a7 req-37a1b90f-f6ea-4187-bf35-df9c633654a1 service nova] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Updating instance_info_cache with network_info: [{"id": "ff25eb15-1af0-4110-9d9a-ecd0a1d4627d", "address": "fa:16:3e:0a:89:6a", "network": {"id": "fd1fe4e0-5d09-43c1-8dbc-1f60cb7f38d7", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1014783449-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.229", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "747d5750a2174aaea71849c70167a816", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapff25eb15-1a", "ovs_interfaceid": "ff25eb15-1af0-4110-9d9a-ecd0a1d4627d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:22:56 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ce753446-a1b9-4c33-accd-7bb5beafa1a7 req-37a1b90f-f6ea-4187-bf35-df9c633654a1 service nova] Releasing lock "refresh_cache-2d78475c-107d-4256-b12e-dbe9c8f3b9b2" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:22:56 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:22:56 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:22:56 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Hypervisor/Node resource view: name=user free_ram=7204MB free_disk=25.464012145996094GB free_vcpus=3 pci_devices=[{"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72351) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Aug 09 02:22:56 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:22:56 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:22:56 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 2d78475c-107d-4256-b12e-dbe9c8f3b9b2 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:22:56 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 74728ca8-34d4-45e6-a1f2-21c0952ff835 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:22:56 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 97b13ccd-15b4-44bf-b865-71b4da861168 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:22:56 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 75bbbead-8628-448b-9484-19bf4ab922ff actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:22:56 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:22:56 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance eb86cb49-2018-42c2-bd97-c662b829775d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:22:56 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 1d772c10-1c8a-4101-9a61-f8a60bb19c66 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:22:56 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:22:56 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance a7f3d076-864b-4b68-9207-214ce2c21444 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:22:56 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Total usable vcpus: 12, total allocated vcpus: 9 {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Aug 09 02:22:56 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Final resource view: name=user phys_ram=16011MB used_ram=5120MB phys_disk=40GB used_disk=8GB total_vcpus=12 used_vcpus=9 pci_stats=[] {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Aug 09 02:22:56 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:57 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:57 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:57 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:57 user nova-compute[72351]: DEBUG nova.compute.manager [req-8ae6e0f7-e738-48f6-8c23-f60194c78444 req-c3fde5b1-130b-4923-92d4-b75d8b41b3e9 service nova] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Received event network-vif-plugged-ff25eb15-1af0-4110-9d9a-ecd0a1d4627d {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:22:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-8ae6e0f7-e738-48f6-8c23-f60194c78444 req-c3fde5b1-130b-4923-92d4-b75d8b41b3e9 service nova] Acquiring lock "2d78475c-107d-4256-b12e-dbe9c8f3b9b2-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:22:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-8ae6e0f7-e738-48f6-8c23-f60194c78444 req-c3fde5b1-130b-4923-92d4-b75d8b41b3e9 service nova] Lock "2d78475c-107d-4256-b12e-dbe9c8f3b9b2-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:22:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-8ae6e0f7-e738-48f6-8c23-f60194c78444 req-c3fde5b1-130b-4923-92d4-b75d8b41b3e9 service nova] Lock "2d78475c-107d-4256-b12e-dbe9c8f3b9b2-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:22:57 user nova-compute[72351]: DEBUG nova.compute.manager [req-8ae6e0f7-e738-48f6-8c23-f60194c78444 req-c3fde5b1-130b-4923-92d4-b75d8b41b3e9 service nova] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] No waiting events found dispatching network-vif-plugged-ff25eb15-1af0-4110-9d9a-ecd0a1d4627d {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:22:57 user nova-compute[72351]: WARNING nova.compute.manager [req-8ae6e0f7-e738-48f6-8c23-f60194c78444 req-c3fde5b1-130b-4923-92d4-b75d8b41b3e9 service nova] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Received unexpected event network-vif-plugged-ff25eb15-1af0-4110-9d9a-ecd0a1d4627d for instance with vm_state rescued and task_state unrescuing.
Aug 09 02:22:57 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:57 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:22:57 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:22:57 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Compute_service record updated for user:user {{(pid=72351) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Aug 09 02:22:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.558s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:22:57 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._cleanup_incomplete_migrations {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:22:57 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Cleaning up deleted instances with incomplete migration  {{(pid=72351) _cleanup_incomplete_migrations /opt/stack/nova/nova/compute/manager.py:11167}}
Aug 09 02:22:57 user nova-compute[72351]: DEBUG nova.compute.manager [req-658e0169-4537-4d06-acf4-78775e601de6 req-c71f7420-6124-4083-8570-2f7a96554f82 service nova] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Received event network-changed-ff25eb15-1af0-4110-9d9a-ecd0a1d4627d {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:22:57 user nova-compute[72351]: DEBUG nova.compute.manager [req-658e0169-4537-4d06-acf4-78775e601de6 req-c71f7420-6124-4083-8570-2f7a96554f82 service nova] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Refreshing instance network info cache due to event network-changed-ff25eb15-1af0-4110-9d9a-ecd0a1d4627d. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:22:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-658e0169-4537-4d06-acf4-78775e601de6 req-c71f7420-6124-4083-8570-2f7a96554f82 service nova] Acquiring lock "refresh_cache-2d78475c-107d-4256-b12e-dbe9c8f3b9b2" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:22:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-658e0169-4537-4d06-acf4-78775e601de6 req-c71f7420-6124-4083-8570-2f7a96554f82 service nova] Acquired lock "refresh_cache-2d78475c-107d-4256-b12e-dbe9c8f3b9b2" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:22:57 user nova-compute[72351]: DEBUG nova.network.neutron [req-658e0169-4537-4d06-acf4-78775e601de6 req-c71f7420-6124-4083-8570-2f7a96554f82 service nova] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Refreshing network info cache for port ff25eb15-1af0-4110-9d9a-ecd0a1d4627d {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:22:57 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:57 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:57 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:22:58 user nova-compute[72351]: DEBUG nova.network.neutron [req-658e0169-4537-4d06-acf4-78775e601de6 req-c71f7420-6124-4083-8570-2f7a96554f82 service nova] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Updated VIF entry in instance network info cache for port ff25eb15-1af0-4110-9d9a-ecd0a1d4627d. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:22:58 user nova-compute[72351]: DEBUG nova.network.neutron [req-658e0169-4537-4d06-acf4-78775e601de6 req-c71f7420-6124-4083-8570-2f7a96554f82 service nova] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Updating instance_info_cache with network_info: [{"id": "ff25eb15-1af0-4110-9d9a-ecd0a1d4627d", "address": "fa:16:3e:0a:89:6a", "network": {"id": "fd1fe4e0-5d09-43c1-8dbc-1f60cb7f38d7", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1014783449-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.229", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "747d5750a2174aaea71849c70167a816", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapff25eb15-1a", "ovs_interfaceid": "ff25eb15-1af0-4110-9d9a-ecd0a1d4627d", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:22:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-658e0169-4537-4d06-acf4-78775e601de6 req-c71f7420-6124-4083-8570-2f7a96554f82 service nova] Releasing lock "refresh_cache-2d78475c-107d-4256-b12e-dbe9c8f3b9b2" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:22:59 user nova-compute[72351]: DEBUG nova.compute.manager [req-3f2d5b65-c620-48c6-9e3d-c023f2600ebd req-71daa772-8856-4387-9b72-e054a1a81de3 service nova] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Received event network-vif-plugged-ff25eb15-1af0-4110-9d9a-ecd0a1d4627d {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:22:59 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-3f2d5b65-c620-48c6-9e3d-c023f2600ebd req-71daa772-8856-4387-9b72-e054a1a81de3 service nova] Acquiring lock "2d78475c-107d-4256-b12e-dbe9c8f3b9b2-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:22:59 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-3f2d5b65-c620-48c6-9e3d-c023f2600ebd req-71daa772-8856-4387-9b72-e054a1a81de3 service nova] Lock "2d78475c-107d-4256-b12e-dbe9c8f3b9b2-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:22:59 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-3f2d5b65-c620-48c6-9e3d-c023f2600ebd req-71daa772-8856-4387-9b72-e054a1a81de3 service nova] Lock "2d78475c-107d-4256-b12e-dbe9c8f3b9b2-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:22:59 user nova-compute[72351]: DEBUG nova.compute.manager [req-3f2d5b65-c620-48c6-9e3d-c023f2600ebd req-71daa772-8856-4387-9b72-e054a1a81de3 service nova] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] No waiting events found dispatching network-vif-plugged-ff25eb15-1af0-4110-9d9a-ecd0a1d4627d {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:22:59 user nova-compute[72351]: WARNING nova.compute.manager [req-3f2d5b65-c620-48c6-9e3d-c023f2600ebd req-71daa772-8856-4387-9b72-e054a1a81de3 service nova] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Received unexpected event network-vif-plugged-ff25eb15-1af0-4110-9d9a-ecd0a1d4627d for instance with vm_state rescued and task_state unrescuing.
Aug 09 02:22:59 user nova-compute[72351]: DEBUG nova.compute.manager [req-3f2d5b65-c620-48c6-9e3d-c023f2600ebd req-71daa772-8856-4387-9b72-e054a1a81de3 service nova] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Received event network-vif-plugged-ff25eb15-1af0-4110-9d9a-ecd0a1d4627d {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:22:59 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-3f2d5b65-c620-48c6-9e3d-c023f2600ebd req-71daa772-8856-4387-9b72-e054a1a81de3 service nova] Acquiring lock "2d78475c-107d-4256-b12e-dbe9c8f3b9b2-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:22:59 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-3f2d5b65-c620-48c6-9e3d-c023f2600ebd req-71daa772-8856-4387-9b72-e054a1a81de3 service nova] Lock "2d78475c-107d-4256-b12e-dbe9c8f3b9b2-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:22:59 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-3f2d5b65-c620-48c6-9e3d-c023f2600ebd req-71daa772-8856-4387-9b72-e054a1a81de3 service nova] Lock "2d78475c-107d-4256-b12e-dbe9c8f3b9b2-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:22:59 user nova-compute[72351]: DEBUG nova.compute.manager [req-3f2d5b65-c620-48c6-9e3d-c023f2600ebd req-71daa772-8856-4387-9b72-e054a1a81de3 service nova] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] No waiting events found dispatching network-vif-plugged-ff25eb15-1af0-4110-9d9a-ecd0a1d4627d {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:22:59 user nova-compute[72351]: WARNING nova.compute.manager [req-3f2d5b65-c620-48c6-9e3d-c023f2600ebd req-71daa772-8856-4387-9b72-e054a1a81de3 service nova] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Received unexpected event network-vif-plugged-ff25eb15-1af0-4110-9d9a-ecd0a1d4627d for instance with vm_state rescued and task_state unrescuing.
Aug 09 02:22:59 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:22:59 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:22:59 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:22:59 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Starting heal instance info cache {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9842}}
Aug 09 02:22:59 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Rebuilding the list of instances to heal {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9846}}
Aug 09 02:22:59 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "refresh_cache-2d78475c-107d-4256-b12e-dbe9c8f3b9b2" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:22:59 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquired lock "refresh_cache-2d78475c-107d-4256-b12e-dbe9c8f3b9b2" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:22:59 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Forcefully refreshing network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1996}}
Aug 09 02:22:59 user nova-compute[72351]: DEBUG nova.objects.instance [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lazy-loading 'info_cache' on Instance uuid 2d78475c-107d-4256-b12e-dbe9c8f3b9b2 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:22:59 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Removed pending event for 2d78475c-107d-4256-b12e-dbe9c8f3b9b2 due to event {{(pid=72351) _event_emit_delayed /opt/stack/nova/nova/virt/libvirt/host.py:438}}
Aug 09 02:22:59 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547779.5402787, 2d78475c-107d-4256-b12e-dbe9c8f3b9b2 => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:22:59 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] VM Resumed (Lifecycle Event)
Aug 09 02:22:59 user nova-compute[72351]: DEBUG nova.compute.manager [None req-e8072ff4-149c-4d64-b563-4c97106e71c7 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:22:59 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:22:59 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: rescued, current task_state: unrescuing, current DB power_state: 1, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:22:59 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] During sync_power_state the instance has a pending task (unrescuing). Skip.
Aug 09 02:22:59 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547779.5441222, 2d78475c-107d-4256-b12e-dbe9c8f3b9b2 => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:22:59 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] VM Started (Lifecycle Event)
Aug 09 02:22:59 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:22:59 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Synchronizing instance power state after lifecycle event "Started"; current vm_state: rescued, current task_state: unrescuing, current DB power_state: 1, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:22:59 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] During sync_power_state the instance has a pending task (unrescuing). Skip.
Aug 09 02:23:00 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Updating instance_info_cache with network_info: [{"id": "ff25eb15-1af0-4110-9d9a-ecd0a1d4627d", "address": "fa:16:3e:0a:89:6a", "network": {"id": "fd1fe4e0-5d09-43c1-8dbc-1f60cb7f38d7", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1014783449-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.229", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "747d5750a2174aaea71849c70167a816", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapff25eb15-1a", "ovs_interfaceid": "ff25eb15-1af0-4110-9d9a-ecd0a1d4627d", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:23:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Releasing lock "refresh_cache-2d78475c-107d-4256-b12e-dbe9c8f3b9b2" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:23:00 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Updated the network info_cache for instance {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9913}}
Aug 09 02:23:00 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:23:00 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:23:00 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:23:00 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:23:00 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:23:00 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72351) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10461}}
Aug 09 02:23:00 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._cleanup_expired_console_auth_tokens {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:23:00 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:23:01 user nova-compute[72351]: DEBUG nova.compute.manager [req-948c6c88-1832-4b39-a246-cf4c1a21311d req-9f0969b2-3f38-4020-9eb0-7ec6e775b800 service nova] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Received event network-changed-be4ec483-8532-45f7-bb9f-0b50952ce94b {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:23:01 user nova-compute[72351]: DEBUG nova.compute.manager [req-948c6c88-1832-4b39-a246-cf4c1a21311d req-9f0969b2-3f38-4020-9eb0-7ec6e775b800 service nova] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Refreshing instance network info cache due to event network-changed-be4ec483-8532-45f7-bb9f-0b50952ce94b. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:23:01 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-948c6c88-1832-4b39-a246-cf4c1a21311d req-9f0969b2-3f38-4020-9eb0-7ec6e775b800 service nova] Acquiring lock "refresh_cache-75bbbead-8628-448b-9484-19bf4ab922ff" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:23:01 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-948c6c88-1832-4b39-a246-cf4c1a21311d req-9f0969b2-3f38-4020-9eb0-7ec6e775b800 service nova] Acquired lock "refresh_cache-75bbbead-8628-448b-9484-19bf4ab922ff" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:23:01 user nova-compute[72351]: DEBUG nova.network.neutron [req-948c6c88-1832-4b39-a246-cf4c1a21311d req-9f0969b2-3f38-4020-9eb0-7ec6e775b800 service nova] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Refreshing network info cache for port be4ec483-8532-45f7-bb9f-0b50952ce94b {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:23:01 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:23:02 user nova-compute[72351]: DEBUG nova.network.neutron [req-948c6c88-1832-4b39-a246-cf4c1a21311d req-9f0969b2-3f38-4020-9eb0-7ec6e775b800 service nova] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Updated VIF entry in instance network info cache for port be4ec483-8532-45f7-bb9f-0b50952ce94b. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:23:02 user nova-compute[72351]: DEBUG nova.network.neutron [req-948c6c88-1832-4b39-a246-cf4c1a21311d req-9f0969b2-3f38-4020-9eb0-7ec6e775b800 service nova] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Updating instance_info_cache with network_info: [{"id": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "address": "fa:16:3e:8b:b5:2e", "network": {"id": "8231f2c2-237e-41c6-8892-720b950c1453", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherB-1917380761-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.30", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fc90e94bbcb54fb6b86a9ac45edb596a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbe4ec483-85", "ovs_interfaceid": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:23:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-948c6c88-1832-4b39-a246-cf4c1a21311d req-9f0969b2-3f38-4020-9eb0-7ec6e775b800 service nova] Releasing lock "refresh_cache-75bbbead-8628-448b-9484-19bf4ab922ff" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:23:02 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:23:06 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:23:07 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:23:07 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:23:11 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:23:11 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:23:12 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:23:16 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:23:17 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:23:18 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._sync_power_states {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:23:19 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Triggering sync for uuid 2d78475c-107d-4256-b12e-dbe9c8f3b9b2 {{(pid=72351) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10252}}
Aug 09 02:23:19 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Triggering sync for uuid 74728ca8-34d4-45e6-a1f2-21c0952ff835 {{(pid=72351) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10252}}
Aug 09 02:23:19 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Triggering sync for uuid 97b13ccd-15b4-44bf-b865-71b4da861168 {{(pid=72351) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10252}}
Aug 09 02:23:19 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Triggering sync for uuid 75bbbead-8628-448b-9484-19bf4ab922ff {{(pid=72351) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10252}}
Aug 09 02:23:19 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Triggering sync for uuid e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335 {{(pid=72351) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10252}}
Aug 09 02:23:19 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Triggering sync for uuid eb86cb49-2018-42c2-bd97-c662b829775d {{(pid=72351) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10252}}
Aug 09 02:23:19 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Triggering sync for uuid 1d772c10-1c8a-4101-9a61-f8a60bb19c66 {{(pid=72351) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10252}}
Aug 09 02:23:19 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Triggering sync for uuid 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a {{(pid=72351) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10252}}
Aug 09 02:23:19 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Triggering sync for uuid a7f3d076-864b-4b68-9207-214ce2c21444 {{(pid=72351) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10252}}
Aug 09 02:23:19 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "2d78475c-107d-4256-b12e-dbe9c8f3b9b2" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:23:19 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "2d78475c-107d-4256-b12e-dbe9c8f3b9b2" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:23:19 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "74728ca8-34d4-45e6-a1f2-21c0952ff835" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:23:19 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "74728ca8-34d4-45e6-a1f2-21c0952ff835" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:23:19 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "97b13ccd-15b4-44bf-b865-71b4da861168" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:23:19 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "97b13ccd-15b4-44bf-b865-71b4da861168" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:23:19 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "75bbbead-8628-448b-9484-19bf4ab922ff" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:23:19 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "75bbbead-8628-448b-9484-19bf4ab922ff" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:23:19 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:23:19 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:23:19 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "eb86cb49-2018-42c2-bd97-c662b829775d" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:23:19 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "eb86cb49-2018-42c2-bd97-c662b829775d" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:23:19 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "1d772c10-1c8a-4101-9a61-f8a60bb19c66" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:23:19 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "1d772c10-1c8a-4101-9a61-f8a60bb19c66" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:23:19 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "585c81a5-ac24-4d12-a8a6-1e7b12a2d53a" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:23:19 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "585c81a5-ac24-4d12-a8a6-1e7b12a2d53a" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:23:19 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "a7f3d076-864b-4b68-9207-214ce2c21444" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:23:19 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "a7f3d076-864b-4b68-9207-214ce2c21444" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:23:19 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "74728ca8-34d4-45e6-a1f2-21c0952ff835" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.078s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:23:19 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "75bbbead-8628-448b-9484-19bf4ab922ff" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.079s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:23:19 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "97b13ccd-15b4-44bf-b865-71b4da861168" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.082s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:23:19 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.082s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:23:19 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "585c81a5-ac24-4d12-a8a6-1e7b12a2d53a" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.082s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:23:19 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "eb86cb49-2018-42c2-bd97-c662b829775d" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.087s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:23:19 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "1d772c10-1c8a-4101-9a61-f8a60bb19c66" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.092s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:23:19 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "2d78475c-107d-4256-b12e-dbe9c8f3b9b2" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.099s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:23:19 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "a7f3d076-864b-4b68-9207-214ce2c21444" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.098s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:23:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:23:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:23:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c871e8d9-7741-41d7-b8d8-c1f9e9176e92 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Acquiring lock "97b13ccd-15b4-44bf-b865-71b4da861168" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:23:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c871e8d9-7741-41d7-b8d8-c1f9e9176e92 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "97b13ccd-15b4-44bf-b865-71b4da861168" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:23:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c871e8d9-7741-41d7-b8d8-c1f9e9176e92 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Acquiring lock "97b13ccd-15b4-44bf-b865-71b4da861168-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:23:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c871e8d9-7741-41d7-b8d8-c1f9e9176e92 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "97b13ccd-15b4-44bf-b865-71b4da861168-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:23:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c871e8d9-7741-41d7-b8d8-c1f9e9176e92 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "97b13ccd-15b4-44bf-b865-71b4da861168-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:23:24 user nova-compute[72351]: INFO nova.compute.manager [None req-c871e8d9-7741-41d7-b8d8-c1f9e9176e92 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Terminating instance
Aug 09 02:23:24 user nova-compute[72351]: DEBUG nova.compute.manager [None req-c871e8d9-7741-41d7-b8d8-c1f9e9176e92 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Start destroying the instance on the hypervisor. {{(pid=72351) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3124}}
Aug 09 02:23:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:23:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:23:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:23:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:23:24 user nova-compute[72351]: DEBUG nova.compute.manager [req-f311ff92-5fc8-425c-bd0a-a38bccdeffcc req-3eeb2e4c-2d0d-4f78-a625-f760a8589d06 service nova] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Received event network-vif-unplugged-74c98543-ce04-43d8-a58c-267e372db698 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:23:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-f311ff92-5fc8-425c-bd0a-a38bccdeffcc req-3eeb2e4c-2d0d-4f78-a625-f760a8589d06 service nova] Acquiring lock "97b13ccd-15b4-44bf-b865-71b4da861168-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:23:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-f311ff92-5fc8-425c-bd0a-a38bccdeffcc req-3eeb2e4c-2d0d-4f78-a625-f760a8589d06 service nova] Lock "97b13ccd-15b4-44bf-b865-71b4da861168-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:23:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-f311ff92-5fc8-425c-bd0a-a38bccdeffcc req-3eeb2e4c-2d0d-4f78-a625-f760a8589d06 service nova] Lock "97b13ccd-15b4-44bf-b865-71b4da861168-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:23:24 user nova-compute[72351]: DEBUG nova.compute.manager [req-f311ff92-5fc8-425c-bd0a-a38bccdeffcc req-3eeb2e4c-2d0d-4f78-a625-f760a8589d06 service nova] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] No waiting events found dispatching network-vif-unplugged-74c98543-ce04-43d8-a58c-267e372db698 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:23:24 user nova-compute[72351]: DEBUG nova.compute.manager [req-f311ff92-5fc8-425c-bd0a-a38bccdeffcc req-3eeb2e4c-2d0d-4f78-a625-f760a8589d06 service nova] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Received event network-vif-unplugged-74c98543-ce04-43d8-a58c-267e372db698 for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:23:25 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Instance destroyed successfully.
Aug 09 02:23:25 user nova-compute[72351]: DEBUG nova.objects.instance [None req-c871e8d9-7741-41d7-b8d8-c1f9e9176e92 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lazy-loading 'resources' on Instance uuid 97b13ccd-15b4-44bf-b865-71b4da861168 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:23:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e7f95695-a02a-4cf1-8213-07527512cf25 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Acquiring lock "74728ca8-34d4-45e6-a1f2-21c0952ff835" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:23:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e7f95695-a02a-4cf1-8213-07527512cf25 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Lock "74728ca8-34d4-45e6-a1f2-21c0952ff835" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:23:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e7f95695-a02a-4cf1-8213-07527512cf25 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Acquiring lock "74728ca8-34d4-45e6-a1f2-21c0952ff835-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:23:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e7f95695-a02a-4cf1-8213-07527512cf25 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Lock "74728ca8-34d4-45e6-a1f2-21c0952ff835-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:23:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e7f95695-a02a-4cf1-8213-07527512cf25 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Lock "74728ca8-34d4-45e6-a1f2-21c0952ff835-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:23:25 user nova-compute[72351]: INFO nova.compute.manager [None req-e7f95695-a02a-4cf1-8213-07527512cf25 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Terminating instance
Aug 09 02:23:25 user nova-compute[72351]: DEBUG nova.compute.manager [None req-e7f95695-a02a-4cf1-8213-07527512cf25 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Start destroying the instance on the hypervisor. {{(pid=72351) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3124}}
Aug 09 02:23:25 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-c871e8d9-7741-41d7-b8d8-c1f9e9176e92 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:18:04Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestMinimumBasicScenario-server-1233822315',display_name='tempest-TestMinimumBasicScenario-server-1233822315',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testminimumbasicscenario-server-1233822315',id=8,image_ref='9eadc1b0-9cb8-4bc1-97d6-af86e590c548',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOk8lLSpSCaCDIKlT4lhWRaHWy+oi2s0IJNKXHyKayH1X4sCjRLnVKeip/ShH5Av4ZZCopMp+S7MD8OAGw8MyvDsE398yCbkjnI/7XCIasyAoPOQXBUUZ52swhczfVlPLg==',key_name='tempest-TestMinimumBasicScenario-1974156570',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:18:12Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='c2f67da658d94fabbab82ee6a6171c4b',ramdisk_id='',reservation_id='r-rhqfwdu0',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='9eadc1b0-9cb8-4bc1-97d6-af86e590c548',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',owner_project_name='tempest-TestMinimumBasicScenario-1380132717',owner_user_name='tempest-TestMinimumBasicScenario-1380132717-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:20:41Z,user_data=None,user_id='bec5b5704e9b4e3480f428d5c037c20d',uuid=97b13ccd-15b4-44bf-b865-71b4da861168,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "74c98543-ce04-43d8-a58c-267e372db698", "address": "fa:16:3e:d4:4d:0a", "network": {"id": "7024d1f6-a400-4482-9364-78ae3b6adc80", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1603258958-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c2f67da658d94fabbab82ee6a6171c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap74c98543-ce", "ovs_interfaceid": "74c98543-ce04-43d8-a58c-267e372db698", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:23:25 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-c871e8d9-7741-41d7-b8d8-c1f9e9176e92 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Converting VIF {"id": "74c98543-ce04-43d8-a58c-267e372db698", "address": "fa:16:3e:d4:4d:0a", "network": {"id": "7024d1f6-a400-4482-9364-78ae3b6adc80", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1603258958-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c2f67da658d94fabbab82ee6a6171c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap74c98543-ce", "ovs_interfaceid": "74c98543-ce04-43d8-a58c-267e372db698", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:23:25 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-c871e8d9-7741-41d7-b8d8-c1f9e9176e92 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:d4:4d:0a,bridge_name='br-int',has_traffic_filtering=True,id=74c98543-ce04-43d8-a58c-267e372db698,network=Network(7024d1f6-a400-4482-9364-78ae3b6adc80),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap74c98543-ce') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:23:25 user nova-compute[72351]: DEBUG os_vif [None req-c871e8d9-7741-41d7-b8d8-c1f9e9176e92 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:d4:4d:0a,bridge_name='br-int',has_traffic_filtering=True,id=74c98543-ce04-43d8-a58c-267e372db698,network=Network(7024d1f6-a400-4482-9364-78ae3b6adc80),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap74c98543-ce') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:23:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:23:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap74c98543-ce, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:23:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:23:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:23:25 user nova-compute[72351]: INFO os_vif [None req-c871e8d9-7741-41d7-b8d8-c1f9e9176e92 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:d4:4d:0a,bridge_name='br-int',has_traffic_filtering=True,id=74c98543-ce04-43d8-a58c-267e372db698,network=Network(7024d1f6-a400-4482-9364-78ae3b6adc80),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap74c98543-ce')
Aug 09 02:23:25 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-c871e8d9-7741-41d7-b8d8-c1f9e9176e92 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Deleting instance files /opt/stack/data/nova/instances/97b13ccd-15b4-44bf-b865-71b4da861168_del
Aug 09 02:23:25 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-c871e8d9-7741-41d7-b8d8-c1f9e9176e92 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Deletion of /opt/stack/data/nova/instances/97b13ccd-15b4-44bf-b865-71b4da861168_del complete
Aug 09 02:23:25 user nova-compute[72351]: INFO nova.compute.manager [None req-c871e8d9-7741-41d7-b8d8-c1f9e9176e92 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Took 0.92 seconds to destroy the instance on the hypervisor.
Aug 09 02:23:25 user nova-compute[72351]: DEBUG oslo.service.loopingcall [None req-c871e8d9-7741-41d7-b8d8-c1f9e9176e92 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72351) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Aug 09 02:23:25 user nova-compute[72351]: DEBUG nova.compute.manager [-] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Deallocating network for instance {{(pid=72351) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2263}}
Aug 09 02:23:25 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] deallocate_for_instance() {{(pid=72351) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1795}}
Aug 09 02:23:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:23:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:23:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:23:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:23:25 user nova-compute[72351]: DEBUG nova.compute.manager [req-36037c6a-5277-42ec-a485-64dfbf0318f6 req-0494d772-5792-45ba-9dd0-2fcb698470d7 service nova] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Received event network-vif-unplugged-3718df01-0e23-424c-8ab1-bf0fd9c7e121 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:23:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-36037c6a-5277-42ec-a485-64dfbf0318f6 req-0494d772-5792-45ba-9dd0-2fcb698470d7 service nova] Acquiring lock "74728ca8-34d4-45e6-a1f2-21c0952ff835-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:23:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-36037c6a-5277-42ec-a485-64dfbf0318f6 req-0494d772-5792-45ba-9dd0-2fcb698470d7 service nova] Lock "74728ca8-34d4-45e6-a1f2-21c0952ff835-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:23:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-36037c6a-5277-42ec-a485-64dfbf0318f6 req-0494d772-5792-45ba-9dd0-2fcb698470d7 service nova] Lock "74728ca8-34d4-45e6-a1f2-21c0952ff835-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:23:25 user nova-compute[72351]: DEBUG nova.compute.manager [req-36037c6a-5277-42ec-a485-64dfbf0318f6 req-0494d772-5792-45ba-9dd0-2fcb698470d7 service nova] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] No waiting events found dispatching network-vif-unplugged-3718df01-0e23-424c-8ab1-bf0fd9c7e121 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:23:25 user nova-compute[72351]: DEBUG nova.compute.manager [req-36037c6a-5277-42ec-a485-64dfbf0318f6 req-0494d772-5792-45ba-9dd0-2fcb698470d7 service nova] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Received event network-vif-unplugged-3718df01-0e23-424c-8ab1-bf0fd9c7e121 for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:23:26 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Instance destroyed successfully.
Aug 09 02:23:26 user nova-compute[72351]: DEBUG nova.objects.instance [None req-e7f95695-a02a-4cf1-8213-07527512cf25 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Lazy-loading 'resources' on Instance uuid 74728ca8-34d4-45e6-a1f2-21c0952ff835 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:23:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-e7f95695-a02a-4cf1-8213-07527512cf25 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:17:19Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-VolumesAdminNegativeTest-server-342268075',display_name='tempest-VolumesAdminNegativeTest-server-342268075',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-volumesadminnegativetest-server-342268075',id=4,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOAriZVJ2fue+G5VkQcpwuO1FraVLGHA1whj7NY2xEK0SFC2eroBPurQgFtB5AxEfAlnwcHpzvvI/FlO3wNg0bKdfmp7QzjOzEDrSqhn/OdHVQ7YHqei4WWY5uwxumXZ+g==',key_name='tempest-keypair-404137544',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:17:36Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='bc5e05fbd7f44f85bc3b1cc169dd03a9',ramdisk_id='',reservation_id='r-1g3ssrep',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-VolumesAdminNegativeTest-35684003',owner_user_name='tempest-VolumesAdminNegativeTest-35684003-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:17:36Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='fb9f73926e0d42f586e7c0055764d611',uuid=74728ca8-34d4-45e6-a1f2-21c0952ff835,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "3718df01-0e23-424c-8ab1-bf0fd9c7e121", "address": "fa:16:3e:7c:0b:0d", "network": {"id": "21efbfa4-f3a2-4317-89fc-3cc7b60fde97", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1919568535-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.239", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bc5e05fbd7f44f85bc3b1cc169dd03a9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3718df01-0e", "ovs_interfaceid": "3718df01-0e23-424c-8ab1-bf0fd9c7e121", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:23:26 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-e7f95695-a02a-4cf1-8213-07527512cf25 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Converting VIF {"id": "3718df01-0e23-424c-8ab1-bf0fd9c7e121", "address": "fa:16:3e:7c:0b:0d", "network": {"id": "21efbfa4-f3a2-4317-89fc-3cc7b60fde97", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1919568535-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.239", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bc5e05fbd7f44f85bc3b1cc169dd03a9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3718df01-0e", "ovs_interfaceid": "3718df01-0e23-424c-8ab1-bf0fd9c7e121", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:23:26 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-e7f95695-a02a-4cf1-8213-07527512cf25 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:7c:0b:0d,bridge_name='br-int',has_traffic_filtering=True,id=3718df01-0e23-424c-8ab1-bf0fd9c7e121,network=Network(21efbfa4-f3a2-4317-89fc-3cc7b60fde97),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3718df01-0e') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:23:26 user nova-compute[72351]: DEBUG os_vif [None req-e7f95695-a02a-4cf1-8213-07527512cf25 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:7c:0b:0d,bridge_name='br-int',has_traffic_filtering=True,id=3718df01-0e23-424c-8ab1-bf0fd9c7e121,network=Network(21efbfa4-f3a2-4317-89fc-3cc7b60fde97),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3718df01-0e') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:23:26 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:23:26 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap3718df01-0e, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:23:26 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:23:26 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:23:26 user nova-compute[72351]: INFO os_vif [None req-e7f95695-a02a-4cf1-8213-07527512cf25 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:7c:0b:0d,bridge_name='br-int',has_traffic_filtering=True,id=3718df01-0e23-424c-8ab1-bf0fd9c7e121,network=Network(21efbfa4-f3a2-4317-89fc-3cc7b60fde97),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3718df01-0e')
Aug 09 02:23:26 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-e7f95695-a02a-4cf1-8213-07527512cf25 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Deleting instance files /opt/stack/data/nova/instances/74728ca8-34d4-45e6-a1f2-21c0952ff835_del
Aug 09 02:23:26 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-e7f95695-a02a-4cf1-8213-07527512cf25 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Deletion of /opt/stack/data/nova/instances/74728ca8-34d4-45e6-a1f2-21c0952ff835_del complete
Aug 09 02:23:26 user nova-compute[72351]: INFO nova.compute.manager [None req-e7f95695-a02a-4cf1-8213-07527512cf25 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Took 0.97 seconds to destroy the instance on the hypervisor.
Aug 09 02:23:26 user nova-compute[72351]: DEBUG oslo.service.loopingcall [None req-e7f95695-a02a-4cf1-8213-07527512cf25 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72351) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Aug 09 02:23:26 user nova-compute[72351]: DEBUG nova.compute.manager [-] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Deallocating network for instance {{(pid=72351) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2263}}
Aug 09 02:23:26 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] deallocate_for_instance() {{(pid=72351) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1795}}
Aug 09 02:23:26 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:23:26 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Took 1.07 seconds to deallocate network for instance.
Aug 09 02:23:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c871e8d9-7741-41d7-b8d8-c1f9e9176e92 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:23:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c871e8d9-7741-41d7-b8d8-c1f9e9176e92 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:23:26 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-c871e8d9-7741-41d7-b8d8-c1f9e9176e92 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Refreshing inventories for resource provider 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:804}}
Aug 09 02:23:26 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-c871e8d9-7741-41d7-b8d8-c1f9e9176e92 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Updating ProviderTree inventory for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 from _refresh_and_get_inventory using data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) _refresh_and_get_inventory /opt/stack/nova/nova/scheduler/client/report.py:768}}
Aug 09 02:23:26 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-c871e8d9-7741-41d7-b8d8-c1f9e9176e92 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Updating inventory in ProviderTree for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 with inventory: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:176}}
Aug 09 02:23:26 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-c871e8d9-7741-41d7-b8d8-c1f9e9176e92 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Refreshing aggregate associations for resource provider 8eae9b37-1236-4e6d-91a5-b650de27b009, aggregates: None {{(pid=72351) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:813}}
Aug 09 02:23:26 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-c871e8d9-7741-41d7-b8d8-c1f9e9176e92 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Refreshing trait associations for resource provider 8eae9b37-1236-4e6d-91a5-b650de27b009, traits: COMPUTE_VIOMMU_MODEL_INTEL,COMPUTE_STORAGE_BUS_IDE,COMPUTE_GRAPHICS_MODEL_NONE,COMPUTE_NET_VIF_MODEL_E1000E,COMPUTE_IMAGE_TYPE_AKI,COMPUTE_VOLUME_ATTACH_WITH_TAG,COMPUTE_NET_VIF_MODEL_VMXNET3,COMPUTE_SOCKET_PCI_NUMA_AFFINITY,COMPUTE_ACCELERATORS,HW_CPU_X86_SSE41,COMPUTE_NET_ATTACH_INTERFACE_WITH_TAG,COMPUTE_GRAPHICS_MODEL_VGA,COMPUTE_GRAPHICS_MODEL_QXL,COMPUTE_VIOMMU_MODEL_AUTO,COMPUTE_SECURITY_UEFI_SECURE_BOOT,COMPUTE_NET_VIF_MODEL_E1000,COMPUTE_TRUSTED_CERTS,COMPUTE_VOLUME_MULTI_ATTACH,COMPUTE_STORAGE_BUS_FDC,COMPUTE_VOLUME_EXTEND,COMPUTE_GRAPHICS_MODEL_CIRRUS,COMPUTE_GRAPHICS_MODEL_VMVGA,COMPUTE_GRAPHICS_MODEL_BOCHS,COMPUTE_DEVICE_TAGGING,COMPUTE_STORAGE_BUS_SCSI,COMPUTE_STORAGE_BUS_VIRTIO,HW_CPU_X86_SSSE3,HW_CPU_X86_SSE2,COMPUTE_IMAGE_TYPE_AMI,COMPUTE_STORAGE_BUS_USB,COMPUTE_NET_VIF_MODEL_PCNET,COMPUTE_NET_VIF_MODEL_VIRTIO,COMPUTE_IMAGE_TYPE_ISO,HW_CPU_X86_SSE42,HW_CPU_X86_SSE,COMPUTE_IMAGE_TYPE_ARI,COMPUTE_IMAGE_TYPE_RAW,COMPUTE_IMAGE_TYPE_QCOW2,COMPUTE_NET_VIF_MODEL_NE2K_PCI,COMPUTE_NODE,HW_CPU_X86_MMX,COMPUTE_NET_VIF_MODEL_RTL8139,COMPUTE_GRAPHICS_MODEL_VIRTIO,COMPUTE_NET_VIF_MODEL_SPAPR_VLAN,COMPUTE_STORAGE_BUS_SATA,COMPUTE_NET_ATTACH_INTERFACE,COMPUTE_RESCUE_BFV {{(pid=72351) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:825}}
Aug 09 02:23:26 user nova-compute[72351]: DEBUG nova.compute.manager [req-77f6cacb-5fb1-4712-a14f-ba5228f9157d req-fbfe4e3d-a6cd-42c1-9573-708438a8d21b service nova] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Received event network-vif-plugged-74c98543-ce04-43d8-a58c-267e372db698 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:23:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-77f6cacb-5fb1-4712-a14f-ba5228f9157d req-fbfe4e3d-a6cd-42c1-9573-708438a8d21b service nova] Acquiring lock "97b13ccd-15b4-44bf-b865-71b4da861168-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:23:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-77f6cacb-5fb1-4712-a14f-ba5228f9157d req-fbfe4e3d-a6cd-42c1-9573-708438a8d21b service nova] Lock "97b13ccd-15b4-44bf-b865-71b4da861168-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:23:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-77f6cacb-5fb1-4712-a14f-ba5228f9157d req-fbfe4e3d-a6cd-42c1-9573-708438a8d21b service nova] Lock "97b13ccd-15b4-44bf-b865-71b4da861168-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:23:26 user nova-compute[72351]: DEBUG nova.compute.manager [req-77f6cacb-5fb1-4712-a14f-ba5228f9157d req-fbfe4e3d-a6cd-42c1-9573-708438a8d21b service nova] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] No waiting events found dispatching network-vif-plugged-74c98543-ce04-43d8-a58c-267e372db698 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:23:26 user nova-compute[72351]: WARNING nova.compute.manager [req-77f6cacb-5fb1-4712-a14f-ba5228f9157d req-fbfe4e3d-a6cd-42c1-9573-708438a8d21b service nova] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Received unexpected event network-vif-plugged-74c98543-ce04-43d8-a58c-267e372db698 for instance with vm_state deleted and task_state None.
Aug 09 02:23:26 user nova-compute[72351]: DEBUG nova.compute.manager [req-77f6cacb-5fb1-4712-a14f-ba5228f9157d req-fbfe4e3d-a6cd-42c1-9573-708438a8d21b service nova] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Received event network-vif-deleted-74c98543-ce04-43d8-a58c-267e372db698 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:23:27 user nova-compute[72351]: INFO nova.compute.manager [None req-51f6ac17-0aff-4407-8eda-a100f56e8ae9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Rescuing
Aug 09 02:23:27 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-51f6ac17-0aff-4407-8eda-a100f56e8ae9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Acquiring lock "refresh_cache-eb86cb49-2018-42c2-bd97-c662b829775d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:23:27 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-51f6ac17-0aff-4407-8eda-a100f56e8ae9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Acquired lock "refresh_cache-eb86cb49-2018-42c2-bd97-c662b829775d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:23:27 user nova-compute[72351]: DEBUG nova.network.neutron [None req-51f6ac17-0aff-4407-8eda-a100f56e8ae9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:23:27 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-c871e8d9-7741-41d7-b8d8-c1f9e9176e92 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:23:27 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-c871e8d9-7741-41d7-b8d8-c1f9e9176e92 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:23:27 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c871e8d9-7741-41d7-b8d8-c1f9e9176e92 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.671s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:23:27 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:23:27 user nova-compute[72351]: INFO nova.scheduler.client.report [None req-c871e8d9-7741-41d7-b8d8-c1f9e9176e92 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Deleted allocations for instance 97b13ccd-15b4-44bf-b865-71b4da861168
Aug 09 02:23:27 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Took 1.04 seconds to deallocate network for instance.
Aug 09 02:23:27 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e7f95695-a02a-4cf1-8213-07527512cf25 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:23:27 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e7f95695-a02a-4cf1-8213-07527512cf25 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:23:27 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c871e8d9-7741-41d7-b8d8-c1f9e9176e92 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "97b13ccd-15b4-44bf-b865-71b4da861168" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.921s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:23:27 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-d91b789d-1e8d-460a-ba76-ac6da1ce7f03 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Acquiring lock "2d78475c-107d-4256-b12e-dbe9c8f3b9b2" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:23:27 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-d91b789d-1e8d-460a-ba76-ac6da1ce7f03 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Lock "2d78475c-107d-4256-b12e-dbe9c8f3b9b2" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:23:27 user nova-compute[72351]: INFO nova.compute.manager [None req-d91b789d-1e8d-460a-ba76-ac6da1ce7f03 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Detaching volume 41ceb107-f270-4b7a-a2b0-14c2dbdd5497
Aug 09 02:23:27 user nova-compute[72351]: INFO nova.virt.block_device [None req-d91b789d-1e8d-460a-ba76-ac6da1ce7f03 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Attempting to driver detach volume 41ceb107-f270-4b7a-a2b0-14c2dbdd5497 from mountpoint /dev/vdb
Aug 09 02:23:27 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-d91b789d-1e8d-460a-ba76-ac6da1ce7f03 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Attempting to detach device vdb from instance 2d78475c-107d-4256-b12e-dbe9c8f3b9b2 from the persistent domain config. {{(pid=72351) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2477}}
Aug 09 02:23:27 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-d91b789d-1e8d-460a-ba76-ac6da1ce7f03 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 02:23:27 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:23:27 user nova-compute[72351]:   <source dev="/dev/sdc"/>
Aug 09 02:23:27 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:23:27 user nova-compute[72351]:   <serial>41ceb107-f270-4b7a-a2b0-14c2dbdd5497</serial>
Aug 09 02:23:27 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Aug 09 02:23:27 user nova-compute[72351]: </disk>
Aug 09 02:23:27 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 02:23:27 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-d91b789d-1e8d-460a-ba76-ac6da1ce7f03 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Successfully detached device vdb from instance 2d78475c-107d-4256-b12e-dbe9c8f3b9b2 from the persistent domain config.
Aug 09 02:23:27 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-d91b789d-1e8d-460a-ba76-ac6da1ce7f03 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 2d78475c-107d-4256-b12e-dbe9c8f3b9b2 from the live domain config. {{(pid=72351) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2513}}
Aug 09 02:23:27 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-d91b789d-1e8d-460a-ba76-ac6da1ce7f03 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 02:23:27 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:23:27 user nova-compute[72351]:   <source dev="/dev/sdc"/>
Aug 09 02:23:27 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:23:27 user nova-compute[72351]:   <serial>41ceb107-f270-4b7a-a2b0-14c2dbdd5497</serial>
Aug 09 02:23:27 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Aug 09 02:23:27 user nova-compute[72351]: </disk>
Aug 09 02:23:27 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 02:23:27 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-e7f95695-a02a-4cf1-8213-07527512cf25 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:23:27 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-e7f95695-a02a-4cf1-8213-07527512cf25 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:23:27 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e7f95695-a02a-4cf1-8213-07527512cf25 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.300s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:23:27 user nova-compute[72351]: INFO nova.scheduler.client.report [None req-e7f95695-a02a-4cf1-8213-07527512cf25 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Deleted allocations for instance 74728ca8-34d4-45e6-a1f2-21c0952ff835
Aug 09 02:23:27 user nova-compute[72351]: DEBUG nova.compute.manager [req-c3719735-6ff9-4003-9972-ac9b917e05c6 req-a5fcab75-7d97-4887-8bb6-8d15324f6636 service nova] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Received event network-vif-plugged-3718df01-0e23-424c-8ab1-bf0fd9c7e121 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:23:27 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c3719735-6ff9-4003-9972-ac9b917e05c6 req-a5fcab75-7d97-4887-8bb6-8d15324f6636 service nova] Acquiring lock "74728ca8-34d4-45e6-a1f2-21c0952ff835-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:23:27 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c3719735-6ff9-4003-9972-ac9b917e05c6 req-a5fcab75-7d97-4887-8bb6-8d15324f6636 service nova] Lock "74728ca8-34d4-45e6-a1f2-21c0952ff835-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:23:27 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c3719735-6ff9-4003-9972-ac9b917e05c6 req-a5fcab75-7d97-4887-8bb6-8d15324f6636 service nova] Lock "74728ca8-34d4-45e6-a1f2-21c0952ff835-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:23:27 user nova-compute[72351]: DEBUG nova.compute.manager [req-c3719735-6ff9-4003-9972-ac9b917e05c6 req-a5fcab75-7d97-4887-8bb6-8d15324f6636 service nova] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] No waiting events found dispatching network-vif-plugged-3718df01-0e23-424c-8ab1-bf0fd9c7e121 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:23:27 user nova-compute[72351]: WARNING nova.compute.manager [req-c3719735-6ff9-4003-9972-ac9b917e05c6 req-a5fcab75-7d97-4887-8bb6-8d15324f6636 service nova] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Received unexpected event network-vif-plugged-3718df01-0e23-424c-8ab1-bf0fd9c7e121 for instance with vm_state deleted and task_state None.
Aug 09 02:23:27 user nova-compute[72351]: DEBUG nova.compute.manager [req-c3719735-6ff9-4003-9972-ac9b917e05c6 req-a5fcab75-7d97-4887-8bb6-8d15324f6636 service nova] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Received event network-vif-deleted-3718df01-0e23-424c-8ab1-bf0fd9c7e121 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:23:27 user nova-compute[72351]: DEBUG nova.network.neutron [None req-51f6ac17-0aff-4407-8eda-a100f56e8ae9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Updating instance_info_cache with network_info: [{"id": "5f97db70-9e5b-4d42-b765-b832fb522f41", "address": "fa:16:3e:dc:da:c7", "network": {"id": "49e8751c-e119-430f-9110-3dc6069b1589", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-899275135-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3aea1e66805146f7a8cd1d97e64a24af", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5f97db70-9e", "ovs_interfaceid": "5f97db70-9e5b-4d42-b765-b832fb522f41", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:23:27 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-51f6ac17-0aff-4407-8eda-a100f56e8ae9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Releasing lock "refresh_cache-eb86cb49-2018-42c2-bd97-c662b829775d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:23:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:23:27 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e7f95695-a02a-4cf1-8213-07527512cf25 tempest-VolumesAdminNegativeTest-35684003 tempest-VolumesAdminNegativeTest-35684003-project-member] Lock "74728ca8-34d4-45e6-a1f2-21c0952ff835" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.528s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:23:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:23:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:23:28 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Received event <DeviceRemovedEvent: 1691547808.0398152, 2d78475c-107d-4256-b12e-dbe9c8f3b9b2 => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=72351) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2360}}
Aug 09 02:23:28 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-d91b789d-1e8d-460a-ba76-ac6da1ce7f03 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 2d78475c-107d-4256-b12e-dbe9c8f3b9b2 {{(pid=72351) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2589}}
Aug 09 02:23:28 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-d91b789d-1e8d-460a-ba76-ac6da1ce7f03 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Successfully detached device vdb from instance 2d78475c-107d-4256-b12e-dbe9c8f3b9b2 from the live domain config.
Aug 09 02:23:28 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-d91b789d-1e8d-460a-ba76-ac6da1ce7f03 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 02:23:28 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-d91b789d-1e8d-460a-ba76-ac6da1ce7f03 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'YaNtwOCg', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc'}, None), 'kwargs': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:23:28 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-d91b789d-1e8d-460a-ba76-ac6da1ce7f03 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:23:28 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-d91b789d-1e8d-460a-ba76-ac6da1ce7f03 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:23:28 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-d91b789d-1e8d-460a-ba76-ac6da1ce7f03 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:23:28 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-d91b789d-1e8d-460a-ba76-ac6da1ce7f03 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 02:23:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:23:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:23:28 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[8855f38a-921b-47d7-b063-189233745f03]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:10b01378-384d-464f-8960-e852bfceae76\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:23:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:23:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:23:28 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[2c178ce7-9be6-4051-a8c9-bce14cb1907e]: (4, ('tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497 (non-flash)\ntcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)\ntcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:10b01378-384d-464f-8960-e852bfceae76 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:23:28 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-d91b789d-1e8d-460a-ba76-ac6da1ce7f03 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497 (non-flash)
Aug 09 02:23:28 user nova-compute[72351]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)
Aug 09 02:23:28 user nova-compute[72351]: tcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:10b01378-384d-464f-8960-e852bfceae76 (non-flash)
Aug 09 02:23:28 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:23:28 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-d91b789d-1e8d-460a-ba76-ac6da1ce7f03 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] iscsi session list stdout=tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497 (non-flash)
Aug 09 02:23:28 user nova-compute[72351]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)
Aug 09 02:23:28 user nova-compute[72351]: tcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:10b01378-384d-464f-8960-e852bfceae76 (non-flash)
Aug 09 02:23:28 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:23:28 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-d91b789d-1e8d-460a-ba76-ac6da1ce7f03 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fcfbc746440>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497'): ({'sdc'}, set())}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 02:23:28 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-d91b789d-1e8d-460a-ba76-ac6da1ce7f03 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Removing single pathed devices sdc {{(pid=72351) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Aug 09 02:23:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:23:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:23:28 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ee6c2f85-8edd-4448-88ac-def7578cad50]: (4, ('path checker states:\nup                  2\npending             1\n\npaths: 0\nbusy: False\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:23:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdc {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:23:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdc" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:23:28 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[394851ae-fa2e-422d-a92b-72026c132532]: (4, ('ok\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:23:28 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-d91b789d-1e8d-460a-ba76-ac6da1ce7f03 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Flushing IO for device /dev/sdc {{(pid=72351) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Aug 09 02:23:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdc {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:23:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdc" returned: 0 in 0.014s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:23:28 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[b1414fbe-8f2d-4576-a307-f2e8b9f9ded6]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:23:28 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-d91b789d-1e8d-460a-ba76-ac6da1ce7f03 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Remove SCSI device /dev/sdc with /sys/block/sdc/device/delete {{(pid=72351) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Aug 09 02:23:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdc/device/delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:23:28 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:23:28 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:23:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdc/device/delete" returned: 0 in 0.060s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:23:28 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[2764a718-b848-4d10-a9bc-1d28df38f0f1]: (4, ('1', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:23:28 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-d91b789d-1e8d-460a-ba76-ac6da1ce7f03 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Checking to see if SCSI volumes sdc have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Aug 09 02:23:28 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-d91b789d-1e8d-460a-ba76-ac6da1ce7f03 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] SCSI volumes sdc have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Aug 09 02:23:28 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-d91b789d-1e8d-460a-ba76-ac6da1ce7f03 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497')] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 02:23:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:23:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:23:28 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[c733a3a6-27d9-450d-9cbb-d701f69d02be]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:23:28 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-d91b789d-1e8d-460a-ba76-ac6da1ce7f03 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:23:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497 -p 172.16.0.220:3260 --logout {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:23:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497 -p 172.16.0.220:3260 --logout" returned: 0 in 0.080s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:23:28 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[c75ce22c-e2f0-4705-ab55-33256a2dca33]: (4, ('Logging out of session [sid: 15, target: iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497, portal: 172.16.0.220,3260]\nLogout of [sid: 15, target: iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:23:28 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-d91b789d-1e8d-460a-ba76-ac6da1ce7f03 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 15, target: iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497, portal: 172.16.0.220,3260]
Aug 09 02:23:28 user nova-compute[72351]: Logout of [sid: 15, target: iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497, portal: 172.16.0.220,3260] successful.
Aug 09 02:23:28 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:23:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497 -p 172.16.0.220:3260 --op delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:23:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:41ceb107-f270-4b7a-a2b0-14c2dbdd5497 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.026s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:23:28 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[defe4e32-e7e7-4a9f-bc40-eea038851207]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:23:28 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-d91b789d-1e8d-460a-ba76-ac6da1ce7f03 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:23:28 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-d91b789d-1e8d-460a-ba76-ac6da1ce7f03 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.326s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:23:28 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-d91b789d-1e8d-460a-ba76-ac6da1ce7f03 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] <== disconnect_volume: return (328ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:23:28 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-d91b789d-1e8d-460a-ba76-ac6da1ce7f03 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 02:23:28 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:23:28 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:23:28 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Instance destroyed successfully.
Aug 09 02:23:28 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-51f6ac17-0aff-4407-8eda-a100f56e8ae9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Attempting rescue
Aug 09 02:23:28 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-51f6ac17-0aff-4407-8eda-a100f56e8ae9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] rescue generated disk_info: {'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'disk.rescue': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} {{(pid=72351) rescue /opt/stack/nova/nova/virt/libvirt/driver.py:4308}}
Aug 09 02:23:28 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-51f6ac17-0aff-4407-8eda-a100f56e8ae9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Instance directory exists: not creating {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4713}}
Aug 09 02:23:28 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-51f6ac17-0aff-4407-8eda-a100f56e8ae9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Creating image(s)
Aug 09 02:23:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-51f6ac17-0aff-4407-8eda-a100f56e8ae9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Acquiring lock "/opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:23:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-51f6ac17-0aff-4407-8eda-a100f56e8ae9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "/opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:23:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-51f6ac17-0aff-4407-8eda-a100f56e8ae9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "/opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:23:28 user nova-compute[72351]: DEBUG nova.objects.instance [None req-51f6ac17-0aff-4407-8eda-a100f56e8ae9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lazy-loading 'trusted_certs' on Instance uuid eb86cb49-2018-42c2-bd97-c662b829775d {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:23:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-51f6ac17-0aff-4407-8eda-a100f56e8ae9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Acquiring lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:23:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-51f6ac17-0aff-4407-8eda-a100f56e8ae9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:23:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-51f6ac17-0aff-4407-8eda-a100f56e8ae9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:23:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-51f6ac17-0aff-4407-8eda-a100f56e8ae9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.180s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:23:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-51f6ac17-0aff-4407-8eda-a100f56e8ae9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.rescue {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:23:29 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-51f6ac17-0aff-4407-8eda-a100f56e8ae9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.rescue" returned: 0 in 0.060s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:23:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-51f6ac17-0aff-4407-8eda-a100f56e8ae9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.247s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:23:29 user nova-compute[72351]: DEBUG nova.objects.instance [None req-51f6ac17-0aff-4407-8eda-a100f56e8ae9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lazy-loading 'migration_context' on Instance uuid eb86cb49-2018-42c2-bd97-c662b829775d {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:23:29 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-51f6ac17-0aff-4407-8eda-a100f56e8ae9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Created local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4851}}
Aug 09 02:23:29 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-51f6ac17-0aff-4407-8eda-a100f56e8ae9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Start _get_guest_xml network_info=[{"id": "5f97db70-9e5b-4d42-b765-b832fb522f41", "address": "fa:16:3e:dc:da:c7", "network": {"id": "49e8751c-e119-430f-9110-3dc6069b1589", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-899275135-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [], "label": "tempest-ServerRescueNegativeTestJSON-899275135-network", "vif_mac": "fa:16:3e:dc:da:c7"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3aea1e66805146f7a8cd1d97e64a24af", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5f97db70-9e", "ovs_interfaceid": "5f97db70-9e5b-4d42-b765-b832fb522f41", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'disk.rescue': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>) rescue={'image_id': '18ccda7f-31d3-4978-ab9d-4ed2f74eff94', 'kernel_id': '', 'ramdisk_id': ''} block_device_info=None {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:23:29 user nova-compute[72351]: DEBUG nova.objects.instance [None req-51f6ac17-0aff-4407-8eda-a100f56e8ae9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lazy-loading 'resources' on Instance uuid eb86cb49-2018-42c2-bd97-c662b829775d {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:23:29 user nova-compute[72351]: DEBUG nova.objects.instance [None req-51f6ac17-0aff-4407-8eda-a100f56e8ae9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lazy-loading 'numa_topology' on Instance uuid eb86cb49-2018-42c2-bd97-c662b829775d {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:23:29 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-51f6ac17-0aff-4407-8eda-a100f56e8ae9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:23:29 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-51f6ac17-0aff-4407-8eda-a100f56e8ae9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:23:29 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-51f6ac17-0aff-4407-8eda-a100f56e8ae9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:23:29 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-51f6ac17-0aff-4407-8eda-a100f56e8ae9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:23:29 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-51f6ac17-0aff-4407-8eda-a100f56e8ae9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:23:29 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-51f6ac17-0aff-4407-8eda-a100f56e8ae9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:23:29 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-51f6ac17-0aff-4407-8eda-a100f56e8ae9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:23:29 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-51f6ac17-0aff-4407-8eda-a100f56e8ae9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:23:29 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-51f6ac17-0aff-4407-8eda-a100f56e8ae9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:23:29 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-51f6ac17-0aff-4407-8eda-a100f56e8ae9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:23:29 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-51f6ac17-0aff-4407-8eda-a100f56e8ae9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:23:29 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-51f6ac17-0aff-4407-8eda-a100f56e8ae9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:23:29 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-51f6ac17-0aff-4407-8eda-a100f56e8ae9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:23:29 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-51f6ac17-0aff-4407-8eda-a100f56e8ae9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:23:29 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-51f6ac17-0aff-4407-8eda-a100f56e8ae9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:23:29 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-51f6ac17-0aff-4407-8eda-a100f56e8ae9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:23:29 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-51f6ac17-0aff-4407-8eda-a100f56e8ae9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:23:29 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-51f6ac17-0aff-4407-8eda-a100f56e8ae9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:23:29 user nova-compute[72351]: DEBUG nova.objects.instance [None req-51f6ac17-0aff-4407-8eda-a100f56e8ae9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lazy-loading 'vcpu_model' on Instance uuid eb86cb49-2018-42c2-bd97-c662b829775d {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:23:29 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-51f6ac17-0aff-4407-8eda-a100f56e8ae9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:21:35Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerRescueNegativeTestJSON-server-69135895',display_name='tempest-ServerRescueNegativeTestJSON-server-69135895',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serverrescuenegativetestjson-server-69135895',id=17,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:21:44Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='3aea1e66805146f7a8cd1d97e64a24af',ramdisk_id='',reservation_id='r-b0vsogov',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerRescueNegativeTestJSON-289364438',owner_user_name='tempest-ServerRescueNegativeTestJSON-289364438-project-member'},tags=<?>,task_state='rescuing',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:21:45Z,user_data=None,user_id='419b28d5a46c481db435715a30ab2614',uuid=eb86cb49-2018-42c2-bd97-c662b829775d,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "5f97db70-9e5b-4d42-b765-b832fb522f41", "address": "fa:16:3e:dc:da:c7", "network": {"id": "49e8751c-e119-430f-9110-3dc6069b1589", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-899275135-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [], "label": "tempest-ServerRescueNegativeTestJSON-899275135-network", "vif_mac": "fa:16:3e:dc:da:c7"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3aea1e66805146f7a8cd1d97e64a24af", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5f97db70-9e", "ovs_interfaceid": "5f97db70-9e5b-4d42-b765-b832fb522f41", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:23:29 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-51f6ac17-0aff-4407-8eda-a100f56e8ae9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Converting VIF {"id": "5f97db70-9e5b-4d42-b765-b832fb522f41", "address": "fa:16:3e:dc:da:c7", "network": {"id": "49e8751c-e119-430f-9110-3dc6069b1589", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-899275135-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [], "label": "tempest-ServerRescueNegativeTestJSON-899275135-network", "vif_mac": "fa:16:3e:dc:da:c7"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3aea1e66805146f7a8cd1d97e64a24af", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5f97db70-9e", "ovs_interfaceid": "5f97db70-9e5b-4d42-b765-b832fb522f41", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:23:29 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-51f6ac17-0aff-4407-8eda-a100f56e8ae9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:dc:da:c7,bridge_name='br-int',has_traffic_filtering=True,id=5f97db70-9e5b-4d42-b765-b832fb522f41,network=Network(49e8751c-e119-430f-9110-3dc6069b1589),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap5f97db70-9e') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:23:29 user nova-compute[72351]: DEBUG nova.objects.instance [None req-51f6ac17-0aff-4407-8eda-a100f56e8ae9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lazy-loading 'pci_devices' on Instance uuid eb86cb49-2018-42c2-bd97-c662b829775d {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:23:29 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-51f6ac17-0aff-4407-8eda-a100f56e8ae9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:23:29 user nova-compute[72351]:   <uuid>eb86cb49-2018-42c2-bd97-c662b829775d</uuid>
Aug 09 02:23:29 user nova-compute[72351]:   <name>instance-00000011</name>
Aug 09 02:23:29 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:23:29 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:23:29 user nova-compute[72351]:   <metadata>
Aug 09 02:23:29 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:23:29 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:23:29 user nova-compute[72351]:       <nova:name>tempest-ServerRescueNegativeTestJSON-server-69135895</nova:name>
Aug 09 02:23:29 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:23:29</nova:creationTime>
Aug 09 02:23:29 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:23:29 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:23:29 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:23:29 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:23:29 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:23:29 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:23:29 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:23:29 user nova-compute[72351]:       <nova:owner>
Aug 09 02:23:29 user nova-compute[72351]:         <nova:user uuid="419b28d5a46c481db435715a30ab2614">tempest-ServerRescueNegativeTestJSON-289364438-project-member</nova:user>
Aug 09 02:23:29 user nova-compute[72351]:         <nova:project uuid="3aea1e66805146f7a8cd1d97e64a24af">tempest-ServerRescueNegativeTestJSON-289364438</nova:project>
Aug 09 02:23:29 user nova-compute[72351]:       </nova:owner>
Aug 09 02:23:29 user nova-compute[72351]:       <nova:root type="image" uuid="18ccda7f-31d3-4978-ab9d-4ed2f74eff94"/>
Aug 09 02:23:29 user nova-compute[72351]:       <nova:ports>
Aug 09 02:23:29 user nova-compute[72351]:         <nova:port uuid="5f97db70-9e5b-4d42-b765-b832fb522f41">
Aug 09 02:23:29 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.5" ipVersion="4"/>
Aug 09 02:23:29 user nova-compute[72351]:         </nova:port>
Aug 09 02:23:29 user nova-compute[72351]:       </nova:ports>
Aug 09 02:23:29 user nova-compute[72351]:     </nova:instance>
Aug 09 02:23:29 user nova-compute[72351]:   </metadata>
Aug 09 02:23:29 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:23:29 user nova-compute[72351]:     <system>
Aug 09 02:23:29 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:23:29 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:23:29 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:23:29 user nova-compute[72351]:       <entry name="serial">eb86cb49-2018-42c2-bd97-c662b829775d</entry>
Aug 09 02:23:29 user nova-compute[72351]:       <entry name="uuid">eb86cb49-2018-42c2-bd97-c662b829775d</entry>
Aug 09 02:23:29 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:23:29 user nova-compute[72351]:     </system>
Aug 09 02:23:29 user nova-compute[72351]:   </sysinfo>
Aug 09 02:23:29 user nova-compute[72351]:   <os>
Aug 09 02:23:29 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:23:29 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:23:29 user nova-compute[72351]:   </os>
Aug 09 02:23:29 user nova-compute[72351]:   <features>
Aug 09 02:23:29 user nova-compute[72351]:     <acpi/>
Aug 09 02:23:29 user nova-compute[72351]:     <apic/>
Aug 09 02:23:29 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:23:29 user nova-compute[72351]:   </features>
Aug 09 02:23:29 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:23:29 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:23:29 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:23:29 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:23:29 user nova-compute[72351]:   </clock>
Aug 09 02:23:29 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:23:29 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:23:29 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:23:29 user nova-compute[72351]:   </cpu>
Aug 09 02:23:29 user nova-compute[72351]:   <devices>
Aug 09 02:23:29 user nova-compute[72351]:     <disk type="file" device="disk">
Aug 09 02:23:29 user nova-compute[72351]:       <driver name="qemu" type="qcow2" cache="none"/>
Aug 09 02:23:29 user nova-compute[72351]:       <source file="/opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.rescue"/>
Aug 09 02:23:29 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:23:29 user nova-compute[72351]:     </disk>
Aug 09 02:23:29 user nova-compute[72351]:     <disk type="file" device="disk">
Aug 09 02:23:29 user nova-compute[72351]:       <driver name="qemu" type="qcow2" cache="none"/>
Aug 09 02:23:29 user nova-compute[72351]:       <source file="/opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk"/>
Aug 09 02:23:29 user nova-compute[72351]:       <target dev="vdb" bus="virtio"/>
Aug 09 02:23:29 user nova-compute[72351]:     </disk>
Aug 09 02:23:29 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:23:29 user nova-compute[72351]:       <mac address="fa:16:3e:dc:da:c7"/>
Aug 09 02:23:29 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:23:29 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:23:29 user nova-compute[72351]:       <target dev="tap5f97db70-9e"/>
Aug 09 02:23:29 user nova-compute[72351]:     </interface>
Aug 09 02:23:29 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:23:29 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/console.log" append="off"/>
Aug 09 02:23:29 user nova-compute[72351]:     </serial>
Aug 09 02:23:29 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:23:29 user nova-compute[72351]:     <video>
Aug 09 02:23:29 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:23:29 user nova-compute[72351]:     </video>
Aug 09 02:23:29 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:23:29 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:23:29 user nova-compute[72351]:     </rng>
Aug 09 02:23:29 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:23:29 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:23:29 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:23:29 user nova-compute[72351]:     </memballoon>
Aug 09 02:23:29 user nova-compute[72351]:   </devices>
Aug 09 02:23:29 user nova-compute[72351]: </domain>
Aug 09 02:23:29 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:23:29 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Instance destroyed successfully.
Aug 09 02:23:29 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-51f6ac17-0aff-4407-8eda-a100f56e8ae9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:23:29 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-51f6ac17-0aff-4407-8eda-a100f56e8ae9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:23:29 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-51f6ac17-0aff-4407-8eda-a100f56e8ae9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] No VIF found with MAC fa:16:3e:dc:da:c7, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:23:29 user nova-compute[72351]: DEBUG nova.objects.instance [None req-d91b789d-1e8d-460a-ba76-ac6da1ce7f03 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Lazy-loading 'flavor' on Instance uuid 2d78475c-107d-4256-b12e-dbe9c8f3b9b2 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:23:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-d91b789d-1e8d-460a-ba76-ac6da1ce7f03 tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Lock "2d78475c-107d-4256-b12e-dbe9c8f3b9b2" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.869s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:23:29 user nova-compute[72351]: DEBUG nova.compute.manager [req-aeaa8d92-99e6-4421-868c-b2c7983c21df req-ecb628e9-5b91-4aff-afdb-985258e76e32 service nova] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Received event network-vif-unplugged-5f97db70-9e5b-4d42-b765-b832fb522f41 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:23:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-aeaa8d92-99e6-4421-868c-b2c7983c21df req-ecb628e9-5b91-4aff-afdb-985258e76e32 service nova] Acquiring lock "eb86cb49-2018-42c2-bd97-c662b829775d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:23:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-aeaa8d92-99e6-4421-868c-b2c7983c21df req-ecb628e9-5b91-4aff-afdb-985258e76e32 service nova] Lock "eb86cb49-2018-42c2-bd97-c662b829775d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:23:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-aeaa8d92-99e6-4421-868c-b2c7983c21df req-ecb628e9-5b91-4aff-afdb-985258e76e32 service nova] Lock "eb86cb49-2018-42c2-bd97-c662b829775d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:23:29 user nova-compute[72351]: DEBUG nova.compute.manager [req-aeaa8d92-99e6-4421-868c-b2c7983c21df req-ecb628e9-5b91-4aff-afdb-985258e76e32 service nova] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] No waiting events found dispatching network-vif-unplugged-5f97db70-9e5b-4d42-b765-b832fb522f41 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:23:29 user nova-compute[72351]: WARNING nova.compute.manager [req-aeaa8d92-99e6-4421-868c-b2c7983c21df req-ecb628e9-5b91-4aff-afdb-985258e76e32 service nova] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Received unexpected event network-vif-unplugged-5f97db70-9e5b-4d42-b765-b832fb522f41 for instance with vm_state active and task_state rescuing.
Aug 09 02:23:29 user nova-compute[72351]: DEBUG nova.compute.manager [req-aeaa8d92-99e6-4421-868c-b2c7983c21df req-ecb628e9-5b91-4aff-afdb-985258e76e32 service nova] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Received event network-vif-plugged-5f97db70-9e5b-4d42-b765-b832fb522f41 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:23:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-aeaa8d92-99e6-4421-868c-b2c7983c21df req-ecb628e9-5b91-4aff-afdb-985258e76e32 service nova] Acquiring lock "eb86cb49-2018-42c2-bd97-c662b829775d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:23:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-aeaa8d92-99e6-4421-868c-b2c7983c21df req-ecb628e9-5b91-4aff-afdb-985258e76e32 service nova] Lock "eb86cb49-2018-42c2-bd97-c662b829775d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:23:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-aeaa8d92-99e6-4421-868c-b2c7983c21df req-ecb628e9-5b91-4aff-afdb-985258e76e32 service nova] Lock "eb86cb49-2018-42c2-bd97-c662b829775d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:23:29 user nova-compute[72351]: DEBUG nova.compute.manager [req-aeaa8d92-99e6-4421-868c-b2c7983c21df req-ecb628e9-5b91-4aff-afdb-985258e76e32 service nova] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] No waiting events found dispatching network-vif-plugged-5f97db70-9e5b-4d42-b765-b832fb522f41 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:23:29 user nova-compute[72351]: WARNING nova.compute.manager [req-aeaa8d92-99e6-4421-868c-b2c7983c21df req-ecb628e9-5b91-4aff-afdb-985258e76e32 service nova] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Received unexpected event network-vif-plugged-5f97db70-9e5b-4d42-b765-b832fb522f41 for instance with vm_state active and task_state rescuing.
Aug 09 02:23:30 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:23:30 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:23:30 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:23:30 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:23:31 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:23:31 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:23:31 user nova-compute[72351]: DEBUG nova.compute.manager [req-b8da64f1-9e3e-4016-9043-9095944addc1 req-b0fa3aec-4557-46b8-8e15-2dfc1235cc97 service nova] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Received event network-vif-plugged-5f97db70-9e5b-4d42-b765-b832fb522f41 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:23:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-b8da64f1-9e3e-4016-9043-9095944addc1 req-b0fa3aec-4557-46b8-8e15-2dfc1235cc97 service nova] Acquiring lock "eb86cb49-2018-42c2-bd97-c662b829775d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:23:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-b8da64f1-9e3e-4016-9043-9095944addc1 req-b0fa3aec-4557-46b8-8e15-2dfc1235cc97 service nova] Lock "eb86cb49-2018-42c2-bd97-c662b829775d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:23:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-b8da64f1-9e3e-4016-9043-9095944addc1 req-b0fa3aec-4557-46b8-8e15-2dfc1235cc97 service nova] Lock "eb86cb49-2018-42c2-bd97-c662b829775d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:23:31 user nova-compute[72351]: DEBUG nova.compute.manager [req-b8da64f1-9e3e-4016-9043-9095944addc1 req-b0fa3aec-4557-46b8-8e15-2dfc1235cc97 service nova] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] No waiting events found dispatching network-vif-plugged-5f97db70-9e5b-4d42-b765-b832fb522f41 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:23:31 user nova-compute[72351]: WARNING nova.compute.manager [req-b8da64f1-9e3e-4016-9043-9095944addc1 req-b0fa3aec-4557-46b8-8e15-2dfc1235cc97 service nova] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Received unexpected event network-vif-plugged-5f97db70-9e5b-4d42-b765-b832fb522f41 for instance with vm_state active and task_state rescuing.
Aug 09 02:23:31 user nova-compute[72351]: DEBUG nova.compute.manager [req-b8da64f1-9e3e-4016-9043-9095944addc1 req-b0fa3aec-4557-46b8-8e15-2dfc1235cc97 service nova] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Received event network-vif-plugged-5f97db70-9e5b-4d42-b765-b832fb522f41 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:23:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-b8da64f1-9e3e-4016-9043-9095944addc1 req-b0fa3aec-4557-46b8-8e15-2dfc1235cc97 service nova] Acquiring lock "eb86cb49-2018-42c2-bd97-c662b829775d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:23:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-b8da64f1-9e3e-4016-9043-9095944addc1 req-b0fa3aec-4557-46b8-8e15-2dfc1235cc97 service nova] Lock "eb86cb49-2018-42c2-bd97-c662b829775d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:23:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-b8da64f1-9e3e-4016-9043-9095944addc1 req-b0fa3aec-4557-46b8-8e15-2dfc1235cc97 service nova] Lock "eb86cb49-2018-42c2-bd97-c662b829775d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:23:31 user nova-compute[72351]: DEBUG nova.compute.manager [req-b8da64f1-9e3e-4016-9043-9095944addc1 req-b0fa3aec-4557-46b8-8e15-2dfc1235cc97 service nova] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] No waiting events found dispatching network-vif-plugged-5f97db70-9e5b-4d42-b765-b832fb522f41 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:23:31 user nova-compute[72351]: WARNING nova.compute.manager [req-b8da64f1-9e3e-4016-9043-9095944addc1 req-b0fa3aec-4557-46b8-8e15-2dfc1235cc97 service nova] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Received unexpected event network-vif-plugged-5f97db70-9e5b-4d42-b765-b832fb522f41 for instance with vm_state active and task_state rescuing.
Aug 09 02:23:32 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:23:33 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Removed pending event for eb86cb49-2018-42c2-bd97-c662b829775d due to event {{(pid=72351) _event_emit_delayed /opt/stack/nova/nova/virt/libvirt/host.py:438}}
Aug 09 02:23:33 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547813.1635592, eb86cb49-2018-42c2-bd97-c662b829775d => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:23:33 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] VM Resumed (Lifecycle Event)
Aug 09 02:23:33 user nova-compute[72351]: DEBUG nova.compute.manager [None req-51f6ac17-0aff-4407-8eda-a100f56e8ae9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:23:33 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:23:33 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: active, current task_state: rescuing, current DB power_state: 1, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:23:33 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] During sync_power_state the instance has a pending task (rescuing). Skip.
Aug 09 02:23:33 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547813.166268, eb86cb49-2018-42c2-bd97-c662b829775d => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:23:33 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] VM Started (Lifecycle Event)
Aug 09 02:23:33 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:23:33 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Synchronizing instance power state after lifecycle event "Started"; current vm_state: active, current task_state: rescuing, current DB power_state: 1, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:23:33 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] During sync_power_state the instance has a pending task (rescuing). Skip.
Aug 09 02:23:36 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:23:40 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691547805.3096633, 97b13ccd-15b4-44bf-b865-71b4da861168 => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:23:40 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] VM Stopped (Lifecycle Event)
Aug 09 02:23:40 user nova-compute[72351]: DEBUG nova.compute.manager [None req-2f0bf8f0-348b-4579-a34e-1d62a1f67de7 None None] [instance: 97b13ccd-15b4-44bf-b865-71b4da861168] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:23:41 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691547806.1855335, 74728ca8-34d4-45e6-a1f2-21c0952ff835 => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:23:41 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] VM Stopped (Lifecycle Event)
Aug 09 02:23:41 user nova-compute[72351]: DEBUG nova.compute.manager [None req-6d9f54c7-415e-49c5-8b97-5963e8386425 None None] [instance: 74728ca8-34d4-45e6-a1f2-21c0952ff835] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:23:41 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:23:42 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:23:46 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:23:47 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:23:47 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:23:48 user nova-compute[72351]: DEBUG nova.compute.manager [req-76ad5938-de9f-4a14-999f-fd8e549a4427 req-205f9f55-68fd-4226-afbf-4dee6b1e4115 service nova] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Received event network-changed-55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:23:48 user nova-compute[72351]: DEBUG nova.compute.manager [req-76ad5938-de9f-4a14-999f-fd8e549a4427 req-205f9f55-68fd-4226-afbf-4dee6b1e4115 service nova] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Refreshing instance network info cache due to event network-changed-55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:23:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-76ad5938-de9f-4a14-999f-fd8e549a4427 req-205f9f55-68fd-4226-afbf-4dee6b1e4115 service nova] Acquiring lock "refresh_cache-1d772c10-1c8a-4101-9a61-f8a60bb19c66" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:23:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-76ad5938-de9f-4a14-999f-fd8e549a4427 req-205f9f55-68fd-4226-afbf-4dee6b1e4115 service nova] Acquired lock "refresh_cache-1d772c10-1c8a-4101-9a61-f8a60bb19c66" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:23:48 user nova-compute[72351]: DEBUG nova.network.neutron [req-76ad5938-de9f-4a14-999f-fd8e549a4427 req-205f9f55-68fd-4226-afbf-4dee6b1e4115 service nova] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Refreshing network info cache for port 55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:23:49 user nova-compute[72351]: DEBUG nova.network.neutron [req-76ad5938-de9f-4a14-999f-fd8e549a4427 req-205f9f55-68fd-4226-afbf-4dee6b1e4115 service nova] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Updated VIF entry in instance network info cache for port 55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:23:49 user nova-compute[72351]: DEBUG nova.network.neutron [req-76ad5938-de9f-4a14-999f-fd8e549a4427 req-205f9f55-68fd-4226-afbf-4dee6b1e4115 service nova] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Updating instance_info_cache with network_info: [{"id": "55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7", "address": "fa:16:3e:75:70:ff", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.234", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap55e3cfd3-c6", "ovs_interfaceid": "55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:23:49 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-76ad5938-de9f-4a14-999f-fd8e549a4427 req-205f9f55-68fd-4226-afbf-4dee6b1e4115 service nova] Releasing lock "refresh_cache-1d772c10-1c8a-4101-9a61-f8a60bb19c66" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:23:51 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:23:51 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:23:52 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:23:54 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:23:54 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:23:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:23:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:23:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:23:54 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Auditing locally available compute resources for user (node: user) {{(pid=72351) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Aug 09 02:23:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:23:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json" returned: 0 in 0.177s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:23:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:23:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json" returned: 0 in 0.145s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:23:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.rescue --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:23:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.rescue --force-share --output=json" returned: 0 in 0.149s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:23:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.rescue --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:23:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.rescue --force-share --output=json" returned: 0 in 0.178s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:23:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:23:55 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:23:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json" returned: 0 in 0.174s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:23:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:23:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json" returned: 0 in 0.154s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:23:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:23:56 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:23:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json" returned: 0 in 0.161s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:23:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:23:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json" returned: 0 in 0.162s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:23:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:23:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2/disk --force-share --output=json" returned: 0 in 0.142s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:23:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:23:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2/disk --force-share --output=json" returned: 0 in 0.180s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:23:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:23:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json" returned: 0 in 0.162s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:23:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:23:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json" returned: 0 in 0.154s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:23:57 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdd (vda) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:23:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:23:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff/disk --force-share --output=json" returned: 0 in 0.155s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:23:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:23:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:23:58 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:23:58 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:23:58 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:23:58 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Hypervisor/Node resource view: name=user free_ram=7476MB free_disk=25.468547821044922GB free_vcpus=5 pci_devices=[{"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72351) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Aug 09 02:23:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:23:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:23:58 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 2d78475c-107d-4256-b12e-dbe9c8f3b9b2 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:23:58 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 75bbbead-8628-448b-9484-19bf4ab922ff actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:23:58 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:23:58 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance eb86cb49-2018-42c2-bd97-c662b829775d actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:23:58 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 1d772c10-1c8a-4101-9a61-f8a60bb19c66 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:23:58 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:23:58 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance a7f3d076-864b-4b68-9207-214ce2c21444 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:23:58 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Total usable vcpus: 12, total allocated vcpus: 7 {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Aug 09 02:23:58 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Final resource view: name=user phys_ram=16011MB used_ram=4096MB phys_disk=40GB used_disk=6GB total_vcpus=12 used_vcpus=7 pci_stats=[] {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Aug 09 02:23:58 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:23:58 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:23:58 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:23:58 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Compute_service record updated for user:user {{(pid=72351) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Aug 09 02:23:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.430s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:23:59 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:23:59 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:23:59 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Starting heal instance info cache {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9842}}
Aug 09 02:23:59 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "refresh_cache-75bbbead-8628-448b-9484-19bf4ab922ff" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:23:59 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquired lock "refresh_cache-75bbbead-8628-448b-9484-19bf4ab922ff" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:23:59 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Forcefully refreshing network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1996}}
Aug 09 02:24:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquiring lock "5c71284c-df41-4990-9a46-7986982bf7e2" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:24:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "5c71284c-df41-4990-9a46-7986982bf7e2" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:24:00 user nova-compute[72351]: DEBUG nova.compute.manager [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Starting instance... {{(pid=72351) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2406}}
Aug 09 02:24:00 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Updating instance_info_cache with network_info: [{"id": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "address": "fa:16:3e:8b:b5:2e", "network": {"id": "8231f2c2-237e-41c6-8892-720b950c1453", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherB-1917380761-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.30", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fc90e94bbcb54fb6b86a9ac45edb596a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbe4ec483-85", "ovs_interfaceid": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:24:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:24:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:24:00 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72351) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Aug 09 02:24:00 user nova-compute[72351]: INFO nova.compute.claims [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Claim successful on node user
Aug 09 02:24:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Releasing lock "refresh_cache-75bbbead-8628-448b-9484-19bf4ab922ff" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:24:00 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Updated the network info_cache for instance {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9913}}
Aug 09 02:24:00 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:24:00 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:24:00 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72351) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10461}}
Aug 09 02:24:00 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:24:00 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:24:01 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:24:01 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:24:01 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:01 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.431s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:24:01 user nova-compute[72351]: DEBUG nova.compute.manager [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Start building networks asynchronously for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2803}}
Aug 09 02:24:01 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Aug 09 02:24:01 user nova-compute[72351]: DEBUG nova.compute.manager [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Allocating IP information in the background. {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1956}}
Aug 09 02:24:01 user nova-compute[72351]: DEBUG nova.network.neutron [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] allocate_for_instance() {{(pid=72351) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Aug 09 02:24:01 user nova-compute[72351]: DEBUG nova.compute.manager [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Start building block device mappings for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2838}}
Aug 09 02:24:01 user nova-compute[72351]: INFO nova.virt.block_device [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Booting with volume snapshot bb0a6bad-4c89-42c0-8828-592b7947df10 at /dev/vda
Aug 09 02:24:01 user nova-compute[72351]: DEBUG nova.policy [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '07d8d5a634274df19c2db769d2d27b29', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4c6bbfd8d904ba1ac69ba5da4564513', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72351) authorize /opt/stack/nova/nova/policy.py:203}}
Aug 09 02:24:02 user nova-compute[72351]: DEBUG nova.network.neutron [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Successfully created port: f0429a54-9d4b-4bf9-be97-1dafb19ebc0b {{(pid=72351) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Aug 09 02:24:02 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:02 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:24:03 user nova-compute[72351]: DEBUG nova.network.neutron [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Successfully updated port: f0429a54-9d4b-4bf9-be97-1dafb19ebc0b {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:24:03 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquiring lock "refresh_cache-5c71284c-df41-4990-9a46-7986982bf7e2" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:24:03 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquired lock "refresh_cache-5c71284c-df41-4990-9a46-7986982bf7e2" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:24:03 user nova-compute[72351]: DEBUG nova.network.neutron [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:24:03 user nova-compute[72351]: DEBUG nova.compute.manager [req-2d572352-fd88-427b-8e56-ebcf8a8fa1cb req-7f67eb09-594e-4ece-8868-d9a33c73cc49 service nova] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Received event network-changed-f0429a54-9d4b-4bf9-be97-1dafb19ebc0b {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:24:03 user nova-compute[72351]: DEBUG nova.compute.manager [req-2d572352-fd88-427b-8e56-ebcf8a8fa1cb req-7f67eb09-594e-4ece-8868-d9a33c73cc49 service nova] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Refreshing instance network info cache due to event network-changed-f0429a54-9d4b-4bf9-be97-1dafb19ebc0b. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:24:03 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-2d572352-fd88-427b-8e56-ebcf8a8fa1cb req-7f67eb09-594e-4ece-8868-d9a33c73cc49 service nova] Acquiring lock "refresh_cache-5c71284c-df41-4990-9a46-7986982bf7e2" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:24:03 user nova-compute[72351]: DEBUG nova.network.neutron [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:24:03 user nova-compute[72351]: DEBUG nova.network.neutron [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Updating instance_info_cache with network_info: [{"id": "f0429a54-9d4b-4bf9-be97-1dafb19ebc0b", "address": "fa:16:3e:78:c7:55", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf0429a54-9d", "ovs_interfaceid": "f0429a54-9d4b-4bf9-be97-1dafb19ebc0b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:24:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Releasing lock "refresh_cache-5c71284c-df41-4990-9a46-7986982bf7e2" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:24:04 user nova-compute[72351]: DEBUG nova.compute.manager [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Instance network_info: |[{"id": "f0429a54-9d4b-4bf9-be97-1dafb19ebc0b", "address": "fa:16:3e:78:c7:55", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf0429a54-9d", "ovs_interfaceid": "f0429a54-9d4b-4bf9-be97-1dafb19ebc0b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1971}}
Aug 09 02:24:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-2d572352-fd88-427b-8e56-ebcf8a8fa1cb req-7f67eb09-594e-4ece-8868-d9a33c73cc49 service nova] Acquired lock "refresh_cache-5c71284c-df41-4990-9a46-7986982bf7e2" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:24:04 user nova-compute[72351]: DEBUG nova.network.neutron [req-2d572352-fd88-427b-8e56-ebcf8a8fa1cb req-7f67eb09-594e-4ece-8868-d9a33c73cc49 service nova] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Refreshing network info cache for port f0429a54-9d4b-4bf9-be97-1dafb19ebc0b {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:24:04 user nova-compute[72351]: DEBUG nova.network.neutron [req-2d572352-fd88-427b-8e56-ebcf8a8fa1cb req-7f67eb09-594e-4ece-8868-d9a33c73cc49 service nova] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Updated VIF entry in instance network info cache for port f0429a54-9d4b-4bf9-be97-1dafb19ebc0b. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:24:04 user nova-compute[72351]: DEBUG nova.network.neutron [req-2d572352-fd88-427b-8e56-ebcf8a8fa1cb req-7f67eb09-594e-4ece-8868-d9a33c73cc49 service nova] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Updating instance_info_cache with network_info: [{"id": "f0429a54-9d4b-4bf9-be97-1dafb19ebc0b", "address": "fa:16:3e:78:c7:55", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf0429a54-9d", "ovs_interfaceid": "f0429a54-9d4b-4bf9-be97-1dafb19ebc0b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:24:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-2d572352-fd88-427b-8e56-ebcf8a8fa1cb req-7f67eb09-594e-4ece-8868-d9a33c73cc49 service nova] Releasing lock "refresh_cache-5c71284c-df41-4990-9a46-7986982bf7e2" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:24:06 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Acquiring lock "433e6729-fdd4-4467-971c-08ca962d887d" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:24:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lock "433e6729-fdd4-4467-971c-08ca962d887d" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:24:06 user nova-compute[72351]: DEBUG nova.compute.manager [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Starting instance... {{(pid=72351) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2406}}
Aug 09 02:24:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:24:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:24:06 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72351) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Aug 09 02:24:06 user nova-compute[72351]: INFO nova.compute.claims [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Claim successful on node user
Aug 09 02:24:07 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:24:07 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:24:07 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.408s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:24:07 user nova-compute[72351]: DEBUG nova.compute.manager [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Start building networks asynchronously for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2803}}
Aug 09 02:24:07 user nova-compute[72351]: DEBUG nova.compute.manager [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Allocating IP information in the background. {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1956}}
Aug 09 02:24:07 user nova-compute[72351]: DEBUG nova.network.neutron [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] allocate_for_instance() {{(pid=72351) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Aug 09 02:24:07 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Aug 09 02:24:07 user nova-compute[72351]: DEBUG nova.compute.manager [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Start building block device mappings for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2838}}
Aug 09 02:24:07 user nova-compute[72351]: DEBUG nova.policy [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '97267b9f897e4034b71627a698210071', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c8ff52319444079b754f79843c67b2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72351) authorize /opt/stack/nova/nova/policy.py:203}}
Aug 09 02:24:07 user nova-compute[72351]: DEBUG nova.compute.manager [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Start spawning the instance on the hypervisor. {{(pid=72351) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2612}}
Aug 09 02:24:07 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Creating instance directory {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4717}}
Aug 09 02:24:07 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Creating image(s)
Aug 09 02:24:07 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Acquiring lock "/opt/stack/data/nova/instances/433e6729-fdd4-4467-971c-08ca962d887d/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:24:07 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lock "/opt/stack/data/nova/instances/433e6729-fdd4-4467-971c-08ca962d887d/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:24:07 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lock "/opt/stack/data/nova/instances/433e6729-fdd4-4467-971c-08ca962d887d/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.042s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:24:07 user nova-compute[72351]: DEBUG os_brick.utils [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:24:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:24:07 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[dc548da8-d632-4de5-8f12-2b9faa94d238]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:a52269602ff4\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:24:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:24:07 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[0024e46b-fe7a-40de-8033-2fa6fa715746]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:24:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:24:07 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[2362b6ba-6c15-4f91-b360-c64850998616]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:24:07 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d379998b-07b5-43a8-b811-3a0c58c06734]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:24:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Running cmd (subprocess): nvme version {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] 'nvme version' failed. Not Retrying. {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:24:07 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.nvmeof [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] nvme not present on system {{(pid=72351) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Aug 09 02:24:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:24:07 user nova-compute[72351]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Aug 09 02:24:07 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[841f7632-4236-4688-a677-f7bd8f30eead]: (4, '') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:24:07 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72351) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Aug 09 02:24:07 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Aug 09 02:24:07 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] LIGHTOS: no hostnqn found. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Aug 09 02:24:07 user nova-compute[72351]: DEBUG os_brick.utils [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] <== get_connector_properties: return (160ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:a52269602ff4', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:24:07 user nova-compute[72351]: DEBUG nova.virt.block_device [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Updating existing volume attachment record: ee71e726-6950-4639-b7ac-1e5acb27949e {{(pid=72351) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Aug 09 02:24:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.171s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:24:07 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Acquiring lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:24:07 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:24:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.128s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:24:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/433e6729-fdd4-4467-971c-08ca962d887d/disk 1073741824 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/433e6729-fdd4-4467-971c-08ca962d887d/disk 1073741824" returned: 0 in 0.051s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:24:07 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.186s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:24:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:07 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.126s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:24:07 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Checking if we can resize image /opt/stack/data/nova/instances/433e6729-fdd4-4467-971c-08ca962d887d/disk. size=1073741824 {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Aug 09 02:24:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/433e6729-fdd4-4467-971c-08ca962d887d/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/433e6729-fdd4-4467-971c-08ca962d887d/disk --force-share --output=json" returned: 0 in 0.130s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:24:08 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Cannot resize image /opt/stack/data/nova/instances/433e6729-fdd4-4467-971c-08ca962d887d/disk to a smaller size. {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Aug 09 02:24:08 user nova-compute[72351]: DEBUG nova.objects.instance [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lazy-loading 'migration_context' on Instance uuid 433e6729-fdd4-4467-971c-08ca962d887d {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:24:08 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Created local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4851}}
Aug 09 02:24:08 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Ensure instance console log exists: /opt/stack/data/nova/instances/433e6729-fdd4-4467-971c-08ca962d887d/console.log {{(pid=72351) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4603}}
Aug 09 02:24:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:24:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:24:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:24:08 user nova-compute[72351]: DEBUG nova.network.neutron [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Successfully created port: 7f4f4044-2549-41b0-b1b8-9154419a8b54 {{(pid=72351) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Aug 09 02:24:08 user nova-compute[72351]: DEBUG nova.network.neutron [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Successfully updated port: 7f4f4044-2549-41b0-b1b8-9154419a8b54 {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:24:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Acquiring lock "refresh_cache-433e6729-fdd4-4467-971c-08ca962d887d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:24:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Acquired lock "refresh_cache-433e6729-fdd4-4467-971c-08ca962d887d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:24:08 user nova-compute[72351]: DEBUG nova.network.neutron [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:24:08 user nova-compute[72351]: DEBUG nova.compute.manager [req-e8d89261-8fa6-4ced-917f-5779b096eb1d req-00ba46de-42f6-4a43-8229-8dc9a434cff3 service nova] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Received event network-changed-7f4f4044-2549-41b0-b1b8-9154419a8b54 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:24:08 user nova-compute[72351]: DEBUG nova.compute.manager [req-e8d89261-8fa6-4ced-917f-5779b096eb1d req-00ba46de-42f6-4a43-8229-8dc9a434cff3 service nova] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Refreshing instance network info cache due to event network-changed-7f4f4044-2549-41b0-b1b8-9154419a8b54. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:24:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-e8d89261-8fa6-4ced-917f-5779b096eb1d req-00ba46de-42f6-4a43-8229-8dc9a434cff3 service nova] Acquiring lock "refresh_cache-433e6729-fdd4-4467-971c-08ca962d887d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:24:08 user nova-compute[72351]: DEBUG nova.network.neutron [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:24:09 user nova-compute[72351]: DEBUG nova.network.neutron [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Updating instance_info_cache with network_info: [{"id": "7f4f4044-2549-41b0-b1b8-9154419a8b54", "address": "fa:16:3e:04:53:28", "network": {"id": "96018a29-3b2e-44de-b2c4-67abc92e3a30", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-924549970-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "1c8ff52319444079b754f79843c67b2b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7f4f4044-25", "ovs_interfaceid": "7f4f4044-2549-41b0-b1b8-9154419a8b54", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:24:09 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Releasing lock "refresh_cache-433e6729-fdd4-4467-971c-08ca962d887d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:24:09 user nova-compute[72351]: DEBUG nova.compute.manager [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Instance network_info: |[{"id": "7f4f4044-2549-41b0-b1b8-9154419a8b54", "address": "fa:16:3e:04:53:28", "network": {"id": "96018a29-3b2e-44de-b2c4-67abc92e3a30", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-924549970-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "1c8ff52319444079b754f79843c67b2b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7f4f4044-25", "ovs_interfaceid": "7f4f4044-2549-41b0-b1b8-9154419a8b54", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1971}}
Aug 09 02:24:09 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-e8d89261-8fa6-4ced-917f-5779b096eb1d req-00ba46de-42f6-4a43-8229-8dc9a434cff3 service nova] Acquired lock "refresh_cache-433e6729-fdd4-4467-971c-08ca962d887d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:24:09 user nova-compute[72351]: DEBUG nova.network.neutron [req-e8d89261-8fa6-4ced-917f-5779b096eb1d req-00ba46de-42f6-4a43-8229-8dc9a434cff3 service nova] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Refreshing network info cache for port 7f4f4044-2549-41b0-b1b8-9154419a8b54 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:24:09 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Start _get_guest_xml network_info=[{"id": "7f4f4044-2549-41b0-b1b8-9154419a8b54", "address": "fa:16:3e:04:53:28", "network": {"id": "96018a29-3b2e-44de-b2c4-67abc92e3a30", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-924549970-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "1c8ff52319444079b754f79843c67b2b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7f4f4044-25", "ovs_interfaceid": "7f4f4044-2549-41b0-b1b8-9154419a8b54", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'size': 0, 'device_type': 'disk', 'encryption_secret_uuid': None, 'guest_format': None, 'encryption_options': None, 'disk_bus': 'virtio', 'encrypted': False, 'encryption_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'image_id': '18ccda7f-31d3-4978-ab9d-4ed2f74eff94'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:24:09 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:24:09 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:24:09 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:24:09 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:24:09 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:24:09 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:24:09 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:24:09 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:24:09 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:24:09 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:24:09 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:24:09 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:24:09 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:24:09 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:24:09 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:24:09 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:24:09 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:24:09 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:24:09 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:24:06Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesSnapshotTestJSON-instance-1098155920',display_name='tempest-VolumesSnapshotTestJSON-instance-1098155920',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-volumessnapshottestjson-instance-1098155920',id=22,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBCAKJgeFJGn197soLO4TKV3jbPbEAz0hnZRWzRGb1+UBTzihAr6/WvY3HiLk3OFxEU+jj/jJC5OiHDoEcaue5hFL+F0PeY1vCaX7fDZVIgbnkzaBADdLVYiBoPx//f/J0g==',key_name='tempest-keypair-1954238676',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='1c8ff52319444079b754f79843c67b2b',ramdisk_id='',reservation_id='r-a3jy1i1h',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesSnapshotTestJSON-343261326',owner_user_name='tempest-VolumesSnapshotTestJSON-343261326-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:24:07Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='97267b9f897e4034b71627a698210071',uuid=433e6729-fdd4-4467-971c-08ca962d887d,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "7f4f4044-2549-41b0-b1b8-9154419a8b54", "address": "fa:16:3e:04:53:28", "network": {"id": "96018a29-3b2e-44de-b2c4-67abc92e3a30", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-924549970-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "1c8ff52319444079b754f79843c67b2b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7f4f4044-25", "ovs_interfaceid": "7f4f4044-2549-41b0-b1b8-9154419a8b54", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:24:09 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Converting VIF {"id": "7f4f4044-2549-41b0-b1b8-9154419a8b54", "address": "fa:16:3e:04:53:28", "network": {"id": "96018a29-3b2e-44de-b2c4-67abc92e3a30", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-924549970-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "1c8ff52319444079b754f79843c67b2b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7f4f4044-25", "ovs_interfaceid": "7f4f4044-2549-41b0-b1b8-9154419a8b54", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:24:09 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:04:53:28,bridge_name='br-int',has_traffic_filtering=True,id=7f4f4044-2549-41b0-b1b8-9154419a8b54,network=Network(96018a29-3b2e-44de-b2c4-67abc92e3a30),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7f4f4044-25') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:24:09 user nova-compute[72351]: DEBUG nova.objects.instance [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lazy-loading 'pci_devices' on Instance uuid 433e6729-fdd4-4467-971c-08ca962d887d {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:24:09 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:24:09 user nova-compute[72351]:   <uuid>433e6729-fdd4-4467-971c-08ca962d887d</uuid>
Aug 09 02:24:09 user nova-compute[72351]:   <name>instance-00000016</name>
Aug 09 02:24:09 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:24:09 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:24:09 user nova-compute[72351]:   <metadata>
Aug 09 02:24:09 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:24:09 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:24:09 user nova-compute[72351]:       <nova:name>tempest-VolumesSnapshotTestJSON-instance-1098155920</nova:name>
Aug 09 02:24:09 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:24:09</nova:creationTime>
Aug 09 02:24:09 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:24:09 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:24:09 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:24:09 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:24:09 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:24:09 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:24:09 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:24:09 user nova-compute[72351]:       <nova:owner>
Aug 09 02:24:09 user nova-compute[72351]:         <nova:user uuid="97267b9f897e4034b71627a698210071">tempest-VolumesSnapshotTestJSON-343261326-project-member</nova:user>
Aug 09 02:24:09 user nova-compute[72351]:         <nova:project uuid="1c8ff52319444079b754f79843c67b2b">tempest-VolumesSnapshotTestJSON-343261326</nova:project>
Aug 09 02:24:09 user nova-compute[72351]:       </nova:owner>
Aug 09 02:24:09 user nova-compute[72351]:       <nova:root type="image" uuid="18ccda7f-31d3-4978-ab9d-4ed2f74eff94"/>
Aug 09 02:24:09 user nova-compute[72351]:       <nova:ports>
Aug 09 02:24:09 user nova-compute[72351]:         <nova:port uuid="7f4f4044-2549-41b0-b1b8-9154419a8b54">
Aug 09 02:24:09 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.13" ipVersion="4"/>
Aug 09 02:24:09 user nova-compute[72351]:         </nova:port>
Aug 09 02:24:09 user nova-compute[72351]:       </nova:ports>
Aug 09 02:24:09 user nova-compute[72351]:     </nova:instance>
Aug 09 02:24:09 user nova-compute[72351]:   </metadata>
Aug 09 02:24:09 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:24:09 user nova-compute[72351]:     <system>
Aug 09 02:24:09 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:24:09 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:24:09 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:24:09 user nova-compute[72351]:       <entry name="serial">433e6729-fdd4-4467-971c-08ca962d887d</entry>
Aug 09 02:24:09 user nova-compute[72351]:       <entry name="uuid">433e6729-fdd4-4467-971c-08ca962d887d</entry>
Aug 09 02:24:09 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:24:09 user nova-compute[72351]:     </system>
Aug 09 02:24:09 user nova-compute[72351]:   </sysinfo>
Aug 09 02:24:09 user nova-compute[72351]:   <os>
Aug 09 02:24:09 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:24:09 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:24:09 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:24:09 user nova-compute[72351]:   </os>
Aug 09 02:24:09 user nova-compute[72351]:   <features>
Aug 09 02:24:09 user nova-compute[72351]:     <acpi/>
Aug 09 02:24:09 user nova-compute[72351]:     <apic/>
Aug 09 02:24:09 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:24:09 user nova-compute[72351]:   </features>
Aug 09 02:24:09 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:24:09 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:24:09 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:24:09 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:24:09 user nova-compute[72351]:   </clock>
Aug 09 02:24:09 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:24:09 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:24:09 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:24:09 user nova-compute[72351]:   </cpu>
Aug 09 02:24:09 user nova-compute[72351]:   <devices>
Aug 09 02:24:09 user nova-compute[72351]:     <disk type="file" device="disk">
Aug 09 02:24:09 user nova-compute[72351]:       <driver name="qemu" type="qcow2" cache="none"/>
Aug 09 02:24:09 user nova-compute[72351]:       <source file="/opt/stack/data/nova/instances/433e6729-fdd4-4467-971c-08ca962d887d/disk"/>
Aug 09 02:24:09 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:24:09 user nova-compute[72351]:     </disk>
Aug 09 02:24:09 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:24:09 user nova-compute[72351]:       <mac address="fa:16:3e:04:53:28"/>
Aug 09 02:24:09 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:24:09 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:24:09 user nova-compute[72351]:       <target dev="tap7f4f4044-25"/>
Aug 09 02:24:09 user nova-compute[72351]:     </interface>
Aug 09 02:24:09 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:24:09 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/433e6729-fdd4-4467-971c-08ca962d887d/console.log" append="off"/>
Aug 09 02:24:09 user nova-compute[72351]:     </serial>
Aug 09 02:24:09 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:24:09 user nova-compute[72351]:     <video>
Aug 09 02:24:09 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:24:09 user nova-compute[72351]:     </video>
Aug 09 02:24:09 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:24:09 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:24:09 user nova-compute[72351]:     </rng>
Aug 09 02:24:09 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:24:09 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:24:09 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:24:09 user nova-compute[72351]:     </memballoon>
Aug 09 02:24:09 user nova-compute[72351]:   </devices>
Aug 09 02:24:09 user nova-compute[72351]: </domain>
Aug 09 02:24:09 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:24:09 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:24:06Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesSnapshotTestJSON-instance-1098155920',display_name='tempest-VolumesSnapshotTestJSON-instance-1098155920',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-volumessnapshottestjson-instance-1098155920',id=22,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBCAKJgeFJGn197soLO4TKV3jbPbEAz0hnZRWzRGb1+UBTzihAr6/WvY3HiLk3OFxEU+jj/jJC5OiHDoEcaue5hFL+F0PeY1vCaX7fDZVIgbnkzaBADdLVYiBoPx//f/J0g==',key_name='tempest-keypair-1954238676',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='1c8ff52319444079b754f79843c67b2b',ramdisk_id='',reservation_id='r-a3jy1i1h',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesSnapshotTestJSON-343261326',owner_user_name='tempest-VolumesSnapshotTestJSON-343261326-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:24:07Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='97267b9f897e4034b71627a698210071',uuid=433e6729-fdd4-4467-971c-08ca962d887d,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "7f4f4044-2549-41b0-b1b8-9154419a8b54", "address": "fa:16:3e:04:53:28", "network": {"id": "96018a29-3b2e-44de-b2c4-67abc92e3a30", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-924549970-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "1c8ff52319444079b754f79843c67b2b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7f4f4044-25", "ovs_interfaceid": "7f4f4044-2549-41b0-b1b8-9154419a8b54", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:24:09 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Converting VIF {"id": "7f4f4044-2549-41b0-b1b8-9154419a8b54", "address": "fa:16:3e:04:53:28", "network": {"id": "96018a29-3b2e-44de-b2c4-67abc92e3a30", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-924549970-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "1c8ff52319444079b754f79843c67b2b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7f4f4044-25", "ovs_interfaceid": "7f4f4044-2549-41b0-b1b8-9154419a8b54", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:24:09 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:04:53:28,bridge_name='br-int',has_traffic_filtering=True,id=7f4f4044-2549-41b0-b1b8-9154419a8b54,network=Network(96018a29-3b2e-44de-b2c4-67abc92e3a30),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7f4f4044-25') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:24:09 user nova-compute[72351]: DEBUG os_vif [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:04:53:28,bridge_name='br-int',has_traffic_filtering=True,id=7f4f4044-2549-41b0-b1b8-9154419a8b54,network=Network(96018a29-3b2e-44de-b2c4-67abc92e3a30),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7f4f4044-25') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:24:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:24:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:24:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap7f4f4044-25, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:24:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap7f4f4044-25, col_values=(('external_ids', {'iface-id': '7f4f4044-2549-41b0-b1b8-9154419a8b54', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:04:53:28', 'vm-uuid': '433e6729-fdd4-4467-971c-08ca962d887d'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:24:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:24:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:09 user nova-compute[72351]: INFO os_vif [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:04:53:28,bridge_name='br-int',has_traffic_filtering=True,id=7f4f4044-2549-41b0-b1b8-9154419a8b54,network=Network(96018a29-3b2e-44de-b2c4-67abc92e3a30),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7f4f4044-25')
Aug 09 02:24:09 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:24:09 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] No VIF found with MAC fa:16:3e:04:53:28, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:24:09 user nova-compute[72351]: DEBUG nova.network.neutron [req-e8d89261-8fa6-4ced-917f-5779b096eb1d req-00ba46de-42f6-4a43-8229-8dc9a434cff3 service nova] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Updated VIF entry in instance network info cache for port 7f4f4044-2549-41b0-b1b8-9154419a8b54. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:24:09 user nova-compute[72351]: DEBUG nova.network.neutron [req-e8d89261-8fa6-4ced-917f-5779b096eb1d req-00ba46de-42f6-4a43-8229-8dc9a434cff3 service nova] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Updating instance_info_cache with network_info: [{"id": "7f4f4044-2549-41b0-b1b8-9154419a8b54", "address": "fa:16:3e:04:53:28", "network": {"id": "96018a29-3b2e-44de-b2c4-67abc92e3a30", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-924549970-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "1c8ff52319444079b754f79843c67b2b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7f4f4044-25", "ovs_interfaceid": "7f4f4044-2549-41b0-b1b8-9154419a8b54", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:24:09 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-e8d89261-8fa6-4ced-917f-5779b096eb1d req-00ba46de-42f6-4a43-8229-8dc9a434cff3 service nova] Releasing lock "refresh_cache-433e6729-fdd4-4467-971c-08ca962d887d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:24:10 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:11 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:11 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:11 user nova-compute[72351]: DEBUG nova.compute.manager [req-d41f7e08-7c5d-49b8-8135-d7d30eac29fe req-39028045-a472-4c13-a177-d3462d3e2d42 service nova] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Received event network-vif-plugged-7f4f4044-2549-41b0-b1b8-9154419a8b54 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:24:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d41f7e08-7c5d-49b8-8135-d7d30eac29fe req-39028045-a472-4c13-a177-d3462d3e2d42 service nova] Acquiring lock "433e6729-fdd4-4467-971c-08ca962d887d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:24:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d41f7e08-7c5d-49b8-8135-d7d30eac29fe req-39028045-a472-4c13-a177-d3462d3e2d42 service nova] Lock "433e6729-fdd4-4467-971c-08ca962d887d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:24:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d41f7e08-7c5d-49b8-8135-d7d30eac29fe req-39028045-a472-4c13-a177-d3462d3e2d42 service nova] Lock "433e6729-fdd4-4467-971c-08ca962d887d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:24:11 user nova-compute[72351]: DEBUG nova.compute.manager [req-d41f7e08-7c5d-49b8-8135-d7d30eac29fe req-39028045-a472-4c13-a177-d3462d3e2d42 service nova] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] No waiting events found dispatching network-vif-plugged-7f4f4044-2549-41b0-b1b8-9154419a8b54 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:24:11 user nova-compute[72351]: WARNING nova.compute.manager [req-d41f7e08-7c5d-49b8-8135-d7d30eac29fe req-39028045-a472-4c13-a177-d3462d3e2d42 service nova] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Received unexpected event network-vif-plugged-7f4f4044-2549-41b0-b1b8-9154419a8b54 for instance with vm_state building and task_state spawning.
Aug 09 02:24:11 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:11 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:12 user nova-compute[72351]: DEBUG nova.compute.manager [req-0dadf8ed-6c4f-4f51-b90f-2f022ba69229 req-b92a479a-04f1-4250-bbbe-3977b360f233 service nova] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Received event network-changed-e521fae4-093d-4c92-a32d-684c1a5137f5 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:24:12 user nova-compute[72351]: DEBUG nova.compute.manager [req-0dadf8ed-6c4f-4f51-b90f-2f022ba69229 req-b92a479a-04f1-4250-bbbe-3977b360f233 service nova] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Refreshing instance network info cache due to event network-changed-e521fae4-093d-4c92-a32d-684c1a5137f5. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:24:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-0dadf8ed-6c4f-4f51-b90f-2f022ba69229 req-b92a479a-04f1-4250-bbbe-3977b360f233 service nova] Acquiring lock "refresh_cache-585c81a5-ac24-4d12-a8a6-1e7b12a2d53a" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:24:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-0dadf8ed-6c4f-4f51-b90f-2f022ba69229 req-b92a479a-04f1-4250-bbbe-3977b360f233 service nova] Acquired lock "refresh_cache-585c81a5-ac24-4d12-a8a6-1e7b12a2d53a" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:24:12 user nova-compute[72351]: DEBUG nova.network.neutron [req-0dadf8ed-6c4f-4f51-b90f-2f022ba69229 req-b92a479a-04f1-4250-bbbe-3977b360f233 service nova] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Refreshing network info cache for port e521fae4-093d-4c92-a32d-684c1a5137f5 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:24:12 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:13 user nova-compute[72351]: DEBUG nova.compute.manager [req-4327c324-9ce8-431a-b8a0-946ccc36011d req-ce93eda8-1697-49ea-9915-573e18378d8a service nova] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Received event network-vif-plugged-7f4f4044-2549-41b0-b1b8-9154419a8b54 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:24:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-4327c324-9ce8-431a-b8a0-946ccc36011d req-ce93eda8-1697-49ea-9915-573e18378d8a service nova] Acquiring lock "433e6729-fdd4-4467-971c-08ca962d887d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:24:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-4327c324-9ce8-431a-b8a0-946ccc36011d req-ce93eda8-1697-49ea-9915-573e18378d8a service nova] Lock "433e6729-fdd4-4467-971c-08ca962d887d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:24:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-4327c324-9ce8-431a-b8a0-946ccc36011d req-ce93eda8-1697-49ea-9915-573e18378d8a service nova] Lock "433e6729-fdd4-4467-971c-08ca962d887d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:24:13 user nova-compute[72351]: DEBUG nova.compute.manager [req-4327c324-9ce8-431a-b8a0-946ccc36011d req-ce93eda8-1697-49ea-9915-573e18378d8a service nova] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] No waiting events found dispatching network-vif-plugged-7f4f4044-2549-41b0-b1b8-9154419a8b54 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:24:13 user nova-compute[72351]: WARNING nova.compute.manager [req-4327c324-9ce8-431a-b8a0-946ccc36011d req-ce93eda8-1697-49ea-9915-573e18378d8a service nova] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Received unexpected event network-vif-plugged-7f4f4044-2549-41b0-b1b8-9154419a8b54 for instance with vm_state building and task_state spawning.
Aug 09 02:24:13 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547853.3676848, 433e6729-fdd4-4467-971c-08ca962d887d => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:24:13 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] VM Resumed (Lifecycle Event)
Aug 09 02:24:13 user nova-compute[72351]: DEBUG nova.compute.manager [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:24:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Guest created on hypervisor {{(pid=72351) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4411}}
Aug 09 02:24:13 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Instance spawned successfully.
Aug 09 02:24:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:894}}
Aug 09 02:24:13 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:24:13 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:24:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Found default for hw_cdrom_bus of ide {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:24:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Found default for hw_disk_bus of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:24:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Found default for hw_input_bus of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:24:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Found default for hw_pointer_model of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:24:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Found default for hw_video_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:24:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Found default for hw_vif_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:24:13 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:24:13 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547853.367872, 433e6729-fdd4-4467-971c-08ca962d887d => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:24:13 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] VM Started (Lifecycle Event)
Aug 09 02:24:13 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:24:13 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:24:13 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:24:13 user nova-compute[72351]: INFO nova.compute.manager [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Took 6.17 seconds to spawn the instance on the hypervisor.
Aug 09 02:24:13 user nova-compute[72351]: DEBUG nova.compute.manager [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:24:13 user nova-compute[72351]: DEBUG nova.network.neutron [req-0dadf8ed-6c4f-4f51-b90f-2f022ba69229 req-b92a479a-04f1-4250-bbbe-3977b360f233 service nova] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Updated VIF entry in instance network info cache for port e521fae4-093d-4c92-a32d-684c1a5137f5. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:24:13 user nova-compute[72351]: DEBUG nova.network.neutron [req-0dadf8ed-6c4f-4f51-b90f-2f022ba69229 req-b92a479a-04f1-4250-bbbe-3977b360f233 service nova] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Updating instance_info_cache with network_info: [{"id": "e521fae4-093d-4c92-a32d-684c1a5137f5", "address": "fa:16:3e:e5:b0:10", "network": {"id": "b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-210689817-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.41", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "69d584c230b94ff8aae722eb2f364c5c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape521fae4-09", "ovs_interfaceid": "e521fae4-093d-4c92-a32d-684c1a5137f5", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:24:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-0dadf8ed-6c4f-4f51-b90f-2f022ba69229 req-b92a479a-04f1-4250-bbbe-3977b360f233 service nova] Releasing lock "refresh_cache-585c81a5-ac24-4d12-a8a6-1e7b12a2d53a" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:24:13 user nova-compute[72351]: INFO nova.compute.manager [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Took 6.92 seconds to build instance.
Aug 09 02:24:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a0d1cfa6-94ba-4615-b389-9b060ba179ea tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lock "433e6729-fdd4-4467-971c-08ca962d887d" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 7.020s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:24:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:16 user nova-compute[72351]: DEBUG nova.compute.manager [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Start spawning the instance on the hypervisor. {{(pid=72351) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2612}}
Aug 09 02:24:16 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Creating instance directory {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4717}}
Aug 09 02:24:16 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Creating image(s)
Aug 09 02:24:16 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Did not create local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4853}}
Aug 09 02:24:16 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Ensure instance console log exists: /opt/stack/data/nova/instances/5c71284c-df41-4990-9a46-7986982bf7e2/console.log {{(pid=72351) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4603}}
Aug 09 02:24:16 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:24:16 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:24:16 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:24:16 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Start _get_guest_xml network_info=[{"id": "f0429a54-9d4b-4bf9-be97-1dafb19ebc0b", "address": "fa:16:3e:78:c7:55", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf0429a54-9d", "ovs_interfaceid": "f0429a54-9d4b-4bf9-be97-1dafb19ebc0b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='d41d8cd98f00b204e9800998ecf8427e',container_format='bare',created_at=2023-08-09T02:23:51Z,direct_url=<?>,disk_format='qcow2',id=7869c8d4-caa3-435c-a4e4-e01eb5198e6f,min_disk=1,min_ram=0,name='tempest-TestVolumeBootPatternsnapshot-1360487297',owner='a4c6bbfd8d904ba1ac69ba5da4564513',properties=ImageMetaProps,protected=<?>,size=0,status='active',tags=<?>,updated_at=2023-08-09T02:23:52Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'device_type': 'disk', 'guest_format': None, 'disk_bus': 'virtio', 'mount_device': '/dev/vda', 'boot_index': 0, 'attachment_id': 'ee71e726-6950-4639-b7ac-1e5acb27949e', 'delete_on_termination': True, 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqns': ['iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'lfCTruNe', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '5c71284c-df41-4990-9a46-7986982bf7e2', 'attached_at': '', 'detached_at': '', 'volume_id': 'c67f9b74-7547-4996-abf6-678dac1ff065', 'serial': 'c67f9b74-7547-4996-abf6-678dac1ff065'}, 'volume_type': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:24:16 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:24:16 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:24:16 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:24:16 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:24:16 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:24:16 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:24:16 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:24:16 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='d41d8cd98f00b204e9800998ecf8427e',container_format='bare',created_at=2023-08-09T02:23:51Z,direct_url=<?>,disk_format='qcow2',id=7869c8d4-caa3-435c-a4e4-e01eb5198e6f,min_disk=1,min_ram=0,name='tempest-TestVolumeBootPatternsnapshot-1360487297',owner='a4c6bbfd8d904ba1ac69ba5da4564513',properties=ImageMetaProps,protected=<?>,size=0,status='active',tags=<?>,updated_at=2023-08-09T02:23:52Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:24:16 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:24:16 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:24:16 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:24:16 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:24:16 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:24:16 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:24:16 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:24:16 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:24:16 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:24:16 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:24:16 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 02:24:16 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'lfCTruNe', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:24:16 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:24:16 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:24:16 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:24:16 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:24:16 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:24:16 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:24:16 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 02:24:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065 -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065 -p 172.16.0.220:3260" returned: 21 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:24:16 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[7a17f7c0-ff1d-435a-8ff1-4f44b39d2505]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:24:16 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Aug 09 02:24:16 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:24:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065 -p 172.16.0.220:3260 --interface default --op new {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:24:16 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[77c9b9be-9414-496d-827e-cd35164f7a06]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065] added\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:24:16 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065] added
Aug 09 02:24:16 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:24:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:24:16 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[af822713-6856-4fa2-a423-12ddbce26465]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:24:16 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:24:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:24:16 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[8e5c0e0b-9133-474a-aee7-8c7740a6268e]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:24:16 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:24:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v lfCTruNe {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v lfCTruNe" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:24:16 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[eba30f9b-6139-4d9b-b40e-539e25201cf8]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:24:16 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'lfCTruNe'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:24:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:24:16 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[c7532a8f-e6ed-46b1-89fe-e799bdc55e42]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:24:16 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:24:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:24:16 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e0674fd8-db14-4224-a473-74c707c82cc3]: (4, ('tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:24:16 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)
Aug 09 02:24:16 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:24:16 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsi session list stdout=tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)
Aug 09 02:24:16 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:24:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065 -p 172.16.0.220:3260 --login {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065 -p 172.16.0.220:3260 --login" returned: 0 in 0.062s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:24:16 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a8a1544b-d3dc-4086-9fe0-81e59b771632]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:24:16 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065, portal: 172.16.0.220,3260]
Aug 09 02:24:16 user nova-compute[72351]: Login to [iface: default, target: iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065, portal: 172.16.0.220,3260] successful.
Aug 09 02:24:16 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:24:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:24:17 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[6a908bd6-e760-4f44-a98b-4e5226362919]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:24:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:24:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:24:17 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[4393ad52-7be6-40db-a9b6-1eb826be9590]: (4, ('tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)\ntcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:24:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)
Aug 09 02:24:17 user nova-compute[72351]: tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065 (non-flash)
Aug 09 02:24:17 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:24:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsi session list stdout=tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)
Aug 09 02:24:17 user nova-compute[72351]: tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065 (non-flash)
Aug 09 02:24:17 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:24:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.292s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:24:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 02:24:17 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] HCTL ('33', '-', '-', 0) found on session 19 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 02:24:17 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 02:24:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:24:17 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[494a30c0-d30f-4911-be79-065a6d62d9bb]: (4, ('- - 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:24:17 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Searching for a device in session 19 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:24:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Connected to sdb using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'lfCTruNe', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 02:24:17 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.330s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:24:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] <== connect_volume: return (1332ms) {'type': 'block', 'scsi_wwn': '23463643234643035', 'path': '/dev/sdb'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:24:18 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23463643234643035', 'path': '/dev/sdb'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 02:24:18 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:23:59Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-image-snapshot-server-291879344',display_name='tempest-TestVolumeBootPattern-image-snapshot-server-291879344',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testvolumebootpattern-image-snapshot-server-291879344',id=21,image_ref='7869c8d4-caa3-435c-a4e4-e01eb5198e6f',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMigRZU/OzbuGai7a1EcqwThXhvudiR9/Q8Da5xVMuKq+4DVrTM+XekLmyXUIzTgyNFsg9BxRHjyOIIoT5ZbBndJV38K0BzyxRl30KiADrnKDbDnCylyfZZ9qqf+nevzqg==',key_name='tempest-keypair-2017099597',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='a4c6bbfd8d904ba1ac69ba5da4564513',ramdisk_id='',reservation_id='r-caoazglt',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_bdm_v2='True',image_boot_roles='reader,member',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_project_name='tempest-TestVolumeBootPattern-350743460',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',image_owner_user_name='tempest-TestVolumeBootPattern-350743460-project-member',image_root_device_name='/dev/vda',network_allocated='True',owner_project_name='tempest-TestVolumeBootPattern-350743460',owner_user_name='tempest-TestVolumeBootPattern-350743460-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:24:01Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='07d8d5a634274df19c2db769d2d27b29',uuid=5c71284c-df41-4990-9a46-7986982bf7e2,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "f0429a54-9d4b-4bf9-be97-1dafb19ebc0b", "address": "fa:16:3e:78:c7:55", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf0429a54-9d", "ovs_interfaceid": "f0429a54-9d4b-4bf9-be97-1dafb19ebc0b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:24:18 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Converting VIF {"id": "f0429a54-9d4b-4bf9-be97-1dafb19ebc0b", "address": "fa:16:3e:78:c7:55", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf0429a54-9d", "ovs_interfaceid": "f0429a54-9d4b-4bf9-be97-1dafb19ebc0b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:24:18 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:78:c7:55,bridge_name='br-int',has_traffic_filtering=True,id=f0429a54-9d4b-4bf9-be97-1dafb19ebc0b,network=Network(5ef428cf-e408-490c-8f60-70934f760f4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf0429a54-9d') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:24:18 user nova-compute[72351]: DEBUG nova.objects.instance [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lazy-loading 'pci_devices' on Instance uuid 5c71284c-df41-4990-9a46-7986982bf7e2 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:24:18 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:24:18 user nova-compute[72351]:   <uuid>5c71284c-df41-4990-9a46-7986982bf7e2</uuid>
Aug 09 02:24:18 user nova-compute[72351]:   <name>instance-00000015</name>
Aug 09 02:24:18 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:24:18 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:24:18 user nova-compute[72351]:   <metadata>
Aug 09 02:24:18 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:24:18 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:24:18 user nova-compute[72351]:       <nova:name>tempest-TestVolumeBootPattern-image-snapshot-server-291879344</nova:name>
Aug 09 02:24:18 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:24:16</nova:creationTime>
Aug 09 02:24:18 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:24:18 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:24:18 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:24:18 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:24:18 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:24:18 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:24:18 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:24:18 user nova-compute[72351]:       <nova:owner>
Aug 09 02:24:18 user nova-compute[72351]:         <nova:user uuid="07d8d5a634274df19c2db769d2d27b29">tempest-TestVolumeBootPattern-350743460-project-member</nova:user>
Aug 09 02:24:18 user nova-compute[72351]:         <nova:project uuid="a4c6bbfd8d904ba1ac69ba5da4564513">tempest-TestVolumeBootPattern-350743460</nova:project>
Aug 09 02:24:18 user nova-compute[72351]:       </nova:owner>
Aug 09 02:24:18 user nova-compute[72351]:       <nova:root type="image" uuid="7869c8d4-caa3-435c-a4e4-e01eb5198e6f"/>
Aug 09 02:24:18 user nova-compute[72351]:       <nova:ports>
Aug 09 02:24:18 user nova-compute[72351]:         <nova:port uuid="f0429a54-9d4b-4bf9-be97-1dafb19ebc0b">
Aug 09 02:24:18 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.7" ipVersion="4"/>
Aug 09 02:24:18 user nova-compute[72351]:         </nova:port>
Aug 09 02:24:18 user nova-compute[72351]:       </nova:ports>
Aug 09 02:24:18 user nova-compute[72351]:     </nova:instance>
Aug 09 02:24:18 user nova-compute[72351]:   </metadata>
Aug 09 02:24:18 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:24:18 user nova-compute[72351]:     <system>
Aug 09 02:24:18 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:24:18 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:24:18 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:24:18 user nova-compute[72351]:       <entry name="serial">5c71284c-df41-4990-9a46-7986982bf7e2</entry>
Aug 09 02:24:18 user nova-compute[72351]:       <entry name="uuid">5c71284c-df41-4990-9a46-7986982bf7e2</entry>
Aug 09 02:24:18 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:24:18 user nova-compute[72351]:     </system>
Aug 09 02:24:18 user nova-compute[72351]:   </sysinfo>
Aug 09 02:24:18 user nova-compute[72351]:   <os>
Aug 09 02:24:18 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:24:18 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:24:18 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:24:18 user nova-compute[72351]:   </os>
Aug 09 02:24:18 user nova-compute[72351]:   <features>
Aug 09 02:24:18 user nova-compute[72351]:     <acpi/>
Aug 09 02:24:18 user nova-compute[72351]:     <apic/>
Aug 09 02:24:18 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:24:18 user nova-compute[72351]:   </features>
Aug 09 02:24:18 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:24:18 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:24:18 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:24:18 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:24:18 user nova-compute[72351]:   </clock>
Aug 09 02:24:18 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:24:18 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:24:18 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:24:18 user nova-compute[72351]:   </cpu>
Aug 09 02:24:18 user nova-compute[72351]:   <devices>
Aug 09 02:24:18 user nova-compute[72351]:     <disk type="block" device="disk">
Aug 09 02:24:18 user nova-compute[72351]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:24:18 user nova-compute[72351]:       <source dev="/dev/sdb"/>
Aug 09 02:24:18 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:24:18 user nova-compute[72351]:       <serial>c67f9b74-7547-4996-abf6-678dac1ff065</serial>
Aug 09 02:24:18 user nova-compute[72351]:     </disk>
Aug 09 02:24:18 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:24:18 user nova-compute[72351]:       <mac address="fa:16:3e:78:c7:55"/>
Aug 09 02:24:18 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:24:18 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:24:18 user nova-compute[72351]:       <target dev="tapf0429a54-9d"/>
Aug 09 02:24:18 user nova-compute[72351]:     </interface>
Aug 09 02:24:18 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:24:18 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/5c71284c-df41-4990-9a46-7986982bf7e2/console.log" append="off"/>
Aug 09 02:24:18 user nova-compute[72351]:     </serial>
Aug 09 02:24:18 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:24:18 user nova-compute[72351]:     <video>
Aug 09 02:24:18 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:24:18 user nova-compute[72351]:     </video>
Aug 09 02:24:18 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:24:18 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:24:18 user nova-compute[72351]:     </rng>
Aug 09 02:24:18 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:24:18 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:24:18 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:24:18 user nova-compute[72351]:     </memballoon>
Aug 09 02:24:18 user nova-compute[72351]:   </devices>
Aug 09 02:24:18 user nova-compute[72351]: </domain>
Aug 09 02:24:18 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:24:18 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:23:59Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-image-snapshot-server-291879344',display_name='tempest-TestVolumeBootPattern-image-snapshot-server-291879344',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testvolumebootpattern-image-snapshot-server-291879344',id=21,image_ref='7869c8d4-caa3-435c-a4e4-e01eb5198e6f',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMigRZU/OzbuGai7a1EcqwThXhvudiR9/Q8Da5xVMuKq+4DVrTM+XekLmyXUIzTgyNFsg9BxRHjyOIIoT5ZbBndJV38K0BzyxRl30KiADrnKDbDnCylyfZZ9qqf+nevzqg==',key_name='tempest-keypair-2017099597',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='a4c6bbfd8d904ba1ac69ba5da4564513',ramdisk_id='',reservation_id='r-caoazglt',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_bdm_v2='True',image_boot_roles='reader,member',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_project_name='tempest-TestVolumeBootPattern-350743460',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',image_owner_user_name='tempest-TestVolumeBootPattern-350743460-project-member',image_root_device_name='/dev/vda',network_allocated='True',owner_project_name='tempest-TestVolumeBootPattern-350743460',owner_user_name='tempest-TestVolumeBootPattern-350743460-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:24:01Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='07d8d5a634274df19c2db769d2d27b29',uuid=5c71284c-df41-4990-9a46-7986982bf7e2,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "f0429a54-9d4b-4bf9-be97-1dafb19ebc0b", "address": "fa:16:3e:78:c7:55", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf0429a54-9d", "ovs_interfaceid": "f0429a54-9d4b-4bf9-be97-1dafb19ebc0b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:24:18 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Converting VIF {"id": "f0429a54-9d4b-4bf9-be97-1dafb19ebc0b", "address": "fa:16:3e:78:c7:55", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf0429a54-9d", "ovs_interfaceid": "f0429a54-9d4b-4bf9-be97-1dafb19ebc0b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:24:18 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:78:c7:55,bridge_name='br-int',has_traffic_filtering=True,id=f0429a54-9d4b-4bf9-be97-1dafb19ebc0b,network=Network(5ef428cf-e408-490c-8f60-70934f760f4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf0429a54-9d') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:24:18 user nova-compute[72351]: DEBUG os_vif [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:78:c7:55,bridge_name='br-int',has_traffic_filtering=True,id=f0429a54-9d4b-4bf9-be97-1dafb19ebc0b,network=Network(5ef428cf-e408-490c-8f60-70934f760f4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf0429a54-9d') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:24:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:24:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:24:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapf0429a54-9d, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:24:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapf0429a54-9d, col_values=(('external_ids', {'iface-id': 'f0429a54-9d4b-4bf9-be97-1dafb19ebc0b', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:78:c7:55', 'vm-uuid': '5c71284c-df41-4990-9a46-7986982bf7e2'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:24:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:24:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:18 user nova-compute[72351]: INFO os_vif [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:78:c7:55,bridge_name='br-int',has_traffic_filtering=True,id=f0429a54-9d4b-4bf9-be97-1dafb19ebc0b,network=Network(5ef428cf-e408-490c-8f60-70934f760f4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf0429a54-9d')
Aug 09 02:24:18 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c4f530e6-9b4d-4e59-9808-db9fcfe767fe tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Acquiring lock "2d78475c-107d-4256-b12e-dbe9c8f3b9b2" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:24:18 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c4f530e6-9b4d-4e59-9808-db9fcfe767fe tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Lock "2d78475c-107d-4256-b12e-dbe9c8f3b9b2" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:24:18 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c4f530e6-9b4d-4e59-9808-db9fcfe767fe tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Acquiring lock "2d78475c-107d-4256-b12e-dbe9c8f3b9b2-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:24:18 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c4f530e6-9b4d-4e59-9808-db9fcfe767fe tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Lock "2d78475c-107d-4256-b12e-dbe9c8f3b9b2-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:24:18 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c4f530e6-9b4d-4e59-9808-db9fcfe767fe tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Lock "2d78475c-107d-4256-b12e-dbe9c8f3b9b2-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:24:18 user nova-compute[72351]: INFO nova.compute.manager [None req-c4f530e6-9b4d-4e59-9808-db9fcfe767fe tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Terminating instance
Aug 09 02:24:18 user nova-compute[72351]: DEBUG nova.compute.manager [None req-c4f530e6-9b4d-4e59-9808-db9fcfe767fe tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Start destroying the instance on the hypervisor. {{(pid=72351) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3124}}
Aug 09 02:24:18 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:24:18 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] No VIF found with MAC fa:16:3e:78:c7:55, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:24:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:18 user nova-compute[72351]: DEBUG nova.compute.manager [req-be8289d0-1ffc-4481-9056-f9d847190c96 req-477793ac-e1ca-4aa9-86d8-290c2a536f04 service nova] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Received event network-vif-unplugged-ff25eb15-1af0-4110-9d9a-ecd0a1d4627d {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:24:18 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-be8289d0-1ffc-4481-9056-f9d847190c96 req-477793ac-e1ca-4aa9-86d8-290c2a536f04 service nova] Acquiring lock "2d78475c-107d-4256-b12e-dbe9c8f3b9b2-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:24:18 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-be8289d0-1ffc-4481-9056-f9d847190c96 req-477793ac-e1ca-4aa9-86d8-290c2a536f04 service nova] Lock "2d78475c-107d-4256-b12e-dbe9c8f3b9b2-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:24:18 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-be8289d0-1ffc-4481-9056-f9d847190c96 req-477793ac-e1ca-4aa9-86d8-290c2a536f04 service nova] Lock "2d78475c-107d-4256-b12e-dbe9c8f3b9b2-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:24:18 user nova-compute[72351]: DEBUG nova.compute.manager [req-be8289d0-1ffc-4481-9056-f9d847190c96 req-477793ac-e1ca-4aa9-86d8-290c2a536f04 service nova] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] No waiting events found dispatching network-vif-unplugged-ff25eb15-1af0-4110-9d9a-ecd0a1d4627d {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:24:18 user nova-compute[72351]: DEBUG nova.compute.manager [req-be8289d0-1ffc-4481-9056-f9d847190c96 req-477793ac-e1ca-4aa9-86d8-290c2a536f04 service nova] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Received event network-vif-unplugged-ff25eb15-1af0-4110-9d9a-ecd0a1d4627d for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:24:19 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:19 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:19 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:19 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:19 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Acquiring lock "831c864a-75d9-42b3-b952-749f4201e40e" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:24:19 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "831c864a-75d9-42b3-b952-749f4201e40e" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:24:19 user nova-compute[72351]: DEBUG nova.compute.manager [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Starting instance... {{(pid=72351) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2406}}
Aug 09 02:24:19 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:24:19 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:24:19 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72351) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Aug 09 02:24:19 user nova-compute[72351]: INFO nova.compute.claims [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Claim successful on node user
Aug 09 02:24:19 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Instance destroyed successfully.
Aug 09 02:24:19 user nova-compute[72351]: DEBUG nova.objects.instance [None req-c4f530e6-9b4d-4e59-9808-db9fcfe767fe tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Lazy-loading 'resources' on Instance uuid 2d78475c-107d-4256-b12e-dbe9c8f3b9b2 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:24:19 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-c4f530e6-9b4d-4e59-9808-db9fcfe767fe tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:17:04Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerStableDeviceRescueTest-server-1763341201',display_name='tempest-ServerStableDeviceRescueTest-server-1763341201',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serverstabledevicerescuetest-server-1763341201',id=2,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBPLsnKPb6VYEzHtPIe2+hjQmOnmHJvA1Ev7/Puz1ZEEURwvwHmMhKqMbBr/N8dvlnbS9aRgyU274IVV3B6lTROiOE1e6tZ3774qYOPMP3+6C+tI9LZ4+zAxuQqNbLNbP+Q==',key_name='tempest-keypair-1212195529',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:21:10Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='747d5750a2174aaea71849c70167a816',ramdisk_id='',reservation_id='r-d0n2amoe',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerStableDeviceRescueTest-790832465',owner_user_name='tempest-ServerStableDeviceRescueTest-790832465-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:23:00Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='157ad7fa9970477a91de3d1347dee441',uuid=2d78475c-107d-4256-b12e-dbe9c8f3b9b2,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "ff25eb15-1af0-4110-9d9a-ecd0a1d4627d", "address": "fa:16:3e:0a:89:6a", "network": {"id": "fd1fe4e0-5d09-43c1-8dbc-1f60cb7f38d7", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1014783449-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.229", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "747d5750a2174aaea71849c70167a816", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapff25eb15-1a", "ovs_interfaceid": "ff25eb15-1af0-4110-9d9a-ecd0a1d4627d", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:24:19 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-c4f530e6-9b4d-4e59-9808-db9fcfe767fe tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Converting VIF {"id": "ff25eb15-1af0-4110-9d9a-ecd0a1d4627d", "address": "fa:16:3e:0a:89:6a", "network": {"id": "fd1fe4e0-5d09-43c1-8dbc-1f60cb7f38d7", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1014783449-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.229", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "747d5750a2174aaea71849c70167a816", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapff25eb15-1a", "ovs_interfaceid": "ff25eb15-1af0-4110-9d9a-ecd0a1d4627d", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:24:19 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-c4f530e6-9b4d-4e59-9808-db9fcfe767fe tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:0a:89:6a,bridge_name='br-int',has_traffic_filtering=True,id=ff25eb15-1af0-4110-9d9a-ecd0a1d4627d,network=Network(fd1fe4e0-5d09-43c1-8dbc-1f60cb7f38d7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapff25eb15-1a') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:24:19 user nova-compute[72351]: DEBUG os_vif [None req-c4f530e6-9b4d-4e59-9808-db9fcfe767fe tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:0a:89:6a,bridge_name='br-int',has_traffic_filtering=True,id=ff25eb15-1af0-4110-9d9a-ecd0a1d4627d,network=Network(fd1fe4e0-5d09-43c1-8dbc-1f60cb7f38d7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapff25eb15-1a') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:24:19 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:19 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapff25eb15-1a, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:24:19 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:19 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:24:19 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:19 user nova-compute[72351]: INFO os_vif [None req-c4f530e6-9b4d-4e59-9808-db9fcfe767fe tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:0a:89:6a,bridge_name='br-int',has_traffic_filtering=True,id=ff25eb15-1af0-4110-9d9a-ecd0a1d4627d,network=Network(fd1fe4e0-5d09-43c1-8dbc-1f60cb7f38d7),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapff25eb15-1a')
Aug 09 02:24:19 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-c4f530e6-9b4d-4e59-9808-db9fcfe767fe tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Deleting instance files /opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2_del
Aug 09 02:24:19 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-c4f530e6-9b4d-4e59-9808-db9fcfe767fe tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Deletion of /opt/stack/data/nova/instances/2d78475c-107d-4256-b12e-dbe9c8f3b9b2_del complete
Aug 09 02:24:19 user nova-compute[72351]: INFO nova.compute.manager [None req-c4f530e6-9b4d-4e59-9808-db9fcfe767fe tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Took 1.68 seconds to destroy the instance on the hypervisor.
Aug 09 02:24:19 user nova-compute[72351]: DEBUG oslo.service.loopingcall [None req-c4f530e6-9b4d-4e59-9808-db9fcfe767fe tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72351) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Aug 09 02:24:19 user nova-compute[72351]: DEBUG nova.compute.manager [-] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Deallocating network for instance {{(pid=72351) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2263}}
Aug 09 02:24:19 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] deallocate_for_instance() {{(pid=72351) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1795}}
Aug 09 02:24:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:20 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:24:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:20 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:24:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.638s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:24:20 user nova-compute[72351]: DEBUG nova.compute.manager [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Start building networks asynchronously for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2803}}
Aug 09 02:24:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:20 user nova-compute[72351]: DEBUG nova.compute.manager [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Allocating IP information in the background. {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1956}}
Aug 09 02:24:20 user nova-compute[72351]: DEBUG nova.network.neutron [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] allocate_for_instance() {{(pid=72351) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Aug 09 02:24:20 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Aug 09 02:24:20 user nova-compute[72351]: DEBUG nova.compute.manager [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Start building block device mappings for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2838}}
Aug 09 02:24:20 user nova-compute[72351]: DEBUG nova.compute.manager [req-d3c52b56-5b1c-4443-aec7-ffe685ee821d req-fd0da846-046a-48a4-a51d-7a40b0720059 service nova] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Received event network-vif-plugged-ff25eb15-1af0-4110-9d9a-ecd0a1d4627d {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:24:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d3c52b56-5b1c-4443-aec7-ffe685ee821d req-fd0da846-046a-48a4-a51d-7a40b0720059 service nova] Acquiring lock "2d78475c-107d-4256-b12e-dbe9c8f3b9b2-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:24:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d3c52b56-5b1c-4443-aec7-ffe685ee821d req-fd0da846-046a-48a4-a51d-7a40b0720059 service nova] Lock "2d78475c-107d-4256-b12e-dbe9c8f3b9b2-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:24:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d3c52b56-5b1c-4443-aec7-ffe685ee821d req-fd0da846-046a-48a4-a51d-7a40b0720059 service nova] Lock "2d78475c-107d-4256-b12e-dbe9c8f3b9b2-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:24:20 user nova-compute[72351]: DEBUG nova.compute.manager [req-d3c52b56-5b1c-4443-aec7-ffe685ee821d req-fd0da846-046a-48a4-a51d-7a40b0720059 service nova] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] No waiting events found dispatching network-vif-plugged-ff25eb15-1af0-4110-9d9a-ecd0a1d4627d {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:24:20 user nova-compute[72351]: WARNING nova.compute.manager [req-d3c52b56-5b1c-4443-aec7-ffe685ee821d req-fd0da846-046a-48a4-a51d-7a40b0720059 service nova] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Received unexpected event network-vif-plugged-ff25eb15-1af0-4110-9d9a-ecd0a1d4627d for instance with vm_state active and task_state deleting.
Aug 09 02:24:20 user nova-compute[72351]: DEBUG nova.compute.manager [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Start spawning the instance on the hypervisor. {{(pid=72351) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2612}}
Aug 09 02:24:20 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Creating instance directory {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4717}}
Aug 09 02:24:20 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Creating image(s)
Aug 09 02:24:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Acquiring lock "/opt/stack/data/nova/instances/831c864a-75d9-42b3-b952-749f4201e40e/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:24:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "/opt/stack/data/nova/instances/831c864a-75d9-42b3-b952-749f4201e40e/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:24:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "/opt/stack/data/nova/instances/831c864a-75d9-42b3-b952-749f4201e40e/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:24:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Acquiring lock "ad7b4169e2e08fd64509e88d2ad0f00130ea9294" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:24:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "ad7b4169e2e08fd64509e88d2ad0f00130ea9294" acquired by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:24:20 user nova-compute[72351]: DEBUG nova.compute.manager [req-4cd60515-7753-4f43-a1e4-cb80234ff88f req-6ad71501-55af-45d4-908c-9b3325f86373 service nova] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Received event network-vif-plugged-f0429a54-9d4b-4bf9-be97-1dafb19ebc0b {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:24:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-4cd60515-7753-4f43-a1e4-cb80234ff88f req-6ad71501-55af-45d4-908c-9b3325f86373 service nova] Acquiring lock "5c71284c-df41-4990-9a46-7986982bf7e2-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:24:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-4cd60515-7753-4f43-a1e4-cb80234ff88f req-6ad71501-55af-45d4-908c-9b3325f86373 service nova] Lock "5c71284c-df41-4990-9a46-7986982bf7e2-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:24:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-4cd60515-7753-4f43-a1e4-cb80234ff88f req-6ad71501-55af-45d4-908c-9b3325f86373 service nova] Lock "5c71284c-df41-4990-9a46-7986982bf7e2-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:24:20 user nova-compute[72351]: DEBUG nova.compute.manager [req-4cd60515-7753-4f43-a1e4-cb80234ff88f req-6ad71501-55af-45d4-908c-9b3325f86373 service nova] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] No waiting events found dispatching network-vif-plugged-f0429a54-9d4b-4bf9-be97-1dafb19ebc0b {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:24:20 user nova-compute[72351]: WARNING nova.compute.manager [req-4cd60515-7753-4f43-a1e4-cb80234ff88f req-6ad71501-55af-45d4-908c-9b3325f86373 service nova] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Received unexpected event network-vif-plugged-f0429a54-9d4b-4bf9-be97-1dafb19ebc0b for instance with vm_state building and task_state spawning.
Aug 09 02:24:20 user nova-compute[72351]: DEBUG nova.policy [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'bec5b5704e9b4e3480f428d5c037c20d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c2f67da658d94fabbab82ee6a6171c4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72351) authorize /opt/stack/nova/nova/policy.py:203}}
Aug 09 02:24:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/ad7b4169e2e08fd64509e88d2ad0f00130ea9294.part --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/ad7b4169e2e08fd64509e88d2ad0f00130ea9294.part --force-share --output=json" returned: 0 in 0.250s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:24:21 user nova-compute[72351]: DEBUG nova.virt.images [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] 16da0824-5647-4984-9a67-fc86d723ef34 was qcow2, converting to raw {{(pid=72351) fetch_to_raw /opt/stack/nova/nova/virt/images.py:165}}
Aug 09 02:24:21 user nova-compute[72351]: DEBUG nova.privsep.utils [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=72351) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Aug 09 02:24:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Running cmd (subprocess): qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/ad7b4169e2e08fd64509e88d2ad0f00130ea9294.part /opt/stack/data/nova/instances/_base/ad7b4169e2e08fd64509e88d2ad0f00130ea9294.converted {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:21 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:24:21 user nova-compute[72351]: DEBUG nova.compute.manager [req-aed47903-9b6f-474e-bb78-2a09e2c830fc req-86dae154-b173-452e-906b-12550a1a3142 service nova] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Received event network-vif-deleted-ff25eb15-1af0-4110-9d9a-ecd0a1d4627d {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:24:21 user nova-compute[72351]: INFO nova.compute.manager [req-aed47903-9b6f-474e-bb78-2a09e2c830fc req-86dae154-b173-452e-906b-12550a1a3142 service nova] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Neutron deleted interface ff25eb15-1af0-4110-9d9a-ecd0a1d4627d; detaching it from the instance and deleting it from the info cache
Aug 09 02:24:21 user nova-compute[72351]: DEBUG nova.network.neutron [req-aed47903-9b6f-474e-bb78-2a09e2c830fc req-86dae154-b173-452e-906b-12550a1a3142 service nova] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:24:21 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Took 2.01 seconds to deallocate network for instance.
Aug 09 02:24:22 user nova-compute[72351]: DEBUG nova.compute.manager [req-aed47903-9b6f-474e-bb78-2a09e2c830fc req-86dae154-b173-452e-906b-12550a1a3142 service nova] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Detach interface failed, port_id=ff25eb15-1af0-4110-9d9a-ecd0a1d4627d, reason: Instance 2d78475c-107d-4256-b12e-dbe9c8f3b9b2 could not be found. {{(pid=72351) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10866}}
Aug 09 02:24:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] CMD "qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/ad7b4169e2e08fd64509e88d2ad0f00130ea9294.part /opt/stack/data/nova/instances/_base/ad7b4169e2e08fd64509e88d2ad0f00130ea9294.converted" returned: 0 in 0.481s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:24:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/ad7b4169e2e08fd64509e88d2ad0f00130ea9294.converted --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c4f530e6-9b4d-4e59-9808-db9fcfe767fe tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:24:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c4f530e6-9b4d-4e59-9808-db9fcfe767fe tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:24:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/ad7b4169e2e08fd64509e88d2ad0f00130ea9294.converted --force-share --output=json" returned: 0 in 0.193s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:24:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "ad7b4169e2e08fd64509e88d2ad0f00130ea9294" "released" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: held 1.502s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:24:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/ad7b4169e2e08fd64509e88d2ad0f00130ea9294 --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:22 user nova-compute[72351]: DEBUG nova.network.neutron [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Successfully created port: 02c9ff8a-e779-443c-9b3b-567d2e69cea0 {{(pid=72351) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Aug 09 02:24:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/ad7b4169e2e08fd64509e88d2ad0f00130ea9294 --force-share --output=json" returned: 0 in 0.154s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:24:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Acquiring lock "ad7b4169e2e08fd64509e88d2ad0f00130ea9294" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:24:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "ad7b4169e2e08fd64509e88d2ad0f00130ea9294" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:24:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/ad7b4169e2e08fd64509e88d2ad0f00130ea9294 --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/ad7b4169e2e08fd64509e88d2ad0f00130ea9294 --force-share --output=json" returned: 0 in 0.145s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:24:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/ad7b4169e2e08fd64509e88d2ad0f00130ea9294,backing_fmt=raw /opt/stack/data/nova/instances/831c864a-75d9-42b3-b952-749f4201e40e/disk 1073741824 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/ad7b4169e2e08fd64509e88d2ad0f00130ea9294,backing_fmt=raw /opt/stack/data/nova/instances/831c864a-75d9-42b3-b952-749f4201e40e/disk 1073741824" returned: 0 in 0.071s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:24:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "ad7b4169e2e08fd64509e88d2ad0f00130ea9294" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.223s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:24:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/ad7b4169e2e08fd64509e88d2ad0f00130ea9294 --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:22 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-c4f530e6-9b4d-4e59-9808-db9fcfe767fe tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:24:22 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-c4f530e6-9b4d-4e59-9808-db9fcfe767fe tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:24:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c4f530e6-9b4d-4e59-9808-db9fcfe767fe tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.674s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:24:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/ad7b4169e2e08fd64509e88d2ad0f00130ea9294 --force-share --output=json" returned: 0 in 0.161s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:24:22 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Checking if we can resize image /opt/stack/data/nova/instances/831c864a-75d9-42b3-b952-749f4201e40e/disk. size=1073741824 {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Aug 09 02:24:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/831c864a-75d9-42b3-b952-749f4201e40e/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:22 user nova-compute[72351]: INFO nova.scheduler.client.report [None req-c4f530e6-9b4d-4e59-9808-db9fcfe767fe tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Deleted allocations for instance 2d78475c-107d-4256-b12e-dbe9c8f3b9b2
Aug 09 02:24:22 user nova-compute[72351]: DEBUG nova.compute.manager [req-aeff2a65-02bf-4a64-b26e-fcf2415618dd req-a52f8fb9-7589-40d5-b33b-3a46a9a5c93e service nova] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Received event network-vif-plugged-f0429a54-9d4b-4bf9-be97-1dafb19ebc0b {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:24:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-aeff2a65-02bf-4a64-b26e-fcf2415618dd req-a52f8fb9-7589-40d5-b33b-3a46a9a5c93e service nova] Acquiring lock "5c71284c-df41-4990-9a46-7986982bf7e2-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:24:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-aeff2a65-02bf-4a64-b26e-fcf2415618dd req-a52f8fb9-7589-40d5-b33b-3a46a9a5c93e service nova] Lock "5c71284c-df41-4990-9a46-7986982bf7e2-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:24:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-aeff2a65-02bf-4a64-b26e-fcf2415618dd req-a52f8fb9-7589-40d5-b33b-3a46a9a5c93e service nova] Lock "5c71284c-df41-4990-9a46-7986982bf7e2-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:24:22 user nova-compute[72351]: DEBUG nova.compute.manager [req-aeff2a65-02bf-4a64-b26e-fcf2415618dd req-a52f8fb9-7589-40d5-b33b-3a46a9a5c93e service nova] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] No waiting events found dispatching network-vif-plugged-f0429a54-9d4b-4bf9-be97-1dafb19ebc0b {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:24:22 user nova-compute[72351]: WARNING nova.compute.manager [req-aeff2a65-02bf-4a64-b26e-fcf2415618dd req-a52f8fb9-7589-40d5-b33b-3a46a9a5c93e service nova] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Received unexpected event network-vif-plugged-f0429a54-9d4b-4bf9-be97-1dafb19ebc0b for instance with vm_state building and task_state spawning.
Aug 09 02:24:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c4f530e6-9b4d-4e59-9808-db9fcfe767fe tempest-ServerStableDeviceRescueTest-790832465 tempest-ServerStableDeviceRescueTest-790832465-project-member] Lock "2d78475c-107d-4256-b12e-dbe9c8f3b9b2" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 4.740s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:24:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/831c864a-75d9-42b3-b952-749f4201e40e/disk --force-share --output=json" returned: 0 in 0.189s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:24:23 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Cannot resize image /opt/stack/data/nova/instances/831c864a-75d9-42b3-b952-749f4201e40e/disk to a smaller size. {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Aug 09 02:24:23 user nova-compute[72351]: DEBUG nova.objects.instance [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lazy-loading 'migration_context' on Instance uuid 831c864a-75d9-42b3-b952-749f4201e40e {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:24:23 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Created local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4851}}
Aug 09 02:24:23 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Ensure instance console log exists: /opt/stack/data/nova/instances/831c864a-75d9-42b3-b952-749f4201e40e/console.log {{(pid=72351) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4603}}
Aug 09 02:24:23 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:24:23 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:24:23 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:24:23 user nova-compute[72351]: DEBUG nova.compute.manager [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:24:23 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Guest created on hypervisor {{(pid=72351) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4411}}
Aug 09 02:24:23 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547863.1294153, 5c71284c-df41-4990-9a46-7986982bf7e2 => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:24:23 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] VM Resumed (Lifecycle Event)
Aug 09 02:24:23 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Instance spawned successfully.
Aug 09 02:24:23 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Attempting to register defaults for the following image properties: ['hw_input_bus', 'hw_pointer_model'] {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:894}}
Aug 09 02:24:23 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:24:23 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Found default for hw_input_bus of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:24:23 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Found default for hw_pointer_model of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:24:23 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:24:23 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:24:23 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547863.1295445, 5c71284c-df41-4990-9a46-7986982bf7e2 => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:24:23 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] VM Started (Lifecycle Event)
Aug 09 02:24:23 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:24:23 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:24:23 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:24:23 user nova-compute[72351]: INFO nova.compute.manager [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Took 6.61 seconds to spawn the instance on the hypervisor.
Aug 09 02:24:23 user nova-compute[72351]: DEBUG nova.compute.manager [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:24:23 user nova-compute[72351]: DEBUG nova.network.neutron [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Successfully updated port: 02c9ff8a-e779-443c-9b3b-567d2e69cea0 {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:24:23 user nova-compute[72351]: INFO nova.compute.manager [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Took 22.60 seconds to build instance.
Aug 09 02:24:23 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Acquiring lock "refresh_cache-831c864a-75d9-42b3-b952-749f4201e40e" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:24:23 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Acquired lock "refresh_cache-831c864a-75d9-42b3-b952-749f4201e40e" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:24:23 user nova-compute[72351]: DEBUG nova.network.neutron [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:24:23 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-edfc6334-b9a5-4fac-9c50-46c705b8c2d0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "5c71284c-df41-4990-9a46-7986982bf7e2" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 22.742s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:24:23 user nova-compute[72351]: DEBUG nova.network.neutron [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:24:23 user nova-compute[72351]: DEBUG nova.network.neutron [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Updating instance_info_cache with network_info: [{"id": "02c9ff8a-e779-443c-9b3b-567d2e69cea0", "address": "fa:16:3e:32:5d:c2", "network": {"id": "7024d1f6-a400-4482-9364-78ae3b6adc80", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1603258958-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c2f67da658d94fabbab82ee6a6171c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap02c9ff8a-e7", "ovs_interfaceid": "02c9ff8a-e779-443c-9b3b-567d2e69cea0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:24:23 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Releasing lock "refresh_cache-831c864a-75d9-42b3-b952-749f4201e40e" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:24:23 user nova-compute[72351]: DEBUG nova.compute.manager [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Instance network_info: |[{"id": "02c9ff8a-e779-443c-9b3b-567d2e69cea0", "address": "fa:16:3e:32:5d:c2", "network": {"id": "7024d1f6-a400-4482-9364-78ae3b6adc80", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1603258958-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c2f67da658d94fabbab82ee6a6171c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap02c9ff8a-e7", "ovs_interfaceid": "02c9ff8a-e779-443c-9b3b-567d2e69cea0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1971}}
Aug 09 02:24:23 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Start _get_guest_xml network_info=[{"id": "02c9ff8a-e779-443c-9b3b-567d2e69cea0", "address": "fa:16:3e:32:5d:c2", "network": {"id": "7024d1f6-a400-4482-9364-78ae3b6adc80", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1603258958-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c2f67da658d94fabbab82ee6a6171c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap02c9ff8a-e7", "ovs_interfaceid": "02c9ff8a-e779-443c-9b3b-567d2e69cea0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:24:16Z,direct_url=<?>,disk_format='qcow2',id=16da0824-5647-4984-9a67-fc86d723ef34,min_disk=0,min_ram=0,name='tempest-scenario-img--1088532137',owner='c2f67da658d94fabbab82ee6a6171c4b',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:24:18Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'size': 0, 'device_type': 'disk', 'encryption_secret_uuid': None, 'guest_format': None, 'encryption_options': None, 'disk_bus': 'virtio', 'encrypted': False, 'encryption_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'image_id': '16da0824-5647-4984-9a67-fc86d723ef34'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:24:23 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:24:23 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:24:23 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:24:23 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:24:23 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:24:23 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:24:23 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:24:23 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:24:16Z,direct_url=<?>,disk_format='qcow2',id=16da0824-5647-4984-9a67-fc86d723ef34,min_disk=0,min_ram=0,name='tempest-scenario-img--1088532137',owner='c2f67da658d94fabbab82ee6a6171c4b',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:24:18Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:24:23 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:24:23 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:24:23 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:24:23 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:24:23 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:24:23 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:24:23 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:24:23 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:24:23 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:24:23 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:24:24 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:24:18Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestMinimumBasicScenario-server-1449267845',display_name='tempest-TestMinimumBasicScenario-server-1449267845',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testminimumbasicscenario-server-1449267845',id=23,image_ref='16da0824-5647-4984-9a67-fc86d723ef34',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMvWYiT3PAj8t6XXpOzeSg0zsvJtWfK76LZNXd5TZSvLf67t8TWrmwgsa6xs6zEQiNUAl4jqZiCkH4fjuSHiqFnl0NsYYcCcqIbK07RAKSi5ixt23WHp3vhu37izfOEYbg==',key_name='tempest-TestMinimumBasicScenario-1710540764',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='c2f67da658d94fabbab82ee6a6171c4b',ramdisk_id='',reservation_id='r-c33grfp2',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='16da0824-5647-4984-9a67-fc86d723ef34',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_min_disk='1',image_min_ram='0',network_allocated='True',owner_project_name='tempest-TestMinimumBasicScenario-1380132717',owner_user_name='tempest-TestMinimumBasicScenario-1380132717-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:24:21Z,user_data=None,user_id='bec5b5704e9b4e3480f428d5c037c20d',uuid=831c864a-75d9-42b3-b952-749f4201e40e,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "02c9ff8a-e779-443c-9b3b-567d2e69cea0", "address": "fa:16:3e:32:5d:c2", "network": {"id": "7024d1f6-a400-4482-9364-78ae3b6adc80", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1603258958-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c2f67da658d94fabbab82ee6a6171c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap02c9ff8a-e7", "ovs_interfaceid": "02c9ff8a-e779-443c-9b3b-567d2e69cea0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:24:24 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Converting VIF {"id": "02c9ff8a-e779-443c-9b3b-567d2e69cea0", "address": "fa:16:3e:32:5d:c2", "network": {"id": "7024d1f6-a400-4482-9364-78ae3b6adc80", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1603258958-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c2f67da658d94fabbab82ee6a6171c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap02c9ff8a-e7", "ovs_interfaceid": "02c9ff8a-e779-443c-9b3b-567d2e69cea0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:24:24 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:32:5d:c2,bridge_name='br-int',has_traffic_filtering=True,id=02c9ff8a-e779-443c-9b3b-567d2e69cea0,network=Network(7024d1f6-a400-4482-9364-78ae3b6adc80),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap02c9ff8a-e7') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:24:24 user nova-compute[72351]: DEBUG nova.objects.instance [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lazy-loading 'pci_devices' on Instance uuid 831c864a-75d9-42b3-b952-749f4201e40e {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:24:24 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:24:24 user nova-compute[72351]:   <uuid>831c864a-75d9-42b3-b952-749f4201e40e</uuid>
Aug 09 02:24:24 user nova-compute[72351]:   <name>instance-00000017</name>
Aug 09 02:24:24 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:24:24 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:24:24 user nova-compute[72351]:   <metadata>
Aug 09 02:24:24 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:24:24 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:24:24 user nova-compute[72351]:       <nova:name>tempest-TestMinimumBasicScenario-server-1449267845</nova:name>
Aug 09 02:24:24 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:24:23</nova:creationTime>
Aug 09 02:24:24 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:24:24 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:24:24 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:24:24 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:24:24 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:24:24 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:24:24 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:24:24 user nova-compute[72351]:       <nova:owner>
Aug 09 02:24:24 user nova-compute[72351]:         <nova:user uuid="bec5b5704e9b4e3480f428d5c037c20d">tempest-TestMinimumBasicScenario-1380132717-project-member</nova:user>
Aug 09 02:24:24 user nova-compute[72351]:         <nova:project uuid="c2f67da658d94fabbab82ee6a6171c4b">tempest-TestMinimumBasicScenario-1380132717</nova:project>
Aug 09 02:24:24 user nova-compute[72351]:       </nova:owner>
Aug 09 02:24:24 user nova-compute[72351]:       <nova:root type="image" uuid="16da0824-5647-4984-9a67-fc86d723ef34"/>
Aug 09 02:24:24 user nova-compute[72351]:       <nova:ports>
Aug 09 02:24:24 user nova-compute[72351]:         <nova:port uuid="02c9ff8a-e779-443c-9b3b-567d2e69cea0">
Aug 09 02:24:24 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.10" ipVersion="4"/>
Aug 09 02:24:24 user nova-compute[72351]:         </nova:port>
Aug 09 02:24:24 user nova-compute[72351]:       </nova:ports>
Aug 09 02:24:24 user nova-compute[72351]:     </nova:instance>
Aug 09 02:24:24 user nova-compute[72351]:   </metadata>
Aug 09 02:24:24 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:24:24 user nova-compute[72351]:     <system>
Aug 09 02:24:24 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:24:24 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:24:24 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:24:24 user nova-compute[72351]:       <entry name="serial">831c864a-75d9-42b3-b952-749f4201e40e</entry>
Aug 09 02:24:24 user nova-compute[72351]:       <entry name="uuid">831c864a-75d9-42b3-b952-749f4201e40e</entry>
Aug 09 02:24:24 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:24:24 user nova-compute[72351]:     </system>
Aug 09 02:24:24 user nova-compute[72351]:   </sysinfo>
Aug 09 02:24:24 user nova-compute[72351]:   <os>
Aug 09 02:24:24 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:24:24 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:24:24 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:24:24 user nova-compute[72351]:   </os>
Aug 09 02:24:24 user nova-compute[72351]:   <features>
Aug 09 02:24:24 user nova-compute[72351]:     <acpi/>
Aug 09 02:24:24 user nova-compute[72351]:     <apic/>
Aug 09 02:24:24 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:24:24 user nova-compute[72351]:   </features>
Aug 09 02:24:24 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:24:24 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:24:24 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:24:24 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:24:24 user nova-compute[72351]:   </clock>
Aug 09 02:24:24 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:24:24 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:24:24 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:24:24 user nova-compute[72351]:   </cpu>
Aug 09 02:24:24 user nova-compute[72351]:   <devices>
Aug 09 02:24:24 user nova-compute[72351]:     <disk type="file" device="disk">
Aug 09 02:24:24 user nova-compute[72351]:       <driver name="qemu" type="qcow2" cache="none"/>
Aug 09 02:24:24 user nova-compute[72351]:       <source file="/opt/stack/data/nova/instances/831c864a-75d9-42b3-b952-749f4201e40e/disk"/>
Aug 09 02:24:24 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:24:24 user nova-compute[72351]:     </disk>
Aug 09 02:24:24 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:24:24 user nova-compute[72351]:       <mac address="fa:16:3e:32:5d:c2"/>
Aug 09 02:24:24 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:24:24 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:24:24 user nova-compute[72351]:       <target dev="tap02c9ff8a-e7"/>
Aug 09 02:24:24 user nova-compute[72351]:     </interface>
Aug 09 02:24:24 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:24:24 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/831c864a-75d9-42b3-b952-749f4201e40e/console.log" append="off"/>
Aug 09 02:24:24 user nova-compute[72351]:     </serial>
Aug 09 02:24:24 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:24:24 user nova-compute[72351]:     <video>
Aug 09 02:24:24 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:24:24 user nova-compute[72351]:     </video>
Aug 09 02:24:24 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:24:24 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:24:24 user nova-compute[72351]:     </rng>
Aug 09 02:24:24 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:24:24 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:24:24 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:24:24 user nova-compute[72351]:     </memballoon>
Aug 09 02:24:24 user nova-compute[72351]:   </devices>
Aug 09 02:24:24 user nova-compute[72351]: </domain>
Aug 09 02:24:24 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:24:24 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:24:18Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestMinimumBasicScenario-server-1449267845',display_name='tempest-TestMinimumBasicScenario-server-1449267845',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testminimumbasicscenario-server-1449267845',id=23,image_ref='16da0824-5647-4984-9a67-fc86d723ef34',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMvWYiT3PAj8t6XXpOzeSg0zsvJtWfK76LZNXd5TZSvLf67t8TWrmwgsa6xs6zEQiNUAl4jqZiCkH4fjuSHiqFnl0NsYYcCcqIbK07RAKSi5ixt23WHp3vhu37izfOEYbg==',key_name='tempest-TestMinimumBasicScenario-1710540764',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='c2f67da658d94fabbab82ee6a6171c4b',ramdisk_id='',reservation_id='r-c33grfp2',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='16da0824-5647-4984-9a67-fc86d723ef34',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_min_disk='1',image_min_ram='0',network_allocated='True',owner_project_name='tempest-TestMinimumBasicScenario-1380132717',owner_user_name='tempest-TestMinimumBasicScenario-1380132717-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:24:21Z,user_data=None,user_id='bec5b5704e9b4e3480f428d5c037c20d',uuid=831c864a-75d9-42b3-b952-749f4201e40e,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "02c9ff8a-e779-443c-9b3b-567d2e69cea0", "address": "fa:16:3e:32:5d:c2", "network": {"id": "7024d1f6-a400-4482-9364-78ae3b6adc80", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1603258958-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c2f67da658d94fabbab82ee6a6171c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap02c9ff8a-e7", "ovs_interfaceid": "02c9ff8a-e779-443c-9b3b-567d2e69cea0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:24:24 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Converting VIF {"id": "02c9ff8a-e779-443c-9b3b-567d2e69cea0", "address": "fa:16:3e:32:5d:c2", "network": {"id": "7024d1f6-a400-4482-9364-78ae3b6adc80", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1603258958-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c2f67da658d94fabbab82ee6a6171c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap02c9ff8a-e7", "ovs_interfaceid": "02c9ff8a-e779-443c-9b3b-567d2e69cea0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:24:24 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:32:5d:c2,bridge_name='br-int',has_traffic_filtering=True,id=02c9ff8a-e779-443c-9b3b-567d2e69cea0,network=Network(7024d1f6-a400-4482-9364-78ae3b6adc80),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap02c9ff8a-e7') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:24:24 user nova-compute[72351]: DEBUG os_vif [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:32:5d:c2,bridge_name='br-int',has_traffic_filtering=True,id=02c9ff8a-e779-443c-9b3b-567d2e69cea0,network=Network(7024d1f6-a400-4482-9364-78ae3b6adc80),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap02c9ff8a-e7') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:24:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:24:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:24:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap02c9ff8a-e7, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:24:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap02c9ff8a-e7, col_values=(('external_ids', {'iface-id': '02c9ff8a-e779-443c-9b3b-567d2e69cea0', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:32:5d:c2', 'vm-uuid': '831c864a-75d9-42b3-b952-749f4201e40e'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:24:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:24:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:24 user nova-compute[72351]: INFO os_vif [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:32:5d:c2,bridge_name='br-int',has_traffic_filtering=True,id=02c9ff8a-e779-443c-9b3b-567d2e69cea0,network=Network(7024d1f6-a400-4482-9364-78ae3b6adc80),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap02c9ff8a-e7')
Aug 09 02:24:24 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:24:24 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] No VIF found with MAC fa:16:3e:32:5d:c2, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:24:25 user nova-compute[72351]: DEBUG nova.compute.manager [req-4d58242b-d36e-4d27-a620-3b3ce6cc9a05 req-785a9fd1-50cc-4929-9531-f600305244bf service nova] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Received event network-changed-02c9ff8a-e779-443c-9b3b-567d2e69cea0 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:24:25 user nova-compute[72351]: DEBUG nova.compute.manager [req-4d58242b-d36e-4d27-a620-3b3ce6cc9a05 req-785a9fd1-50cc-4929-9531-f600305244bf service nova] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Refreshing instance network info cache due to event network-changed-02c9ff8a-e779-443c-9b3b-567d2e69cea0. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:24:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-4d58242b-d36e-4d27-a620-3b3ce6cc9a05 req-785a9fd1-50cc-4929-9531-f600305244bf service nova] Acquiring lock "refresh_cache-831c864a-75d9-42b3-b952-749f4201e40e" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:24:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-4d58242b-d36e-4d27-a620-3b3ce6cc9a05 req-785a9fd1-50cc-4929-9531-f600305244bf service nova] Acquired lock "refresh_cache-831c864a-75d9-42b3-b952-749f4201e40e" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:24:25 user nova-compute[72351]: DEBUG nova.network.neutron [req-4d58242b-d36e-4d27-a620-3b3ce6cc9a05 req-785a9fd1-50cc-4929-9531-f600305244bf service nova] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Refreshing network info cache for port 02c9ff8a-e779-443c-9b3b-567d2e69cea0 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:24:25 user nova-compute[72351]: DEBUG nova.compute.manager [None req-7f376306-dbe6-4463-b946-f29c638e2129 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:24:25 user nova-compute[72351]: INFO nova.compute.manager [None req-7f376306-dbe6-4463-b946-f29c638e2129 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] instance snapshotting
Aug 09 02:24:25 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-7f376306-dbe6-4463-b946-f29c638e2129 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Beginning live snapshot process
Aug 09 02:24:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:26 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-7f376306-dbe6-4463-b946-f29c638e2129 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json -f qcow2 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:26 user nova-compute[72351]: DEBUG nova.compute.manager [req-cfd0e84d-6ecd-44b1-90eb-b84996dbdcb9 req-448c8dd9-a623-4cd0-a5e4-c0788fcc472e service nova] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Received event network-vif-plugged-02c9ff8a-e779-443c-9b3b-567d2e69cea0 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:24:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-cfd0e84d-6ecd-44b1-90eb-b84996dbdcb9 req-448c8dd9-a623-4cd0-a5e4-c0788fcc472e service nova] Acquiring lock "831c864a-75d9-42b3-b952-749f4201e40e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:24:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-cfd0e84d-6ecd-44b1-90eb-b84996dbdcb9 req-448c8dd9-a623-4cd0-a5e4-c0788fcc472e service nova] Lock "831c864a-75d9-42b3-b952-749f4201e40e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:24:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-cfd0e84d-6ecd-44b1-90eb-b84996dbdcb9 req-448c8dd9-a623-4cd0-a5e4-c0788fcc472e service nova] Lock "831c864a-75d9-42b3-b952-749f4201e40e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:24:26 user nova-compute[72351]: DEBUG nova.compute.manager [req-cfd0e84d-6ecd-44b1-90eb-b84996dbdcb9 req-448c8dd9-a623-4cd0-a5e4-c0788fcc472e service nova] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] No waiting events found dispatching network-vif-plugged-02c9ff8a-e779-443c-9b3b-567d2e69cea0 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:24:26 user nova-compute[72351]: WARNING nova.compute.manager [req-cfd0e84d-6ecd-44b1-90eb-b84996dbdcb9 req-448c8dd9-a623-4cd0-a5e4-c0788fcc472e service nova] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Received unexpected event network-vif-plugged-02c9ff8a-e779-443c-9b3b-567d2e69cea0 for instance with vm_state building and task_state spawning.
Aug 09 02:24:26 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-7f376306-dbe6-4463-b946-f29c638e2129 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json -f qcow2" returned: 0 in 0.265s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:24:26 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-7f376306-dbe6-4463-b946-f29c638e2129 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json -f qcow2 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:26 user nova-compute[72351]: DEBUG nova.network.neutron [req-4d58242b-d36e-4d27-a620-3b3ce6cc9a05 req-785a9fd1-50cc-4929-9531-f600305244bf service nova] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Updated VIF entry in instance network info cache for port 02c9ff8a-e779-443c-9b3b-567d2e69cea0. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:24:26 user nova-compute[72351]: DEBUG nova.network.neutron [req-4d58242b-d36e-4d27-a620-3b3ce6cc9a05 req-785a9fd1-50cc-4929-9531-f600305244bf service nova] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Updating instance_info_cache with network_info: [{"id": "02c9ff8a-e779-443c-9b3b-567d2e69cea0", "address": "fa:16:3e:32:5d:c2", "network": {"id": "7024d1f6-a400-4482-9364-78ae3b6adc80", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1603258958-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c2f67da658d94fabbab82ee6a6171c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap02c9ff8a-e7", "ovs_interfaceid": "02c9ff8a-e779-443c-9b3b-567d2e69cea0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:24:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-4d58242b-d36e-4d27-a620-3b3ce6cc9a05 req-785a9fd1-50cc-4929-9531-f600305244bf service nova] Releasing lock "refresh_cache-831c864a-75d9-42b3-b952-749f4201e40e" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:24:26 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-7f376306-dbe6-4463-b946-f29c638e2129 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json -f qcow2" returned: 0 in 0.172s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:24:26 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-7f376306-dbe6-4463-b946-f29c638e2129 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:26 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:26 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-7f376306-dbe6-4463-b946-f29c638e2129 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.170s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:24:26 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-7f376306-dbe6-4463-b946-f29c638e2129 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/snapshots/tmpr35ij9ur/d8f4aef62f03433c9e62609c73237521.delta 1073741824 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:26 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:26 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-7f376306-dbe6-4463-b946-f29c638e2129 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/snapshots/tmpr35ij9ur/d8f4aef62f03433c9e62609c73237521.delta 1073741824" returned: 0 in 0.080s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:24:26 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-7f376306-dbe6-4463-b946-f29c638e2129 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Quiescing instance not available: QEMU guest agent is not enabled.
Aug 09 02:24:26 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Acquiring lock "0825f13f-c923-417d-8fa0-f4e61616e4db" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:24:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Lock "0825f13f-c923-417d-8fa0-f4e61616e4db" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.008s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:24:26 user nova-compute[72351]: DEBUG nova.compute.manager [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Starting instance... {{(pid=72351) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2406}}
Aug 09 02:24:27 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:24:27 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:24:27 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72351) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Aug 09 02:24:27 user nova-compute[72351]: INFO nova.compute.claims [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Claim successful on node user
Aug 09 02:24:27 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:24:27 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-7f376306-dbe6-4463-b946-f29c638e2129 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] COPY block job progress, current cursor: 0 final cursor: 75497472 {{(pid=72351) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Aug 09 02:24:27 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:24:27 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.679s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:24:27 user nova-compute[72351]: DEBUG nova.compute.manager [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Start building networks asynchronously for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2803}}
Aug 09 02:24:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:27 user nova-compute[72351]: DEBUG nova.compute.manager [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Allocating IP information in the background. {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1956}}
Aug 09 02:24:27 user nova-compute[72351]: DEBUG nova.network.neutron [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] allocate_for_instance() {{(pid=72351) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Aug 09 02:24:27 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Aug 09 02:24:28 user nova-compute[72351]: DEBUG nova.compute.manager [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Start building block device mappings for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2838}}
Aug 09 02:24:28 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-7f376306-dbe6-4463-b946-f29c638e2129 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] COPY block job progress, current cursor: 26542080 final cursor: 75497472 {{(pid=72351) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Aug 09 02:24:28 user nova-compute[72351]: DEBUG nova.compute.manager [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Start spawning the instance on the hypervisor. {{(pid=72351) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2612}}
Aug 09 02:24:28 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Creating instance directory {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4717}}
Aug 09 02:24:28 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Creating image(s)
Aug 09 02:24:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Acquiring lock "/opt/stack/data/nova/instances/0825f13f-c923-417d-8fa0-f4e61616e4db/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:24:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Lock "/opt/stack/data/nova/instances/0825f13f-c923-417d-8fa0-f4e61616e4db/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:24:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Lock "/opt/stack/data/nova/instances/0825f13f-c923-417d-8fa0-f4e61616e4db/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.009s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:24:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:28 user nova-compute[72351]: DEBUG nova.compute.manager [req-77312e6b-2280-489b-80a8-7b94ce02ab91 req-64d6410a-771c-4e29-b3a1-7dad21f28b17 service nova] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Received event network-vif-plugged-02c9ff8a-e779-443c-9b3b-567d2e69cea0 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:24:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-77312e6b-2280-489b-80a8-7b94ce02ab91 req-64d6410a-771c-4e29-b3a1-7dad21f28b17 service nova] Acquiring lock "831c864a-75d9-42b3-b952-749f4201e40e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:24:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-77312e6b-2280-489b-80a8-7b94ce02ab91 req-64d6410a-771c-4e29-b3a1-7dad21f28b17 service nova] Lock "831c864a-75d9-42b3-b952-749f4201e40e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:24:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-77312e6b-2280-489b-80a8-7b94ce02ab91 req-64d6410a-771c-4e29-b3a1-7dad21f28b17 service nova] Lock "831c864a-75d9-42b3-b952-749f4201e40e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:24:28 user nova-compute[72351]: DEBUG nova.compute.manager [req-77312e6b-2280-489b-80a8-7b94ce02ab91 req-64d6410a-771c-4e29-b3a1-7dad21f28b17 service nova] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] No waiting events found dispatching network-vif-plugged-02c9ff8a-e779-443c-9b3b-567d2e69cea0 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:24:28 user nova-compute[72351]: WARNING nova.compute.manager [req-77312e6b-2280-489b-80a8-7b94ce02ab91 req-64d6410a-771c-4e29-b3a1-7dad21f28b17 service nova] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Received unexpected event network-vif-plugged-02c9ff8a-e779-443c-9b3b-567d2e69cea0 for instance with vm_state building and task_state spawning.
Aug 09 02:24:28 user nova-compute[72351]: DEBUG nova.policy [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '693fd098694d41dd91edb717fd8170ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1af75c014509460eab5821b6a5c1e124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72351) authorize /opt/stack/nova/nova/policy.py:203}}
Aug 09 02:24:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.157s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:24:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Acquiring lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:24:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:24:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:28 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-7f376306-dbe6-4463-b946-f29c638e2129 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] COPY block job progress, current cursor: 75497472 final cursor: 75497472 {{(pid=72351) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Aug 09 02:24:28 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-7f376306-dbe6-4463-b946-f29c638e2129 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Skipping quiescing instance: QEMU guest agent is not enabled.
Aug 09 02:24:28 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547868.7190278, 831c864a-75d9-42b3-b952-749f4201e40e => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:24:28 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] VM Resumed (Lifecycle Event)
Aug 09 02:24:28 user nova-compute[72351]: DEBUG nova.compute.manager [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:24:28 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Guest created on hypervisor {{(pid=72351) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4411}}
Aug 09 02:24:28 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Instance spawned successfully.
Aug 09 02:24:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.189s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:24:28 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:894}}
Aug 09 02:24:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/0825f13f-c923-417d-8fa0-f4e61616e4db/disk 1073741824 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:28 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:24:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/0825f13f-c923-417d-8fa0-f4e61616e4db/disk 1073741824" returned: 0 in 0.075s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:24:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.274s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:24:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:28 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:24:28 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Found default for hw_cdrom_bus of ide {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:24:28 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Found default for hw_disk_bus of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:24:28 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Found default for hw_input_bus of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:24:28 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Found default for hw_pointer_model of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:24:28 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Found default for hw_video_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:24:28 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Found default for hw_vif_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:24:28 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:24:28 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547868.722199, 831c864a-75d9-42b3-b952-749f4201e40e => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:24:28 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] VM Started (Lifecycle Event)
Aug 09 02:24:28 user nova-compute[72351]: DEBUG nova.privsep.utils [None req-7f376306-dbe6-4463-b946-f29c638e2129 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=72351) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Aug 09 02:24:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-7f376306-dbe6-4463-b946-f29c638e2129 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Running cmd (subprocess): qemu-img convert -t none -O qcow2 -f qcow2 /opt/stack/data/nova/instances/snapshots/tmpr35ij9ur/d8f4aef62f03433c9e62609c73237521.delta /opt/stack/data/nova/instances/snapshots/tmpr35ij9ur/d8f4aef62f03433c9e62609c73237521 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:29 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:24:29 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.187s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:24:29 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Checking if we can resize image /opt/stack/data/nova/instances/0825f13f-c923-417d-8fa0-f4e61616e4db/disk. size=1073741824 {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Aug 09 02:24:29 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/0825f13f-c923-417d-8fa0-f4e61616e4db/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:29 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:29 user nova-compute[72351]: INFO nova.compute.manager [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Took 8.30 seconds to spawn the instance on the hypervisor.
Aug 09 02:24:29 user nova-compute[72351]: DEBUG nova.compute.manager [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:24:29 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:24:29 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:24:29 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/0825f13f-c923-417d-8fa0-f4e61616e4db/disk --force-share --output=json" returned: 0 in 0.173s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:24:29 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Cannot resize image /opt/stack/data/nova/instances/0825f13f-c923-417d-8fa0-f4e61616e4db/disk to a smaller size. {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Aug 09 02:24:29 user nova-compute[72351]: DEBUG nova.objects.instance [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Lazy-loading 'migration_context' on Instance uuid 0825f13f-c923-417d-8fa0-f4e61616e4db {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:24:29 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Created local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4851}}
Aug 09 02:24:29 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Ensure instance console log exists: /opt/stack/data/nova/instances/0825f13f-c923-417d-8fa0-f4e61616e4db/console.log {{(pid=72351) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4603}}
Aug 09 02:24:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:24:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:24:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:24:29 user nova-compute[72351]: INFO nova.compute.manager [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Took 9.55 seconds to build instance.
Aug 09 02:24:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-460ba91d-bff9-43aa-80e8-b7ff55f57dff tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "831c864a-75d9-42b3-b952-749f4201e40e" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 9.786s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:24:29 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-7f376306-dbe6-4463-b946-f29c638e2129 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] CMD "qemu-img convert -t none -O qcow2 -f qcow2 /opt/stack/data/nova/instances/snapshots/tmpr35ij9ur/d8f4aef62f03433c9e62609c73237521.delta /opt/stack/data/nova/instances/snapshots/tmpr35ij9ur/d8f4aef62f03433c9e62609c73237521" returned: 0 in 0.691s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:24:29 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-7f376306-dbe6-4463-b946-f29c638e2129 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Snapshot extracted, beginning image upload
Aug 09 02:24:29 user nova-compute[72351]: DEBUG nova.network.neutron [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Successfully created port: 10079897-954d-49a0-9007-51efb10241bc {{(pid=72351) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Aug 09 02:24:30 user nova-compute[72351]: DEBUG nova.network.neutron [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Successfully updated port: 10079897-954d-49a0-9007-51efb10241bc {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:24:30 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Acquiring lock "refresh_cache-0825f13f-c923-417d-8fa0-f4e61616e4db" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:24:30 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Acquired lock "refresh_cache-0825f13f-c923-417d-8fa0-f4e61616e4db" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:24:30 user nova-compute[72351]: DEBUG nova.network.neutron [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:24:30 user nova-compute[72351]: DEBUG nova.compute.manager [req-6a1de485-8dcd-4ba3-936b-965a74f21078 req-434c4d74-3c4b-44a0-b7fa-bf80054df9d1 service nova] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Received event network-changed-10079897-954d-49a0-9007-51efb10241bc {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:24:30 user nova-compute[72351]: DEBUG nova.compute.manager [req-6a1de485-8dcd-4ba3-936b-965a74f21078 req-434c4d74-3c4b-44a0-b7fa-bf80054df9d1 service nova] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Refreshing instance network info cache due to event network-changed-10079897-954d-49a0-9007-51efb10241bc. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:24:30 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-6a1de485-8dcd-4ba3-936b-965a74f21078 req-434c4d74-3c4b-44a0-b7fa-bf80054df9d1 service nova] Acquiring lock "refresh_cache-0825f13f-c923-417d-8fa0-f4e61616e4db" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:24:30 user nova-compute[72351]: DEBUG nova.network.neutron [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:24:31 user nova-compute[72351]: DEBUG nova.network.neutron [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Updating instance_info_cache with network_info: [{"id": "10079897-954d-49a0-9007-51efb10241bc", "address": "fa:16:3e:0d:57:48", "network": {"id": "f9d6cd8a-f9ec-46e3-a28a-22530e7ddda2", "bridge": "br-int", "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.142", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.1"}}], "meta": {"injected": false, "tenant_id": "f9af963f7c574a7ab2deedfcf953a666", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap10079897-95", "ovs_interfaceid": "10079897-954d-49a0-9007-51efb10241bc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:24:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Releasing lock "refresh_cache-0825f13f-c923-417d-8fa0-f4e61616e4db" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:24:31 user nova-compute[72351]: DEBUG nova.compute.manager [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Instance network_info: |[{"id": "10079897-954d-49a0-9007-51efb10241bc", "address": "fa:16:3e:0d:57:48", "network": {"id": "f9d6cd8a-f9ec-46e3-a28a-22530e7ddda2", "bridge": "br-int", "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.142", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.1"}}], "meta": {"injected": false, "tenant_id": "f9af963f7c574a7ab2deedfcf953a666", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap10079897-95", "ovs_interfaceid": "10079897-954d-49a0-9007-51efb10241bc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1971}}
Aug 09 02:24:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-6a1de485-8dcd-4ba3-936b-965a74f21078 req-434c4d74-3c4b-44a0-b7fa-bf80054df9d1 service nova] Acquired lock "refresh_cache-0825f13f-c923-417d-8fa0-f4e61616e4db" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:24:31 user nova-compute[72351]: DEBUG nova.network.neutron [req-6a1de485-8dcd-4ba3-936b-965a74f21078 req-434c4d74-3c4b-44a0-b7fa-bf80054df9d1 service nova] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Refreshing network info cache for port 10079897-954d-49a0-9007-51efb10241bc {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:24:31 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Start _get_guest_xml network_info=[{"id": "10079897-954d-49a0-9007-51efb10241bc", "address": "fa:16:3e:0d:57:48", "network": {"id": "f9d6cd8a-f9ec-46e3-a28a-22530e7ddda2", "bridge": "br-int", "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.142", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.1"}}], "meta": {"injected": false, "tenant_id": "f9af963f7c574a7ab2deedfcf953a666", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap10079897-95", "ovs_interfaceid": "10079897-954d-49a0-9007-51efb10241bc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'size': 0, 'device_type': 'disk', 'encryption_secret_uuid': None, 'guest_format': None, 'encryption_options': None, 'disk_bus': 'virtio', 'encrypted': False, 'encryption_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'image_id': '18ccda7f-31d3-4978-ab9d-4ed2f74eff94'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:24:31 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:24:31 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:24:31 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:24:31 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:24:31 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:24:31 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:24:31 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:24:31 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:24:31 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:24:31 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:24:31 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:24:31 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:24:31 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:24:31 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:24:31 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:24:31 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:24:31 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:24:31 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:24:31 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:24:26Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesNegativeTest-instance-120898461',display_name='tempest-VolumesNegativeTest-instance-120898461',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-volumesnegativetest-instance-120898461',id=24,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='1af75c014509460eab5821b6a5c1e124',ramdisk_id='',reservation_id='r-05i3waz6',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesNegativeTest-1226059554',owner_user_name='tempest-VolumesNegativeTest-1226059554-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:24:28Z,user_data=None,user_id='693fd098694d41dd91edb717fd8170ba',uuid=0825f13f-c923-417d-8fa0-f4e61616e4db,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "10079897-954d-49a0-9007-51efb10241bc", "address": "fa:16:3e:0d:57:48", "network": {"id": "f9d6cd8a-f9ec-46e3-a28a-22530e7ddda2", "bridge": "br-int", "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.142", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.1"}}], "meta": {"injected": false, "tenant_id": "f9af963f7c574a7ab2deedfcf953a666", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap10079897-95", "ovs_interfaceid": "10079897-954d-49a0-9007-51efb10241bc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:24:31 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Converting VIF {"id": "10079897-954d-49a0-9007-51efb10241bc", "address": "fa:16:3e:0d:57:48", "network": {"id": "f9d6cd8a-f9ec-46e3-a28a-22530e7ddda2", "bridge": "br-int", "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.142", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.1"}}], "meta": {"injected": false, "tenant_id": "f9af963f7c574a7ab2deedfcf953a666", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap10079897-95", "ovs_interfaceid": "10079897-954d-49a0-9007-51efb10241bc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:24:31 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:0d:57:48,bridge_name='br-int',has_traffic_filtering=True,id=10079897-954d-49a0-9007-51efb10241bc,network=Network(f9d6cd8a-f9ec-46e3-a28a-22530e7ddda2),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap10079897-95') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:24:31 user nova-compute[72351]: DEBUG nova.objects.instance [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Lazy-loading 'pci_devices' on Instance uuid 0825f13f-c923-417d-8fa0-f4e61616e4db {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:24:31 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:24:31 user nova-compute[72351]:   <uuid>0825f13f-c923-417d-8fa0-f4e61616e4db</uuid>
Aug 09 02:24:31 user nova-compute[72351]:   <name>instance-00000018</name>
Aug 09 02:24:31 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:24:31 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:24:31 user nova-compute[72351]:   <metadata>
Aug 09 02:24:31 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:24:31 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:24:31 user nova-compute[72351]:       <nova:name>tempest-VolumesNegativeTest-instance-120898461</nova:name>
Aug 09 02:24:31 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:24:31</nova:creationTime>
Aug 09 02:24:31 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:24:31 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:24:31 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:24:31 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:24:31 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:24:31 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:24:31 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:24:31 user nova-compute[72351]:       <nova:owner>
Aug 09 02:24:31 user nova-compute[72351]:         <nova:user uuid="693fd098694d41dd91edb717fd8170ba">tempest-VolumesNegativeTest-1226059554-project-member</nova:user>
Aug 09 02:24:31 user nova-compute[72351]:         <nova:project uuid="1af75c014509460eab5821b6a5c1e124">tempest-VolumesNegativeTest-1226059554</nova:project>
Aug 09 02:24:31 user nova-compute[72351]:       </nova:owner>
Aug 09 02:24:31 user nova-compute[72351]:       <nova:root type="image" uuid="18ccda7f-31d3-4978-ab9d-4ed2f74eff94"/>
Aug 09 02:24:31 user nova-compute[72351]:       <nova:ports>
Aug 09 02:24:31 user nova-compute[72351]:         <nova:port uuid="10079897-954d-49a0-9007-51efb10241bc">
Aug 09 02:24:31 user nova-compute[72351]:           <nova:ip type="fixed" address="192.168.233.142" ipVersion="4"/>
Aug 09 02:24:31 user nova-compute[72351]:         </nova:port>
Aug 09 02:24:31 user nova-compute[72351]:       </nova:ports>
Aug 09 02:24:31 user nova-compute[72351]:     </nova:instance>
Aug 09 02:24:31 user nova-compute[72351]:   </metadata>
Aug 09 02:24:31 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:24:31 user nova-compute[72351]:     <system>
Aug 09 02:24:31 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:24:31 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:24:31 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:24:31 user nova-compute[72351]:       <entry name="serial">0825f13f-c923-417d-8fa0-f4e61616e4db</entry>
Aug 09 02:24:31 user nova-compute[72351]:       <entry name="uuid">0825f13f-c923-417d-8fa0-f4e61616e4db</entry>
Aug 09 02:24:31 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:24:31 user nova-compute[72351]:     </system>
Aug 09 02:24:31 user nova-compute[72351]:   </sysinfo>
Aug 09 02:24:31 user nova-compute[72351]:   <os>
Aug 09 02:24:31 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:24:31 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:24:31 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:24:31 user nova-compute[72351]:   </os>
Aug 09 02:24:31 user nova-compute[72351]:   <features>
Aug 09 02:24:31 user nova-compute[72351]:     <acpi/>
Aug 09 02:24:31 user nova-compute[72351]:     <apic/>
Aug 09 02:24:31 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:24:31 user nova-compute[72351]:   </features>
Aug 09 02:24:31 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:24:31 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:24:31 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:24:31 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:24:31 user nova-compute[72351]:   </clock>
Aug 09 02:24:31 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:24:31 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:24:31 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:24:31 user nova-compute[72351]:   </cpu>
Aug 09 02:24:31 user nova-compute[72351]:   <devices>
Aug 09 02:24:31 user nova-compute[72351]:     <disk type="file" device="disk">
Aug 09 02:24:31 user nova-compute[72351]:       <driver name="qemu" type="qcow2" cache="none"/>
Aug 09 02:24:31 user nova-compute[72351]:       <source file="/opt/stack/data/nova/instances/0825f13f-c923-417d-8fa0-f4e61616e4db/disk"/>
Aug 09 02:24:31 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:24:31 user nova-compute[72351]:     </disk>
Aug 09 02:24:31 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:24:31 user nova-compute[72351]:       <mac address="fa:16:3e:0d:57:48"/>
Aug 09 02:24:31 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:24:31 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:24:31 user nova-compute[72351]:       <target dev="tap10079897-95"/>
Aug 09 02:24:31 user nova-compute[72351]:     </interface>
Aug 09 02:24:31 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:24:31 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/0825f13f-c923-417d-8fa0-f4e61616e4db/console.log" append="off"/>
Aug 09 02:24:31 user nova-compute[72351]:     </serial>
Aug 09 02:24:31 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:24:31 user nova-compute[72351]:     <video>
Aug 09 02:24:31 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:24:31 user nova-compute[72351]:     </video>
Aug 09 02:24:31 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:24:31 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:24:31 user nova-compute[72351]:     </rng>
Aug 09 02:24:31 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:24:31 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:24:31 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:24:31 user nova-compute[72351]:     </memballoon>
Aug 09 02:24:31 user nova-compute[72351]:   </devices>
Aug 09 02:24:31 user nova-compute[72351]: </domain>
Aug 09 02:24:31 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:24:31 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:24:26Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesNegativeTest-instance-120898461',display_name='tempest-VolumesNegativeTest-instance-120898461',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-volumesnegativetest-instance-120898461',id=24,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='1af75c014509460eab5821b6a5c1e124',ramdisk_id='',reservation_id='r-05i3waz6',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesNegativeTest-1226059554',owner_user_name='tempest-VolumesNegativeTest-1226059554-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:24:28Z,user_data=None,user_id='693fd098694d41dd91edb717fd8170ba',uuid=0825f13f-c923-417d-8fa0-f4e61616e4db,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "10079897-954d-49a0-9007-51efb10241bc", "address": "fa:16:3e:0d:57:48", "network": {"id": "f9d6cd8a-f9ec-46e3-a28a-22530e7ddda2", "bridge": "br-int", "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.142", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.1"}}], "meta": {"injected": false, "tenant_id": "f9af963f7c574a7ab2deedfcf953a666", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap10079897-95", "ovs_interfaceid": "10079897-954d-49a0-9007-51efb10241bc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:24:31 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Converting VIF {"id": "10079897-954d-49a0-9007-51efb10241bc", "address": "fa:16:3e:0d:57:48", "network": {"id": "f9d6cd8a-f9ec-46e3-a28a-22530e7ddda2", "bridge": "br-int", "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.142", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.1"}}], "meta": {"injected": false, "tenant_id": "f9af963f7c574a7ab2deedfcf953a666", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap10079897-95", "ovs_interfaceid": "10079897-954d-49a0-9007-51efb10241bc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:24:31 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:0d:57:48,bridge_name='br-int',has_traffic_filtering=True,id=10079897-954d-49a0-9007-51efb10241bc,network=Network(f9d6cd8a-f9ec-46e3-a28a-22530e7ddda2),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap10079897-95') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:24:31 user nova-compute[72351]: DEBUG os_vif [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:0d:57:48,bridge_name='br-int',has_traffic_filtering=True,id=10079897-954d-49a0-9007-51efb10241bc,network=Network(f9d6cd8a-f9ec-46e3-a28a-22530e7ddda2),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap10079897-95') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:24:31 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:31 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:24:31 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:24:31 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:31 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap10079897-95, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:24:31 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap10079897-95, col_values=(('external_ids', {'iface-id': '10079897-954d-49a0-9007-51efb10241bc', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:0d:57:48', 'vm-uuid': '0825f13f-c923-417d-8fa0-f4e61616e4db'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:24:31 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:31 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:24:31 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:31 user nova-compute[72351]: INFO os_vif [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:0d:57:48,bridge_name='br-int',has_traffic_filtering=True,id=10079897-954d-49a0-9007-51efb10241bc,network=Network(f9d6cd8a-f9ec-46e3-a28a-22530e7ddda2),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap10079897-95')
Aug 09 02:24:31 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:24:31 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] No VIF found with MAC fa:16:3e:0d:57:48, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:24:32 user nova-compute[72351]: DEBUG nova.network.neutron [req-6a1de485-8dcd-4ba3-936b-965a74f21078 req-434c4d74-3c4b-44a0-b7fa-bf80054df9d1 service nova] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Updated VIF entry in instance network info cache for port 10079897-954d-49a0-9007-51efb10241bc. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:24:32 user nova-compute[72351]: DEBUG nova.network.neutron [req-6a1de485-8dcd-4ba3-936b-965a74f21078 req-434c4d74-3c4b-44a0-b7fa-bf80054df9d1 service nova] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Updating instance_info_cache with network_info: [{"id": "10079897-954d-49a0-9007-51efb10241bc", "address": "fa:16:3e:0d:57:48", "network": {"id": "f9d6cd8a-f9ec-46e3-a28a-22530e7ddda2", "bridge": "br-int", "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.142", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.1"}}], "meta": {"injected": false, "tenant_id": "f9af963f7c574a7ab2deedfcf953a666", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap10079897-95", "ovs_interfaceid": "10079897-954d-49a0-9007-51efb10241bc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:24:32 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-6a1de485-8dcd-4ba3-936b-965a74f21078 req-434c4d74-3c4b-44a0-b7fa-bf80054df9d1 service nova] Releasing lock "refresh_cache-0825f13f-c923-417d-8fa0-f4e61616e4db" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:24:32 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-7f376306-dbe6-4463-b946-f29c638e2129 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Snapshot image upload complete
Aug 09 02:24:32 user nova-compute[72351]: INFO nova.compute.manager [None req-7f376306-dbe6-4463-b946-f29c638e2129 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Took 7.19 seconds to snapshot the instance on the hypervisor.
Aug 09 02:24:32 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:33 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:33 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:33 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:33 user nova-compute[72351]: DEBUG nova.compute.manager [req-74e4493a-c800-4dfb-8b0e-76ae73ec560e req-e01143b8-01f9-4a25-945b-07715348ffe8 service nova] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Received event network-vif-plugged-10079897-954d-49a0-9007-51efb10241bc {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:24:33 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-74e4493a-c800-4dfb-8b0e-76ae73ec560e req-e01143b8-01f9-4a25-945b-07715348ffe8 service nova] Acquiring lock "0825f13f-c923-417d-8fa0-f4e61616e4db-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:24:33 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-74e4493a-c800-4dfb-8b0e-76ae73ec560e req-e01143b8-01f9-4a25-945b-07715348ffe8 service nova] Lock "0825f13f-c923-417d-8fa0-f4e61616e4db-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:24:33 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-74e4493a-c800-4dfb-8b0e-76ae73ec560e req-e01143b8-01f9-4a25-945b-07715348ffe8 service nova] Lock "0825f13f-c923-417d-8fa0-f4e61616e4db-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:24:33 user nova-compute[72351]: DEBUG nova.compute.manager [req-74e4493a-c800-4dfb-8b0e-76ae73ec560e req-e01143b8-01f9-4a25-945b-07715348ffe8 service nova] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] No waiting events found dispatching network-vif-plugged-10079897-954d-49a0-9007-51efb10241bc {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:24:33 user nova-compute[72351]: WARNING nova.compute.manager [req-74e4493a-c800-4dfb-8b0e-76ae73ec560e req-e01143b8-01f9-4a25-945b-07715348ffe8 service nova] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Received unexpected event network-vif-plugged-10079897-954d-49a0-9007-51efb10241bc for instance with vm_state building and task_state spawning.
Aug 09 02:24:33 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:33 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:33 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:34 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:34 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691547859.7964103, 2d78475c-107d-4256-b12e-dbe9c8f3b9b2 => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:24:34 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] VM Stopped (Lifecycle Event)
Aug 09 02:24:34 user nova-compute[72351]: DEBUG nova.compute.manager [None req-35f5c906-f878-4434-ba85-28d8878c57a9 None None] [instance: 2d78475c-107d-4256-b12e-dbe9c8f3b9b2] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:24:35 user nova-compute[72351]: DEBUG nova.compute.manager [req-7cf4d2d9-eb61-4257-a7b6-c3fcc7e16369 req-514beac0-0c06-4585-b12d-1350b6b16cdb service nova] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Received event network-vif-plugged-10079897-954d-49a0-9007-51efb10241bc {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:24:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-7cf4d2d9-eb61-4257-a7b6-c3fcc7e16369 req-514beac0-0c06-4585-b12d-1350b6b16cdb service nova] Acquiring lock "0825f13f-c923-417d-8fa0-f4e61616e4db-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:24:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-7cf4d2d9-eb61-4257-a7b6-c3fcc7e16369 req-514beac0-0c06-4585-b12d-1350b6b16cdb service nova] Lock "0825f13f-c923-417d-8fa0-f4e61616e4db-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:24:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-7cf4d2d9-eb61-4257-a7b6-c3fcc7e16369 req-514beac0-0c06-4585-b12d-1350b6b16cdb service nova] Lock "0825f13f-c923-417d-8fa0-f4e61616e4db-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:24:35 user nova-compute[72351]: DEBUG nova.compute.manager [req-7cf4d2d9-eb61-4257-a7b6-c3fcc7e16369 req-514beac0-0c06-4585-b12d-1350b6b16cdb service nova] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] No waiting events found dispatching network-vif-plugged-10079897-954d-49a0-9007-51efb10241bc {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:24:35 user nova-compute[72351]: WARNING nova.compute.manager [req-7cf4d2d9-eb61-4257-a7b6-c3fcc7e16369 req-514beac0-0c06-4585-b12d-1350b6b16cdb service nova] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Received unexpected event network-vif-plugged-10079897-954d-49a0-9007-51efb10241bc for instance with vm_state building and task_state spawning.
Aug 09 02:24:35 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547875.6780362, 0825f13f-c923-417d-8fa0-f4e61616e4db => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:24:35 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] VM Resumed (Lifecycle Event)
Aug 09 02:24:35 user nova-compute[72351]: DEBUG nova.compute.manager [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:24:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Guest created on hypervisor {{(pid=72351) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4411}}
Aug 09 02:24:35 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Instance spawned successfully.
Aug 09 02:24:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:894}}
Aug 09 02:24:35 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:24:35 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:24:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Found default for hw_cdrom_bus of ide {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:24:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Found default for hw_disk_bus of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:24:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Found default for hw_input_bus of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:24:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Found default for hw_pointer_model of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:24:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Found default for hw_video_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:24:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Found default for hw_vif_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:24:35 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:24:35 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547875.6799803, 0825f13f-c923-417d-8fa0-f4e61616e4db => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:24:35 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] VM Started (Lifecycle Event)
Aug 09 02:24:35 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:24:35 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:24:35 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:24:35 user nova-compute[72351]: INFO nova.compute.manager [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Took 7.51 seconds to spawn the instance on the hypervisor.
Aug 09 02:24:35 user nova-compute[72351]: DEBUG nova.compute.manager [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:24:36 user nova-compute[72351]: INFO nova.compute.manager [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Took 8.99 seconds to build instance.
Aug 09 02:24:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-76ed0e94-b66b-4e9c-9782-0a16430add37 tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Lock "0825f13f-c923-417d-8fa0-f4e61616e4db" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 9.138s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:24:36 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:36 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:37 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:41 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:42 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:46 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:47 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:47 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:51 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:52 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:55 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:56 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:56 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b1b1cc98-da64-4caa-a586-7b197a0c1ee6 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Acquiring lock "75bbbead-8628-448b-9484-19bf4ab922ff" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:24:56 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b1b1cc98-da64-4caa-a586-7b197a0c1ee6 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Lock "75bbbead-8628-448b-9484-19bf4ab922ff" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:24:56 user nova-compute[72351]: DEBUG nova.objects.instance [None req-b1b1cc98-da64-4caa-a586-7b197a0c1ee6 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Lazy-loading 'flavor' on Instance uuid 75bbbead-8628-448b-9484-19bf4ab922ff {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:24:56 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:24:56 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:24:56 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72351) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10461}}
Aug 09 02:24:56 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:24:56 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b1b1cc98-da64-4caa-a586-7b197a0c1ee6 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Lock "75bbbead-8628-448b-9484-19bf4ab922ff" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.054s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:24:56 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:24:56 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:24:56 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:24:56 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Auditing locally available compute resources for user (node: user) {{(pid=72351) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Aug 09 02:24:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b1b1cc98-da64-4caa-a586-7b197a0c1ee6 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Acquiring lock "75bbbead-8628-448b-9484-19bf4ab922ff" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:24:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b1b1cc98-da64-4caa-a586-7b197a0c1ee6 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Lock "75bbbead-8628-448b-9484-19bf4ab922ff" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:24:57 user nova-compute[72351]: INFO nova.compute.manager [None req-b1b1cc98-da64-4caa-a586-7b197a0c1ee6 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Attaching volume 2db51b4f-d032-4367-bb6f-8964563eb8fd to /dev/vdb
Aug 09 02:24:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:57 user nova-compute[72351]: DEBUG os_brick.utils [None req-b1b1cc98-da64-4caa-a586-7b197a0c1ee6 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:24:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.013s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:24:57 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[17ccbbf3-fee1-4008-a387-7a9dcefbcfe7]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:a52269602ff4\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:24:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:24:57 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[60b7bb78-7a8f-4689-90c8-4bed7af14af0]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:24:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.015s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:24:57 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[9e97d2c9-e016-42d6-b343-e95144f18f89]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:24:57 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ca102b01-58c2-4953-b112-0f6ef6de0a0f]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:24:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-b1b1cc98-da64-4caa-a586-7b197a0c1ee6 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Running cmd (subprocess): nvme version {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-b1b1cc98-da64-4caa-a586-7b197a0c1ee6 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] 'nvme version' failed. Not Retrying. {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:24:57 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.nvmeof [None req-b1b1cc98-da64-4caa-a586-7b197a0c1ee6 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] nvme not present on system {{(pid=72351) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Aug 09 02:24:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json" returned: 0 in 0.175s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:24:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:24:57 user nova-compute[72351]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Aug 09 02:24:57 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a213078f-f33f-43ba-9cf8-46dd5b22b0a6]: (4, '') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:24:57 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-b1b1cc98-da64-4caa-a586-7b197a0c1ee6 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72351) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Aug 09 02:24:57 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-b1b1cc98-da64-4caa-a586-7b197a0c1ee6 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Aug 09 02:24:57 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-b1b1cc98-da64-4caa-a586-7b197a0c1ee6 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] LIGHTOS: no hostnqn found. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Aug 09 02:24:57 user nova-compute[72351]: DEBUG os_brick.utils [None req-b1b1cc98-da64-4caa-a586-7b197a0c1ee6 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] <== get_connector_properties: return (160ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:a52269602ff4', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:24:57 user nova-compute[72351]: DEBUG nova.virt.block_device [None req-b1b1cc98-da64-4caa-a586-7b197a0c1ee6 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Updating existing volume attachment record: 929c7cfc-6708-4e28-8f3b-78e3195f4dbf {{(pid=72351) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Aug 09 02:24:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json" returned: 0 in 0.139s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:24:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/0825f13f-c923-417d-8fa0-f4e61616e4db/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/0825f13f-c923-417d-8fa0-f4e61616e4db/disk --force-share --output=json" returned: 0 in 0.159s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:24:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/0825f13f-c923-417d-8fa0-f4e61616e4db/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/0825f13f-c923-417d-8fa0-f4e61616e4db/disk --force-share --output=json" returned: 0 in 0.146s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:24:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.rescue --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:57 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:24:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.rescue --force-share --output=json" returned: 0 in 0.141s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:24:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.rescue --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.rescue --force-share --output=json" returned: 0 in 0.158s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:24:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json" returned: 0 in 0.149s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:24:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json" returned: 0 in 0.190s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:24:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json" returned: 0 in 0.133s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:24:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json" returned: 0 in 0.149s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:24:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/433e6729-fdd4-4467-971c-08ca962d887d/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Acquiring lock "f457e6ea-8d06-46c6-be2e-f650e28a3b69" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:24:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Lock "f457e6ea-8d06-46c6-be2e-f650e28a3b69" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:24:58 user nova-compute[72351]: DEBUG nova.compute.manager [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Starting instance... {{(pid=72351) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2406}}
Aug 09 02:24:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/433e6729-fdd4-4467-971c-08ca962d887d/disk --force-share --output=json" returned: 0 in 0.146s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:24:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/433e6729-fdd4-4467-971c-08ca962d887d/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:24:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:24:58 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72351) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Aug 09 02:24:58 user nova-compute[72351]: INFO nova.compute.claims [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Claim successful on node user
Aug 09 02:24:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/433e6729-fdd4-4467-971c-08ca962d887d/disk --force-share --output=json" returned: 0 in 0.167s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:24:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json" returned: 0 in 0.162s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:24:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json" returned: 0 in 0.153s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:24:59 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdd (vda) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:24:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:59 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:24:59 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:24:59 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.550s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:24:59 user nova-compute[72351]: DEBUG nova.compute.manager [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Start building networks asynchronously for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2803}}
Aug 09 02:24:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff/disk --force-share --output=json" returned: 0 in 0.140s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:24:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:59 user nova-compute[72351]: DEBUG nova.compute.manager [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Allocating IP information in the background. {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1956}}
Aug 09 02:24:59 user nova-compute[72351]: DEBUG nova.network.neutron [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] allocate_for_instance() {{(pid=72351) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Aug 09 02:24:59 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Aug 09 02:24:59 user nova-compute[72351]: DEBUG nova.compute.manager [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Start building block device mappings for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2838}}
Aug 09 02:24:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff/disk --force-share --output=json" returned: 0 in 0.147s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:24:59 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:24:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/831c864a-75d9-42b3-b952-749f4201e40e/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:59 user nova-compute[72351]: DEBUG nova.policy [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '8159c9e78af04e01a1faf9b3cfc3591e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bde0634670914ae8bfbbc6b31c09d6dd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72351) authorize /opt/stack/nova/nova/policy.py:203}}
Aug 09 02:24:59 user nova-compute[72351]: DEBUG nova.compute.manager [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Start spawning the instance on the hypervisor. {{(pid=72351) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2612}}
Aug 09 02:24:59 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Creating instance directory {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4717}}
Aug 09 02:24:59 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Creating image(s)
Aug 09 02:24:59 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Acquiring lock "/opt/stack/data/nova/instances/f457e6ea-8d06-46c6-be2e-f650e28a3b69/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:24:59 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Lock "/opt/stack/data/nova/instances/f457e6ea-8d06-46c6-be2e-f650e28a3b69/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:24:59 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Lock "/opt/stack/data/nova/instances/f457e6ea-8d06-46c6-be2e-f650e28a3b69/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:24:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/831c864a-75d9-42b3-b952-749f4201e40e/disk --force-share --output=json" returned: 0 in 0.158s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:24:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/831c864a-75d9-42b3-b952-749f4201e40e/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:24:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.198s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:24:59 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Acquiring lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:24:59 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:24:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/831c864a-75d9-42b3-b952-749f4201e40e/disk --force-share --output=json" returned: 0 in 0.145s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:00 user nova-compute[72351]: DEBUG nova.network.neutron [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Successfully created port: dc244cf9-b0f6-4906-b9f3-53634b1847e3 {{(pid=72351) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Aug 09 02:25:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.536s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/f457e6ea-8d06-46c6-be2e-f650e28a3b69/disk 1073741824 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:00 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:25:00 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:25:00 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Hypervisor/Node resource view: name=user free_ram=7129MB free_disk=25.424327850341797GB free_vcpus=2 pci_devices=[{"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72351) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Aug 09 02:25:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:25:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:25:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/f457e6ea-8d06-46c6-be2e-f650e28a3b69/disk 1073741824" returned: 0 in 0.057s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.597s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:25:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:00 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 75bbbead-8628-448b-9484-19bf4ab922ff actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:25:00 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:25:00 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance eb86cb49-2018-42c2-bd97-c662b829775d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:25:00 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 1d772c10-1c8a-4101-9a61-f8a60bb19c66 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:25:00 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:25:00 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance a7f3d076-864b-4b68-9207-214ce2c21444 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:25:00 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 5c71284c-df41-4990-9a46-7986982bf7e2 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:25:00 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 433e6729-fdd4-4467-971c-08ca962d887d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:25:00 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 831c864a-75d9-42b3-b952-749f4201e40e actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:25:00 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 0825f13f-c923-417d-8fa0-f4e61616e4db actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:25:00 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance f457e6ea-8d06-46c6-be2e-f650e28a3b69 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:25:00 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Total usable vcpus: 12, total allocated vcpus: 11 {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Aug 09 02:25:00 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Final resource view: name=user phys_ram=16011MB used_ram=6144MB phys_disk=40GB used_disk=9GB total_vcpus=12 used_vcpus=11 pci_stats=[] {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Aug 09 02:25:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.160s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:00 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Checking if we can resize image /opt/stack/data/nova/instances/f457e6ea-8d06-46c6-be2e-f650e28a3b69/disk. size=1073741824 {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Aug 09 02:25:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f457e6ea-8d06-46c6-be2e-f650e28a3b69/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f457e6ea-8d06-46c6-be2e-f650e28a3b69/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:00 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Cannot resize image /opt/stack/data/nova/instances/f457e6ea-8d06-46c6-be2e-f650e28a3b69/disk to a smaller size. {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Aug 09 02:25:00 user nova-compute[72351]: DEBUG nova.objects.instance [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Lazy-loading 'migration_context' on Instance uuid f457e6ea-8d06-46c6-be2e-f650e28a3b69 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:25:00 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Created local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4851}}
Aug 09 02:25:00 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Ensure instance console log exists: /opt/stack/data/nova/instances/f457e6ea-8d06-46c6-be2e-f650e28a3b69/console.log {{(pid=72351) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4603}}
Aug 09 02:25:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:25:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:25:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:25:01 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:25:01 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:25:01 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Compute_service record updated for user:user {{(pid=72351) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Aug 09 02:25:01 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.553s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:25:01 user nova-compute[72351]: DEBUG nova.network.neutron [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Successfully updated port: dc244cf9-b0f6-4906-b9f3-53634b1847e3 {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:25:01 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Acquiring lock "refresh_cache-f457e6ea-8d06-46c6-be2e-f650e28a3b69" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:25:01 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Acquired lock "refresh_cache-f457e6ea-8d06-46c6-be2e-f650e28a3b69" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:25:01 user nova-compute[72351]: DEBUG nova.network.neutron [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:25:01 user nova-compute[72351]: DEBUG nova.compute.manager [req-8b33cd6b-333f-4203-96f4-531dee963fc1 req-40c63799-24e2-4bbe-b7b8-bcf65b33c722 service nova] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Received event network-changed-dc244cf9-b0f6-4906-b9f3-53634b1847e3 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:25:01 user nova-compute[72351]: DEBUG nova.compute.manager [req-8b33cd6b-333f-4203-96f4-531dee963fc1 req-40c63799-24e2-4bbe-b7b8-bcf65b33c722 service nova] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Refreshing instance network info cache due to event network-changed-dc244cf9-b0f6-4906-b9f3-53634b1847e3. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:25:01 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-8b33cd6b-333f-4203-96f4-531dee963fc1 req-40c63799-24e2-4bbe-b7b8-bcf65b33c722 service nova] Acquiring lock "refresh_cache-f457e6ea-8d06-46c6-be2e-f650e28a3b69" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:25:01 user nova-compute[72351]: DEBUG nova.network.neutron [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:25:01 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:01 user nova-compute[72351]: DEBUG nova.network.neutron [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Updating instance_info_cache with network_info: [{"id": "dc244cf9-b0f6-4906-b9f3-53634b1847e3", "address": "fa:16:3e:0e:77:88", "network": {"id": "3fdd9499-7552-4ecc-a897-ed4c58be9519", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-969934765-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bde0634670914ae8bfbbc6b31c09d6dd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdc244cf9-b0", "ovs_interfaceid": "dc244cf9-b0f6-4906-b9f3-53634b1847e3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:25:01 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Releasing lock "refresh_cache-f457e6ea-8d06-46c6-be2e-f650e28a3b69" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:25:01 user nova-compute[72351]: DEBUG nova.compute.manager [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Instance network_info: |[{"id": "dc244cf9-b0f6-4906-b9f3-53634b1847e3", "address": "fa:16:3e:0e:77:88", "network": {"id": "3fdd9499-7552-4ecc-a897-ed4c58be9519", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-969934765-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bde0634670914ae8bfbbc6b31c09d6dd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdc244cf9-b0", "ovs_interfaceid": "dc244cf9-b0f6-4906-b9f3-53634b1847e3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1971}}
Aug 09 02:25:01 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-8b33cd6b-333f-4203-96f4-531dee963fc1 req-40c63799-24e2-4bbe-b7b8-bcf65b33c722 service nova] Acquired lock "refresh_cache-f457e6ea-8d06-46c6-be2e-f650e28a3b69" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:25:01 user nova-compute[72351]: DEBUG nova.network.neutron [req-8b33cd6b-333f-4203-96f4-531dee963fc1 req-40c63799-24e2-4bbe-b7b8-bcf65b33c722 service nova] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Refreshing network info cache for port dc244cf9-b0f6-4906-b9f3-53634b1847e3 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:25:01 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Start _get_guest_xml network_info=[{"id": "dc244cf9-b0f6-4906-b9f3-53634b1847e3", "address": "fa:16:3e:0e:77:88", "network": {"id": "3fdd9499-7552-4ecc-a897-ed4c58be9519", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-969934765-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bde0634670914ae8bfbbc6b31c09d6dd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdc244cf9-b0", "ovs_interfaceid": "dc244cf9-b0f6-4906-b9f3-53634b1847e3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk.config': {'bus': 'ide', 'dev': 'hda', 'type': 'cdrom'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'size': 0, 'device_type': 'disk', 'encryption_secret_uuid': None, 'guest_format': None, 'encryption_options': None, 'disk_bus': 'virtio', 'encrypted': False, 'encryption_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'image_id': '18ccda7f-31d3-4978-ab9d-4ed2f74eff94'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:25:01 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:25:01 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:25:01 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:25:01 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:25:01 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:25:01 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:25:01 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:25:01 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:25:01 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:25:01 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:25:01 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:25:01 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:25:01 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:25:01 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:25:01 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:25:01 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:25:01 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:25:01 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:25:01 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-08-09T02:24:58Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-443615500',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-443615500',id=25,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBK8nrHk7zghGHoCUEI3vExNw2oXJc2YwDe7QrMxgC2DNUP/PkVzpLwy9DWnyTLCCR4ddGxe1t7xIbXts5zbiafpc3PmU6by4kyzLopbWZ1BYcJyQcmguVSz4dz8HMbCR9g==',key_name='tempest-keypair-1705746290',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='bde0634670914ae8bfbbc6b31c09d6dd',ramdisk_id='',reservation_id='r-5xxup5ze',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedAttachmentsTest-480515778',owner_user_name='tempest-TaggedAttachmentsTest-480515778-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:25:00Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='8159c9e78af04e01a1faf9b3cfc3591e',uuid=f457e6ea-8d06-46c6-be2e-f650e28a3b69,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "dc244cf9-b0f6-4906-b9f3-53634b1847e3", "address": "fa:16:3e:0e:77:88", "network": {"id": "3fdd9499-7552-4ecc-a897-ed4c58be9519", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-969934765-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bde0634670914ae8bfbbc6b31c09d6dd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdc244cf9-b0", "ovs_interfaceid": "dc244cf9-b0f6-4906-b9f3-53634b1847e3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:25:01 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Converting VIF {"id": "dc244cf9-b0f6-4906-b9f3-53634b1847e3", "address": "fa:16:3e:0e:77:88", "network": {"id": "3fdd9499-7552-4ecc-a897-ed4c58be9519", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-969934765-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bde0634670914ae8bfbbc6b31c09d6dd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdc244cf9-b0", "ovs_interfaceid": "dc244cf9-b0f6-4906-b9f3-53634b1847e3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:25:01 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:0e:77:88,bridge_name='br-int',has_traffic_filtering=True,id=dc244cf9-b0f6-4906-b9f3-53634b1847e3,network=Network(3fdd9499-7552-4ecc-a897-ed4c58be9519),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdc244cf9-b0') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:25:01 user nova-compute[72351]: DEBUG nova.objects.instance [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Lazy-loading 'pci_devices' on Instance uuid f457e6ea-8d06-46c6-be2e-f650e28a3b69 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:25:01 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:25:01 user nova-compute[72351]:   <uuid>f457e6ea-8d06-46c6-be2e-f650e28a3b69</uuid>
Aug 09 02:25:01 user nova-compute[72351]:   <name>instance-00000019</name>
Aug 09 02:25:01 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:25:01 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:25:01 user nova-compute[72351]:   <metadata>
Aug 09 02:25:01 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:25:01 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:25:01 user nova-compute[72351]:       <nova:name>tempest-device-tagging-server-443615500</nova:name>
Aug 09 02:25:01 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:25:01</nova:creationTime>
Aug 09 02:25:01 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:25:01 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:25:01 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:25:01 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:25:01 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:25:01 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:25:01 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:25:01 user nova-compute[72351]:       <nova:owner>
Aug 09 02:25:01 user nova-compute[72351]:         <nova:user uuid="8159c9e78af04e01a1faf9b3cfc3591e">tempest-TaggedAttachmentsTest-480515778-project-member</nova:user>
Aug 09 02:25:01 user nova-compute[72351]:         <nova:project uuid="bde0634670914ae8bfbbc6b31c09d6dd">tempest-TaggedAttachmentsTest-480515778</nova:project>
Aug 09 02:25:01 user nova-compute[72351]:       </nova:owner>
Aug 09 02:25:01 user nova-compute[72351]:       <nova:root type="image" uuid="18ccda7f-31d3-4978-ab9d-4ed2f74eff94"/>
Aug 09 02:25:01 user nova-compute[72351]:       <nova:ports>
Aug 09 02:25:01 user nova-compute[72351]:         <nova:port uuid="dc244cf9-b0f6-4906-b9f3-53634b1847e3">
Aug 09 02:25:01 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.9" ipVersion="4"/>
Aug 09 02:25:01 user nova-compute[72351]:         </nova:port>
Aug 09 02:25:01 user nova-compute[72351]:       </nova:ports>
Aug 09 02:25:01 user nova-compute[72351]:     </nova:instance>
Aug 09 02:25:01 user nova-compute[72351]:   </metadata>
Aug 09 02:25:01 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:25:01 user nova-compute[72351]:     <system>
Aug 09 02:25:01 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:25:01 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:25:01 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:25:01 user nova-compute[72351]:       <entry name="serial">f457e6ea-8d06-46c6-be2e-f650e28a3b69</entry>
Aug 09 02:25:01 user nova-compute[72351]:       <entry name="uuid">f457e6ea-8d06-46c6-be2e-f650e28a3b69</entry>
Aug 09 02:25:01 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:25:01 user nova-compute[72351]:     </system>
Aug 09 02:25:01 user nova-compute[72351]:   </sysinfo>
Aug 09 02:25:01 user nova-compute[72351]:   <os>
Aug 09 02:25:01 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:25:01 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:25:01 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:25:01 user nova-compute[72351]:   </os>
Aug 09 02:25:01 user nova-compute[72351]:   <features>
Aug 09 02:25:01 user nova-compute[72351]:     <acpi/>
Aug 09 02:25:01 user nova-compute[72351]:     <apic/>
Aug 09 02:25:01 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:25:01 user nova-compute[72351]:   </features>
Aug 09 02:25:01 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:25:01 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:25:01 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:25:01 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:25:01 user nova-compute[72351]:   </clock>
Aug 09 02:25:01 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:25:01 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:25:01 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:25:01 user nova-compute[72351]:   </cpu>
Aug 09 02:25:01 user nova-compute[72351]:   <devices>
Aug 09 02:25:01 user nova-compute[72351]:     <disk type="file" device="disk">
Aug 09 02:25:01 user nova-compute[72351]:       <driver name="qemu" type="qcow2" cache="none"/>
Aug 09 02:25:01 user nova-compute[72351]:       <source file="/opt/stack/data/nova/instances/f457e6ea-8d06-46c6-be2e-f650e28a3b69/disk"/>
Aug 09 02:25:01 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:25:01 user nova-compute[72351]:     </disk>
Aug 09 02:25:01 user nova-compute[72351]:     <disk type="file" device="cdrom">
Aug 09 02:25:01 user nova-compute[72351]:       <driver name="qemu" type="raw" cache="none"/>
Aug 09 02:25:01 user nova-compute[72351]:       <source file="/opt/stack/data/nova/instances/f457e6ea-8d06-46c6-be2e-f650e28a3b69/disk.config"/>
Aug 09 02:25:01 user nova-compute[72351]:       <target dev="hda" bus="ide"/>
Aug 09 02:25:01 user nova-compute[72351]:     </disk>
Aug 09 02:25:01 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:25:01 user nova-compute[72351]:       <mac address="fa:16:3e:0e:77:88"/>
Aug 09 02:25:01 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:25:01 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:25:01 user nova-compute[72351]:       <target dev="tapdc244cf9-b0"/>
Aug 09 02:25:01 user nova-compute[72351]:     </interface>
Aug 09 02:25:01 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:25:01 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/f457e6ea-8d06-46c6-be2e-f650e28a3b69/console.log" append="off"/>
Aug 09 02:25:01 user nova-compute[72351]:     </serial>
Aug 09 02:25:01 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:25:01 user nova-compute[72351]:     <video>
Aug 09 02:25:01 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:25:01 user nova-compute[72351]:     </video>
Aug 09 02:25:01 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:25:01 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:25:01 user nova-compute[72351]:     </rng>
Aug 09 02:25:01 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:25:01 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:25:01 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:25:01 user nova-compute[72351]:     </memballoon>
Aug 09 02:25:01 user nova-compute[72351]:   </devices>
Aug 09 02:25:01 user nova-compute[72351]: </domain>
Aug 09 02:25:01 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:25:01 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-08-09T02:24:58Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-443615500',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-443615500',id=25,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBK8nrHk7zghGHoCUEI3vExNw2oXJc2YwDe7QrMxgC2DNUP/PkVzpLwy9DWnyTLCCR4ddGxe1t7xIbXts5zbiafpc3PmU6by4kyzLopbWZ1BYcJyQcmguVSz4dz8HMbCR9g==',key_name='tempest-keypair-1705746290',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='bde0634670914ae8bfbbc6b31c09d6dd',ramdisk_id='',reservation_id='r-5xxup5ze',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedAttachmentsTest-480515778',owner_user_name='tempest-TaggedAttachmentsTest-480515778-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:25:00Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='8159c9e78af04e01a1faf9b3cfc3591e',uuid=f457e6ea-8d06-46c6-be2e-f650e28a3b69,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "dc244cf9-b0f6-4906-b9f3-53634b1847e3", "address": "fa:16:3e:0e:77:88", "network": {"id": "3fdd9499-7552-4ecc-a897-ed4c58be9519", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-969934765-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bde0634670914ae8bfbbc6b31c09d6dd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdc244cf9-b0", "ovs_interfaceid": "dc244cf9-b0f6-4906-b9f3-53634b1847e3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:25:01 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Converting VIF {"id": "dc244cf9-b0f6-4906-b9f3-53634b1847e3", "address": "fa:16:3e:0e:77:88", "network": {"id": "3fdd9499-7552-4ecc-a897-ed4c58be9519", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-969934765-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bde0634670914ae8bfbbc6b31c09d6dd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdc244cf9-b0", "ovs_interfaceid": "dc244cf9-b0f6-4906-b9f3-53634b1847e3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:25:01 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:0e:77:88,bridge_name='br-int',has_traffic_filtering=True,id=dc244cf9-b0f6-4906-b9f3-53634b1847e3,network=Network(3fdd9499-7552-4ecc-a897-ed4c58be9519),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdc244cf9-b0') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:25:01 user nova-compute[72351]: DEBUG os_vif [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:0e:77:88,bridge_name='br-int',has_traffic_filtering=True,id=dc244cf9-b0f6-4906-b9f3-53634b1847e3,network=Network(3fdd9499-7552-4ecc-a897-ed4c58be9519),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdc244cf9-b0') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:25:01 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:01 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:25:01 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:25:01 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:01 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapdc244cf9-b0, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:25:01 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapdc244cf9-b0, col_values=(('external_ids', {'iface-id': 'dc244cf9-b0f6-4906-b9f3-53634b1847e3', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:0e:77:88', 'vm-uuid': 'f457e6ea-8d06-46c6-be2e-f650e28a3b69'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:25:01 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:01 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:25:01 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:01 user nova-compute[72351]: INFO os_vif [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:0e:77:88,bridge_name='br-int',has_traffic_filtering=True,id=dc244cf9-b0f6-4906-b9f3-53634b1847e3,network=Network(3fdd9499-7552-4ecc-a897-ed4c58be9519),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdc244cf9-b0')
Aug 09 02:25:01 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:25:01 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] No BDM found with device name hda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:25:01 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] No VIF found with MAC fa:16:3e:0e:77:88, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:25:01 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Using config drive
Aug 09 02:25:02 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Creating config drive at /opt/stack/data/nova/instances/f457e6ea-8d06-46c6-be2e-f650e28a3b69/disk.config
Aug 09 02:25:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Running cmd (subprocess): genisoimage -o /opt/stack/data/nova/instances/f457e6ea-8d06-46c6-be2e-f650e28a3b69/disk.config -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 0.0.0 -quiet -J -r -V config-2 /tmp/tmpug3gsb6e {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:02 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:25:02 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:25:02 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:25:02 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Starting heal instance info cache {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9842}}
Aug 09 02:25:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] CMD "genisoimage -o /opt/stack/data/nova/instances/f457e6ea-8d06-46c6-be2e-f650e28a3b69/disk.config -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 0.0.0 -quiet -J -r -V config-2 /tmp/tmpug3gsb6e" returned: 0 in 0.071s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:02 user nova-compute[72351]: DEBUG nova.network.neutron [req-8b33cd6b-333f-4203-96f4-531dee963fc1 req-40c63799-24e2-4bbe-b7b8-bcf65b33c722 service nova] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Updated VIF entry in instance network info cache for port dc244cf9-b0f6-4906-b9f3-53634b1847e3. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:25:02 user nova-compute[72351]: DEBUG nova.network.neutron [req-8b33cd6b-333f-4203-96f4-531dee963fc1 req-40c63799-24e2-4bbe-b7b8-bcf65b33c722 service nova] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Updating instance_info_cache with network_info: [{"id": "dc244cf9-b0f6-4906-b9f3-53634b1847e3", "address": "fa:16:3e:0e:77:88", "network": {"id": "3fdd9499-7552-4ecc-a897-ed4c58be9519", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-969934765-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bde0634670914ae8bfbbc6b31c09d6dd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdc244cf9-b0", "ovs_interfaceid": "dc244cf9-b0f6-4906-b9f3-53634b1847e3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:25:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "refresh_cache-e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:25:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquired lock "refresh_cache-e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:25:02 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Forcefully refreshing network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1996}}
Aug 09 02:25:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-8b33cd6b-333f-4203-96f4-531dee963fc1 req-40c63799-24e2-4bbe-b7b8-bcf65b33c722 service nova] Releasing lock "refresh_cache-f457e6ea-8d06-46c6-be2e-f650e28a3b69" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:25:02 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Updating instance_info_cache with network_info: [{"id": "9481e457-9c23-4c45-b0bd-1faf63f8b7b8", "address": "fa:16:3e:69:03:30", "network": {"id": "49e8751c-e119-430f-9110-3dc6069b1589", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-899275135-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3aea1e66805146f7a8cd1d97e64a24af", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9481e457-9c", "ovs_interfaceid": "9481e457-9c23-4c45-b0bd-1faf63f8b7b8", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:25:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Releasing lock "refresh_cache-e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:25:02 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Updated the network info_cache for instance {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9913}}
Aug 09 02:25:02 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:25:02 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:25:02 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:02 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:25:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:03 user nova-compute[72351]: DEBUG nova.compute.manager [req-f18268d4-2a63-4d8f-b072-e59508bf2d7b req-fca4f024-f1ad-4820-ae23-29c052d674e0 service nova] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Received event network-vif-plugged-dc244cf9-b0f6-4906-b9f3-53634b1847e3 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:25:03 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-f18268d4-2a63-4d8f-b072-e59508bf2d7b req-fca4f024-f1ad-4820-ae23-29c052d674e0 service nova] Acquiring lock "f457e6ea-8d06-46c6-be2e-f650e28a3b69-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:25:03 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-f18268d4-2a63-4d8f-b072-e59508bf2d7b req-fca4f024-f1ad-4820-ae23-29c052d674e0 service nova] Lock "f457e6ea-8d06-46c6-be2e-f650e28a3b69-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:25:03 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-f18268d4-2a63-4d8f-b072-e59508bf2d7b req-fca4f024-f1ad-4820-ae23-29c052d674e0 service nova] Lock "f457e6ea-8d06-46c6-be2e-f650e28a3b69-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:25:03 user nova-compute[72351]: DEBUG nova.compute.manager [req-f18268d4-2a63-4d8f-b072-e59508bf2d7b req-fca4f024-f1ad-4820-ae23-29c052d674e0 service nova] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] No waiting events found dispatching network-vif-plugged-dc244cf9-b0f6-4906-b9f3-53634b1847e3 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:25:03 user nova-compute[72351]: WARNING nova.compute.manager [req-f18268d4-2a63-4d8f-b072-e59508bf2d7b req-fca4f024-f1ad-4820-ae23-29c052d674e0 service nova] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Received unexpected event network-vif-plugged-dc244cf9-b0f6-4906-b9f3-53634b1847e3 for instance with vm_state building and task_state spawning.
Aug 09 02:25:03 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:25:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:05 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547905.641297, f457e6ea-8d06-46c6-be2e-f650e28a3b69 => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:25:05 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] VM Resumed (Lifecycle Event)
Aug 09 02:25:05 user nova-compute[72351]: DEBUG nova.compute.manager [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:25:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Guest created on hypervisor {{(pid=72351) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4411}}
Aug 09 02:25:05 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Instance spawned successfully.
Aug 09 02:25:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:894}}
Aug 09 02:25:05 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:25:05 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:25:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Found default for hw_cdrom_bus of ide {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:25:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Found default for hw_disk_bus of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:25:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Found default for hw_input_bus of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:25:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Found default for hw_pointer_model of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:25:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Found default for hw_video_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:25:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Found default for hw_vif_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:25:05 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:25:05 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547905.6424673, f457e6ea-8d06-46c6-be2e-f650e28a3b69 => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:25:05 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] VM Started (Lifecycle Event)
Aug 09 02:25:05 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:25:05 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:25:05 user nova-compute[72351]: DEBUG nova.compute.manager [req-738b6b10-1c24-4cea-a206-820feee9ba47 req-1a1e939d-13c3-42ce-8df6-9e6a1da918fa service nova] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Received event network-vif-plugged-dc244cf9-b0f6-4906-b9f3-53634b1847e3 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:25:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-738b6b10-1c24-4cea-a206-820feee9ba47 req-1a1e939d-13c3-42ce-8df6-9e6a1da918fa service nova] Acquiring lock "f457e6ea-8d06-46c6-be2e-f650e28a3b69-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:25:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-738b6b10-1c24-4cea-a206-820feee9ba47 req-1a1e939d-13c3-42ce-8df6-9e6a1da918fa service nova] Lock "f457e6ea-8d06-46c6-be2e-f650e28a3b69-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:25:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-738b6b10-1c24-4cea-a206-820feee9ba47 req-1a1e939d-13c3-42ce-8df6-9e6a1da918fa service nova] Lock "f457e6ea-8d06-46c6-be2e-f650e28a3b69-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:25:05 user nova-compute[72351]: DEBUG nova.compute.manager [req-738b6b10-1c24-4cea-a206-820feee9ba47 req-1a1e939d-13c3-42ce-8df6-9e6a1da918fa service nova] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] No waiting events found dispatching network-vif-plugged-dc244cf9-b0f6-4906-b9f3-53634b1847e3 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:25:05 user nova-compute[72351]: WARNING nova.compute.manager [req-738b6b10-1c24-4cea-a206-820feee9ba47 req-1a1e939d-13c3-42ce-8df6-9e6a1da918fa service nova] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Received unexpected event network-vif-plugged-dc244cf9-b0f6-4906-b9f3-53634b1847e3 for instance with vm_state building and task_state spawning.
Aug 09 02:25:05 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:25:05 user nova-compute[72351]: INFO nova.compute.manager [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Took 6.01 seconds to spawn the instance on the hypervisor.
Aug 09 02:25:05 user nova-compute[72351]: DEBUG nova.compute.manager [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:25:05 user nova-compute[72351]: INFO nova.compute.manager [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Took 6.91 seconds to build instance.
Aug 09 02:25:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b65219e0-7889-4d26-a63a-df6f884a6eb0 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Lock "f457e6ea-8d06-46c6-be2e-f650e28a3b69" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 7.069s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:25:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b1b1cc98-da64-4caa-a586-7b197a0c1ee6 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 02:25:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-b1b1cc98-da64-4caa-a586-7b197a0c1ee6 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'oTnweUhv', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:25:05 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-b1b1cc98-da64-4caa-a586-7b197a0c1ee6 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:25:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-b1b1cc98-da64-4caa-a586-7b197a0c1ee6 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:25:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-b1b1cc98-da64-4caa-a586-7b197a0c1ee6 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:25:05 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-b1b1cc98-da64-4caa-a586-7b197a0c1ee6 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:25:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-b1b1cc98-da64-4caa-a586-7b197a0c1ee6 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:25:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-b1b1cc98-da64-4caa-a586-7b197a0c1ee6 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:25:05 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-b1b1cc98-da64-4caa-a586-7b197a0c1ee6 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 02:25:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260" returned: 21 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:06 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[8e758e8f-12f1-424d-a56f-055e5319f30c]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:25:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-b1b1cc98-da64-4caa-a586-7b197a0c1ee6 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Aug 09 02:25:06 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:25:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 --interface default --op new {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:06 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[7e1e32e7-0df0-438e-8163-21984002d929]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd] added\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:25:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-b1b1cc98-da64-4caa-a586-7b197a0c1ee6 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd] added
Aug 09 02:25:06 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:25:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.042s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:06 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[aa9ab984-f470-4aec-9f91-dbb619ed8bc1]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:25:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-b1b1cc98-da64-4caa-a586-7b197a0c1ee6 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:25:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:06 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e9901fa7-44fd-4558-a2cf-7d58ff08000e]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:25:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-b1b1cc98-da64-4caa-a586-7b197a0c1ee6 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:25:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 --op update -n node.session.auth.username -v oTnweUhv {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 --op update -n node.session.auth.username -v oTnweUhv" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:06 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[2fdff3a9-84ee-49ec-8bd0-a3aefba389ac]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:25:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-b1b1cc98-da64-4caa-a586-7b197a0c1ee6 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'oTnweUhv'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:25:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:06 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[fa6d28ce-c0c8-4a9b-9d46-534e2978f002]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:25:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-b1b1cc98-da64-4caa-a586-7b197a0c1ee6 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:25:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:06 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[0974ca51-75a8-4fc2-9343-a23b8a63ae6a]: (4, ('tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)\ntcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:25:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-b1b1cc98-da64-4caa-a586-7b197a0c1ee6 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)
Aug 09 02:25:06 user nova-compute[72351]: tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065 (non-flash)
Aug 09 02:25:06 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:25:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-b1b1cc98-da64-4caa-a586-7b197a0c1ee6 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] iscsi session list stdout=tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)
Aug 09 02:25:06 user nova-compute[72351]: tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065 (non-flash)
Aug 09 02:25:06 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:25:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 --login {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 --login" returned: 0 in 0.059s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:06 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[03cbae4a-e057-49f5-9edb-cb2526d54a52]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:25:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-b1b1cc98-da64-4caa-a586-7b197a0c1ee6 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd, portal: 172.16.0.220,3260]
Aug 09 02:25:06 user nova-compute[72351]: Login to [iface: default, target: iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd, portal: 172.16.0.220,3260] successful.
Aug 09 02:25:06 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:25:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.029s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:06 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[3369460c-b7fb-469c-9b45-9547638b239a]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:25:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-b1b1cc98-da64-4caa-a586-7b197a0c1ee6 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:25:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.026s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:06 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[cb738cb9-0f81-4aa9-b050-505faf1126d0]: (4, ('tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)\ntcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065 (non-flash)\ntcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:25:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-b1b1cc98-da64-4caa-a586-7b197a0c1ee6 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)
Aug 09 02:25:06 user nova-compute[72351]: tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065 (non-flash)
Aug 09 02:25:06 user nova-compute[72351]: tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:25:06 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:25:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-b1b1cc98-da64-4caa-a586-7b197a0c1ee6 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] iscsi session list stdout=tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)
Aug 09 02:25:06 user nova-compute[72351]: tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065 (non-flash)
Aug 09 02:25:06 user nova-compute[72351]: tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:25:06 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:25:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-b1b1cc98-da64-4caa-a586-7b197a0c1ee6 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.305s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:25:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-b1b1cc98-da64-4caa-a586-7b197a0c1ee6 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 02:25:06 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-b1b1cc98-da64-4caa-a586-7b197a0c1ee6 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] HCTL ('34', '-', '-', 0) found on session 20 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 02:25:06 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-b1b1cc98-da64-4caa-a586-7b197a0c1ee6 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Scanning host 34 c: -, t: -, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 02:25:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.024s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:06 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[44c5fa8d-c777-46b7-bf71-ae1ceec549a5]: (4, ('- - 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:25:06 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-b1b1cc98-da64-4caa-a586-7b197a0c1ee6 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Searching for a device in session 20 and hctl ['34', '*', '*', 0] yield: None {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:25:06 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:07 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-b1b1cc98-da64-4caa-a586-7b197a0c1ee6 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Searching for a device in session 20 and hctl ['34', '*', '*', 0] yield: sdc {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:25:07 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-b1b1cc98-da64-4caa-a586-7b197a0c1ee6 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Connected to sdc using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'oTnweUhv', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 02:25:07 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-b1b1cc98-da64-4caa-a586-7b197a0c1ee6 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.341s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:25:07 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-b1b1cc98-da64-4caa-a586-7b197a0c1ee6 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] <== connect_volume: return (1342ms) {'type': 'block', 'scsi_wwn': '23062643536626331', 'path': '/dev/sdc'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:25:07 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b1b1cc98-da64-4caa-a586-7b197a0c1ee6 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23062643536626331', 'path': '/dev/sdc'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 02:25:07 user nova-compute[72351]: DEBUG nova.objects.instance [None req-b1b1cc98-da64-4caa-a586-7b197a0c1ee6 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Lazy-loading 'flavor' on Instance uuid 75bbbead-8628-448b-9484-19bf4ab922ff {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:25:07 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-b1b1cc98-da64-4caa-a586-7b197a0c1ee6 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] attach device xml: <disk type="block" device="disk">
Aug 09 02:25:07 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:25:07 user nova-compute[72351]:   <source dev="/dev/sdc"/>
Aug 09 02:25:07 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:25:07 user nova-compute[72351]:   <serial>2db51b4f-d032-4367-bb6f-8964563eb8fd</serial>
Aug 09 02:25:07 user nova-compute[72351]: </disk>
Aug 09 02:25:07 user nova-compute[72351]:  {{(pid=72351) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Aug 09 02:25:07 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b1b1cc98-da64-4caa-a586-7b197a0c1ee6 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:25:07 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b1b1cc98-da64-4caa-a586-7b197a0c1ee6 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:25:07 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b1b1cc98-da64-4caa-a586-7b197a0c1ee6 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] No VIF found with MAC fa:16:3e:8b:b5:2e, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:25:07 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:07 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b1b1cc98-da64-4caa-a586-7b197a0c1ee6 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Lock "75bbbead-8628-448b-9484-19bf4ab922ff" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 10.877s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:25:10 user nova-compute[72351]: DEBUG nova.compute.manager [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Stashing vm_state: active {{(pid=72351) _prep_resize /opt/stack/nova/nova/compute/manager.py:5535}}
Aug 09 02:25:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.resize_claim" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:25:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.resize_claim" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:25:11 user nova-compute[72351]: DEBUG nova.objects.instance [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Lazy-loading 'pci_requests' on Instance uuid 75bbbead-8628-448b-9484-19bf4ab922ff {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:25:11 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72351) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Aug 09 02:25:11 user nova-compute[72351]: INFO nova.compute.claims [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Claim successful on node user
Aug 09 02:25:11 user nova-compute[72351]: DEBUG nova.objects.instance [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Lazy-loading 'resources' on Instance uuid 75bbbead-8628-448b-9484-19bf4ab922ff {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:25:11 user nova-compute[72351]: DEBUG nova.objects.instance [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Lazy-loading 'pci_devices' on Instance uuid 75bbbead-8628-448b-9484-19bf4ab922ff {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:25:11 user nova-compute[72351]: INFO nova.compute.resource_tracker [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Updating resource usage from migration 161f097f-81e6-4201-8af0-05f72c1ba9e0
Aug 09 02:25:11 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:25:11 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:25:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.resize_claim" :: held 0.527s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:25:11 user nova-compute[72351]: INFO nova.compute.manager [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Migrating
Aug 09 02:25:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Acquiring lock "compute-rpcapi-router" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:25:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Acquired lock "compute-rpcapi-router" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:25:11 user nova-compute[72351]: INFO nova.compute.rpcapi [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Automatically selected compute RPC version 6.2 from minimum service version 66
Aug 09 02:25:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Releasing lock "compute-rpcapi-router" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:25:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Acquiring lock "refresh_cache-75bbbead-8628-448b-9484-19bf4ab922ff" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:25:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Acquired lock "refresh_cache-75bbbead-8628-448b-9484-19bf4ab922ff" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:25:11 user nova-compute[72351]: DEBUG nova.network.neutron [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:25:11 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:12 user nova-compute[72351]: DEBUG nova.network.neutron [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Updating instance_info_cache with network_info: [{"id": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "address": "fa:16:3e:8b:b5:2e", "network": {"id": "8231f2c2-237e-41c6-8892-720b950c1453", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherB-1917380761-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.30", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fc90e94bbcb54fb6b86a9ac45edb596a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbe4ec483-85", "ovs_interfaceid": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:25:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Releasing lock "refresh_cache-75bbbead-8628-448b-9484-19bf4ab922ff" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:25:12 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Starting migrate_disk_and_power_off {{(pid=72351) migrate_disk_and_power_off /opt/stack/nova/nova/virt/libvirt/driver.py:11487}}
Aug 09 02:25:12 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:12 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:12 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:12 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:12 user nova-compute[72351]: DEBUG nova.compute.manager [req-4288733f-3573-4528-966e-5174cd13e880 req-8c494642-526b-4fc2-b01c-d6bdf821b145 service nova] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Received event network-vif-unplugged-be4ec483-8532-45f7-bb9f-0b50952ce94b {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:25:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-4288733f-3573-4528-966e-5174cd13e880 req-8c494642-526b-4fc2-b01c-d6bdf821b145 service nova] Acquiring lock "75bbbead-8628-448b-9484-19bf4ab922ff-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:25:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-4288733f-3573-4528-966e-5174cd13e880 req-8c494642-526b-4fc2-b01c-d6bdf821b145 service nova] Lock "75bbbead-8628-448b-9484-19bf4ab922ff-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:25:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-4288733f-3573-4528-966e-5174cd13e880 req-8c494642-526b-4fc2-b01c-d6bdf821b145 service nova] Lock "75bbbead-8628-448b-9484-19bf4ab922ff-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:25:12 user nova-compute[72351]: DEBUG nova.compute.manager [req-4288733f-3573-4528-966e-5174cd13e880 req-8c494642-526b-4fc2-b01c-d6bdf821b145 service nova] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] No waiting events found dispatching network-vif-unplugged-be4ec483-8532-45f7-bb9f-0b50952ce94b {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:25:12 user nova-compute[72351]: WARNING nova.compute.manager [req-4288733f-3573-4528-966e-5174cd13e880 req-8c494642-526b-4fc2-b01c-d6bdf821b145 service nova] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Received unexpected event network-vif-unplugged-be4ec483-8532-45f7-bb9f-0b50952ce94b for instance with vm_state active and task_state resize_migrating.
Aug 09 02:25:13 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Instance destroyed successfully.
Aug 09 02:25:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:21:09Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestOtherB-server-634605118',display_name='tempest-ServerActionsTestOtherB-server-634605118',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serveractionstestotherb-server-634605118',id=15,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLdtpWQKmHav2tGxZemUVMEaT9TXyD7Duhy5QV84PPI+5KrgLVkbxVzWgryocScOebEprbcJ80bjBPnYhLtQJYSe5MXXz6GOQHLb+tJCZajhrDoMbn41/bPAnB5B3oFwCw==',key_name='tempest-keypair-644333596',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:21:17Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=MigrationContext,new_flavor=Flavor(7),node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='fc90e94bbcb54fb6b86a9ac45edb596a',ramdisk_id='',reservation_id='r-ozz8gdfj',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=ServiceList,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',old_vm_state='active',owner_project_name='tempest-ServerActionsTestOtherB-1726991791',owner_user_name='tempest-ServerActionsTestOtherB-1726991791-project-member'},tags=<?>,task_state='resize_migrating',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:25:11Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='19668b0c790d49dd830c39e3c2be29bf',uuid=75bbbead-8628-448b-9484-19bf4ab922ff,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "address": "fa:16:3e:8b:b5:2e", "network": {"id": "8231f2c2-237e-41c6-8892-720b950c1453", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherB-1917380761-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.30", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestOtherB-1917380761-network", "vif_mac": "fa:16:3e:8b:b5:2e"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fc90e94bbcb54fb6b86a9ac45edb596a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbe4ec483-85", "ovs_interfaceid": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:25:13 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Converting VIF {"id": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "address": "fa:16:3e:8b:b5:2e", "network": {"id": "8231f2c2-237e-41c6-8892-720b950c1453", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherB-1917380761-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.30", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestOtherB-1917380761-network", "vif_mac": "fa:16:3e:8b:b5:2e"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fc90e94bbcb54fb6b86a9ac45edb596a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbe4ec483-85", "ovs_interfaceid": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:25:13 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:8b:b5:2e,bridge_name='br-int',has_traffic_filtering=True,id=be4ec483-8532-45f7-bb9f-0b50952ce94b,network=Network(8231f2c2-237e-41c6-8892-720b950c1453),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapbe4ec483-85') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:25:13 user nova-compute[72351]: DEBUG os_vif [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:8b:b5:2e,bridge_name='br-int',has_traffic_filtering=True,id=be4ec483-8532-45f7-bb9f-0b50952ce94b,network=Network(8231f2c2-237e-41c6-8892-720b950c1453),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapbe4ec483-85') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:25:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapbe4ec483-85, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:25:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:25:13 user nova-compute[72351]: INFO os_vif [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:8b:b5:2e,bridge_name='br-int',has_traffic_filtering=True,id=be4ec483-8532-45f7-bb9f-0b50952ce94b,network=Network(8231f2c2-237e-41c6-8892-720b950c1453),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapbe4ec483-85')
Aug 09 02:25:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 02:25:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'oTnweUhv', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc'}, None), 'kwargs': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:25:13 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:25:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:25:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:25:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 02:25:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:13 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[acf1f064-28bb-4d1f-8bc4-d702417e4d15]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:25:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.027s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:13 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[bdb6b215-1b06-45a8-bb63-e59d5ae4558d]: (4, ('tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)\ntcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065 (non-flash)\ntcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:25:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)
Aug 09 02:25:13 user nova-compute[72351]: tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065 (non-flash)
Aug 09 02:25:13 user nova-compute[72351]: tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:25:13 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:25:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] iscsi session list stdout=tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)
Aug 09 02:25:13 user nova-compute[72351]: tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065 (non-flash)
Aug 09 02:25:13 user nova-compute[72351]: tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:25:13 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:25:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fcfdc1ef250>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd'): ({'sdc'}, set())}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 02:25:13 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Removing single pathed devices sdc {{(pid=72351) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Aug 09 02:25:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:13 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[0d170f4a-7bd3-4a00-b0f2-12f1c737b479]: (4, ('path checker states:\nup                  3\n\npaths: 0\nbusy: False\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:25:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdc {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdc" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:13 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[4b2381d7-d383-4a4a-8f39-038ba92c51b9]: (4, ('ok\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:25:13 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Flushing IO for device /dev/sdc {{(pid=72351) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Aug 09 02:25:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdc {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdc" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:13 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d560c49c-ec51-42bf-8a5e-cf735112460d]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:25:13 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Remove SCSI device /dev/sdc with /sys/block/sdc/device/delete {{(pid=72351) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Aug 09 02:25:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdc/device/delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdc/device/delete" returned: 0 in 0.046s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:13 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[907aab91-b956-4ce1-9aa6-3e7933ac39fa]: (4, ('1', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:25:13 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Checking to see if SCSI volumes sdc have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Aug 09 02:25:13 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] SCSI volumes sdc have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Aug 09 02:25:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd')] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 02:25:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:13 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[9710b44e-6f57-4d65-9d30-eb4da45c00d6]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:25:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:25:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 --logout {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 --logout" returned: 0 in 0.048s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:13 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[6edcb254-ad9a-4a8e-bde7-aa810801421d]: (4, ('Logging out of session [sid: 20, target: iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd, portal: 172.16.0.220,3260]\nLogout of [sid: 20, target: iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:25:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 20, target: iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd, portal: 172.16.0.220,3260]
Aug 09 02:25:13 user nova-compute[72351]: Logout of [sid: 20, target: iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd, portal: 172.16.0.220,3260] successful.
Aug 09 02:25:13 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:25:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 --op delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 --op delete" returned: 0 in 0.025s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:13 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[c747a9a2-8b89-4b53-8dab-6d315f72954a]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:25:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:25:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.289s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:25:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] <== disconnect_volume: return (291ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:25:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 02:25:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff/disk --force-share --output=json" returned: 0 in 0.132s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff/disk --force-share --output=json" returned: 0 in 0.130s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] skipping disk /dev/sdc (vdb) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:25:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Running cmd (subprocess): cp -r /opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff_resize/disk /opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff/disk {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] CMD "cp -r /opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff_resize/disk /opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff/disk" returned: 0 in 0.319s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Running cmd (subprocess): cp -r /opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff_resize/disk.info /opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff/disk.info {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] CMD "cp -r /opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff_resize/disk.info /opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff/disk.info" returned: 0 in 0.036s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:14 user nova-compute[72351]: DEBUG nova.compute.manager [req-41425fb6-c46f-40e2-9a53-3f7e65e3331c req-8b1c7c54-3479-427d-8dc7-2c4eefff4913 service nova] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Received event network-vif-plugged-be4ec483-8532-45f7-bb9f-0b50952ce94b {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:25:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-41425fb6-c46f-40e2-9a53-3f7e65e3331c req-8b1c7c54-3479-427d-8dc7-2c4eefff4913 service nova] Acquiring lock "75bbbead-8628-448b-9484-19bf4ab922ff-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:25:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-41425fb6-c46f-40e2-9a53-3f7e65e3331c req-8b1c7c54-3479-427d-8dc7-2c4eefff4913 service nova] Lock "75bbbead-8628-448b-9484-19bf4ab922ff-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:25:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-41425fb6-c46f-40e2-9a53-3f7e65e3331c req-8b1c7c54-3479-427d-8dc7-2c4eefff4913 service nova] Lock "75bbbead-8628-448b-9484-19bf4ab922ff-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:25:14 user nova-compute[72351]: DEBUG nova.compute.manager [req-41425fb6-c46f-40e2-9a53-3f7e65e3331c req-8b1c7c54-3479-427d-8dc7-2c4eefff4913 service nova] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] No waiting events found dispatching network-vif-plugged-be4ec483-8532-45f7-bb9f-0b50952ce94b {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:25:14 user nova-compute[72351]: WARNING nova.compute.manager [req-41425fb6-c46f-40e2-9a53-3f7e65e3331c req-8b1c7c54-3479-427d-8dc7-2c4eefff4913 service nova] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Received unexpected event network-vif-plugged-be4ec483-8532-45f7-bb9f-0b50952ce94b for instance with vm_state active and task_state resize_migrating.
Aug 09 02:25:15 user nova-compute[72351]: DEBUG nova.network.neutron [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Port be4ec483-8532-45f7-bb9f-0b50952ce94b binding to destination host user is already ACTIVE {{(pid=72351) migrate_instance_start /opt/stack/nova/nova/network/neutron.py:3163}}
Aug 09 02:25:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Acquiring lock "75bbbead-8628-448b-9484-19bf4ab922ff-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:25:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Lock "75bbbead-8628-448b-9484-19bf4ab922ff-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:25:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Lock "75bbbead-8628-448b-9484-19bf4ab922ff-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:25:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Acquiring lock "refresh_cache-75bbbead-8628-448b-9484-19bf4ab922ff" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:25:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Acquired lock "refresh_cache-75bbbead-8628-448b-9484-19bf4ab922ff" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:25:15 user nova-compute[72351]: DEBUG nova.network.neutron [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:25:16 user nova-compute[72351]: DEBUG nova.network.neutron [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Updating instance_info_cache with network_info: [{"id": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "address": "fa:16:3e:8b:b5:2e", "network": {"id": "8231f2c2-237e-41c6-8892-720b950c1453", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherB-1917380761-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.30", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fc90e94bbcb54fb6b86a9ac45edb596a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbe4ec483-85", "ovs_interfaceid": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:25:16 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Releasing lock "refresh_cache-75bbbead-8628-448b-9484-19bf4ab922ff" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:25:16 user nova-compute[72351]: DEBUG os_brick.utils [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:25:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.015s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:16 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[dc25b86f-bb0c-4f29-a10a-e82699608ab8]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:a52269602ff4\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:25:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:16 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[cd089677-8950-4278-9a17-f5557d41e2da]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:25:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:16 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[9eb0c110-3aaa-4e83-8ca8-4244c9b1ab35]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:25:16 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d83bae01-6605-4042-852f-01df4fe58956]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:25:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Running cmd (subprocess): nvme version {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] 'nvme version' failed. Not Retrying. {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:25:16 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.nvmeof [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] nvme not present on system {{(pid=72351) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Aug 09 02:25:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:25:16 user nova-compute[72351]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Aug 09 02:25:16 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[df881cb8-8dfb-4a04-adfa-1597b293d198]: (4, '') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:25:16 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72351) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Aug 09 02:25:16 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Aug 09 02:25:16 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] LIGHTOS: no hostnqn found. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Aug 09 02:25:16 user nova-compute[72351]: DEBUG os_brick.utils [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] <== get_connector_properties: return (139ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:a52269602ff4', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:25:17 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:23 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:25 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Starting finish_migration {{(pid=72351) finish_migration /opt/stack/nova/nova/virt/libvirt/driver.py:11674}}
Aug 09 02:25:25 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Instance directory exists: not creating {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4713}}
Aug 09 02:25:25 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Creating image(s)
Aug 09 02:25:25 user nova-compute[72351]: DEBUG nova.objects.instance [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Lazy-loading 'trusted_certs' on Instance uuid 75bbbead-8628-448b-9484-19bf4ab922ff {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:25:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.141s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:25 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Checking if we can resize image /opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff/disk. size=21474836480 {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Aug 09 02:25:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Running cmd (subprocess): qemu-img resize /opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff/disk 21474836480 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] CMD "qemu-img resize /opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff/disk 21474836480" returned: 0 in 0.051s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:25 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Did not create local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4853}}
Aug 09 02:25:25 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Ensure instance console log exists: /opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff/console.log {{(pid=72351) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4603}}
Aug 09 02:25:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:25:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:25:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:25:25 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Start _get_guest_xml network_info=[{"id": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "address": "fa:16:3e:8b:b5:2e", "network": {"id": "8231f2c2-237e-41c6-8892-720b950c1453", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherB-1917380761-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.30", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestOtherB-1917380761-network", "vif_mac": "fa:16:3e:8b:b5:2e"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fc90e94bbcb54fb6b86a9ac45edb596a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbe4ec483-85", "ovs_interfaceid": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vdb': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'size': 0, 'device_type': 'disk', 'encryption_secret_uuid': None, 'guest_format': None, 'encryption_options': None, 'disk_bus': 'virtio', 'encrypted': False, 'encryption_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'image_id': '18ccda7f-31d3-4978-ab9d-4ed2f74eff94'}], 'ephemerals': [], 'block_device_mapping': [{'device_type': 'disk', 'guest_format': None, 'disk_bus': 'virtio', 'mount_device': '/dev/vdb', 'boot_index': None, 'attachment_id': '1ebf014f-9b07-4325-9229-97a38f9f28d5', 'delete_on_termination': False, 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqns': ['iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'oTnweUhv', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'attaching', 'instance': '75bbbead-8628-448b-9484-19bf4ab922ff', 'attached_at': '2023-08-09T02:25:25.000000', 'detached_at': '', 'volume_id': '2db51b4f-d032-4367-bb6f-8964563eb8fd', 'serial': '2db51b4f-d032-4367-bb6f-8964563eb8fd'}, 'volume_type': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:25:26 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:25:26 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:25:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:06Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='2',id=7,is_public=True,memory_mb=2048,name='m1.small',projects=<?>,root_gb=20,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG nova.objects.instance [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Lazy-loading 'vcpu_model' on Instance uuid 75bbbead-8628-448b-9484-19bf4ab922ff {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'oTnweUhv', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:25:26 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:25:26 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:25:26 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:25:26 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:25:26 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 02:25:26 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260" returned: 21 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[8bb20613-dc66-4d8c-ab10-be41ddcf331c]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Aug 09 02:25:26 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 --interface default --op new {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[3d69e5c7-c159-4945-947f-26d10dbbbd7a]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd] added\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd] added
Aug 09 02:25:26 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[36e83804-3e23-4eec-9791-eb7a5c53b914]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[7e2bbd0a-91d1-4d05-b092-0a182248a055]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 --op update -n node.session.auth.username -v oTnweUhv {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 --op update -n node.session.auth.username -v oTnweUhv" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[74dcc4bf-7bb3-43f6-ab01-62723872eb01]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'oTnweUhv'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.026s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d8ed7a74-01eb-4e32-bebd-3eefa9af811c]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[7bd172f9-9081-44f2-99de-c30e4cde1740]: (4, ('tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)\ntcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)
Aug 09 02:25:26 user nova-compute[72351]: tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065 (non-flash)
Aug 09 02:25:26 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] iscsi session list stdout=tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)
Aug 09 02:25:26 user nova-compute[72351]: tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065 (non-flash)
Aug 09 02:25:26 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 --login {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 --login" returned: 0 in 0.070s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[cfb9508e-f321-45bc-b673-84a2db289884]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd, portal: 172.16.0.220,3260]
Aug 09 02:25:26 user nova-compute[72351]: Login to [iface: default, target: iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd, portal: 172.16.0.220,3260] successful.
Aug 09 02:25:26 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.030s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[023452d9-73ce-4d02-b0e5-fea92fbfe020]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e297bb49-71c5-4bab-b081-6b1686f640b5]: (4, ('tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)\ntcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065 (non-flash)\ntcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)
Aug 09 02:25:26 user nova-compute[72351]: tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065 (non-flash)
Aug 09 02:25:26 user nova-compute[72351]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:25:26 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] iscsi session list stdout=tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)
Aug 09 02:25:26 user nova-compute[72351]: tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065 (non-flash)
Aug 09 02:25:26 user nova-compute[72351]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:25:26 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.312s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] HCTL ('34', '-', '-', 0) found on session 21 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Scanning host 34 c: -, t: -, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[93841ec7-88f9-45f5-9c4b-e12c848ef15a]: (4, ('- - 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:25:26 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Searching for a device in session 21 and hctl ['34', '*', '*', 0] yield: None {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:25:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:27 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Searching for a device in session 21 and hctl ['34', '*', '*', 0] yield: sdc {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:25:27 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Connected to sdc using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'oTnweUhv', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 02:25:27 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.341s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:25:27 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] <== connect_volume: return (1342ms) {'type': 'block', 'scsi_wwn': '23062643536626331', 'path': '/dev/sdc'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:25:27 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23062643536626331', 'path': '/dev/sdc'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 02:25:27 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:21:09Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestOtherB-server-634605118',display_name='tempest-ServerActionsTestOtherB-server-634605118',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(7),hidden=False,host='user',hostname='tempest-serveractionstestotherb-server-634605118',id=15,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=7,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLdtpWQKmHav2tGxZemUVMEaT9TXyD7Duhy5QV84PPI+5KrgLVkbxVzWgryocScOebEprbcJ80bjBPnYhLtQJYSe5MXXz6GOQHLb+tJCZajhrDoMbn41/bPAnB5B3oFwCw==',key_name='tempest-keypair-644333596',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:21:17Z,launched_on='user',locked=False,locked_by=None,memory_mb=2048,metadata={},migration_context=MigrationContext,new_flavor=Flavor(7),node='user',numa_topology=None,old_flavor=Flavor(6),os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='fc90e94bbcb54fb6b86a9ac45edb596a',ramdisk_id='',reservation_id='r-ozz8gdfj',resources=None,root_device_name='/dev/vda',root_gb=20,security_groups=SecurityGroupList,services=ServiceList,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',old_vm_state='active',owner_project_name='tempest-ServerActionsTestOtherB-1726991791',owner_user_name='tempest-ServerActionsTestOtherB-1726991791-project-member'},tags=<?>,task_state='resize_finish',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:25:16Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='19668b0c790d49dd830c39e3c2be29bf',uuid=75bbbead-8628-448b-9484-19bf4ab922ff,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "address": "fa:16:3e:8b:b5:2e", "network": {"id": "8231f2c2-237e-41c6-8892-720b950c1453", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherB-1917380761-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.30", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestOtherB-1917380761-network", "vif_mac": "fa:16:3e:8b:b5:2e"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fc90e94bbcb54fb6b86a9ac45edb596a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbe4ec483-85", "ovs_interfaceid": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:25:27 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Converting VIF {"id": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "address": "fa:16:3e:8b:b5:2e", "network": {"id": "8231f2c2-237e-41c6-8892-720b950c1453", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherB-1917380761-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.30", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestOtherB-1917380761-network", "vif_mac": "fa:16:3e:8b:b5:2e"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fc90e94bbcb54fb6b86a9ac45edb596a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbe4ec483-85", "ovs_interfaceid": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:25:27 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:8b:b5:2e,bridge_name='br-int',has_traffic_filtering=True,id=be4ec483-8532-45f7-bb9f-0b50952ce94b,network=Network(8231f2c2-237e-41c6-8892-720b950c1453),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapbe4ec483-85') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:25:27 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:25:27 user nova-compute[72351]:   <uuid>75bbbead-8628-448b-9484-19bf4ab922ff</uuid>
Aug 09 02:25:27 user nova-compute[72351]:   <name>instance-0000000f</name>
Aug 09 02:25:27 user nova-compute[72351]:   <memory>2097152</memory>
Aug 09 02:25:27 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:25:27 user nova-compute[72351]:   <metadata>
Aug 09 02:25:27 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:25:27 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:25:27 user nova-compute[72351]:       <nova:name>tempest-ServerActionsTestOtherB-server-634605118</nova:name>
Aug 09 02:25:27 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:25:26</nova:creationTime>
Aug 09 02:25:27 user nova-compute[72351]:       <nova:flavor name="m1.small">
Aug 09 02:25:27 user nova-compute[72351]:         <nova:memory>2048</nova:memory>
Aug 09 02:25:27 user nova-compute[72351]:         <nova:disk>20</nova:disk>
Aug 09 02:25:27 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:25:27 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:25:27 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:25:27 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:25:27 user nova-compute[72351]:       <nova:owner>
Aug 09 02:25:27 user nova-compute[72351]:         <nova:user uuid="19668b0c790d49dd830c39e3c2be29bf">tempest-ServerActionsTestOtherB-1726991791-project-member</nova:user>
Aug 09 02:25:27 user nova-compute[72351]:         <nova:project uuid="fc90e94bbcb54fb6b86a9ac45edb596a">tempest-ServerActionsTestOtherB-1726991791</nova:project>
Aug 09 02:25:27 user nova-compute[72351]:       </nova:owner>
Aug 09 02:25:27 user nova-compute[72351]:       <nova:root type="image" uuid="18ccda7f-31d3-4978-ab9d-4ed2f74eff94"/>
Aug 09 02:25:27 user nova-compute[72351]:       <nova:ports>
Aug 09 02:25:27 user nova-compute[72351]:         <nova:port uuid="be4ec483-8532-45f7-bb9f-0b50952ce94b">
Aug 09 02:25:27 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.14" ipVersion="4"/>
Aug 09 02:25:27 user nova-compute[72351]:         </nova:port>
Aug 09 02:25:27 user nova-compute[72351]:       </nova:ports>
Aug 09 02:25:27 user nova-compute[72351]:     </nova:instance>
Aug 09 02:25:27 user nova-compute[72351]:   </metadata>
Aug 09 02:25:27 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:25:27 user nova-compute[72351]:     <system>
Aug 09 02:25:27 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:25:27 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:25:27 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:25:27 user nova-compute[72351]:       <entry name="serial">75bbbead-8628-448b-9484-19bf4ab922ff</entry>
Aug 09 02:25:27 user nova-compute[72351]:       <entry name="uuid">75bbbead-8628-448b-9484-19bf4ab922ff</entry>
Aug 09 02:25:27 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:25:27 user nova-compute[72351]:     </system>
Aug 09 02:25:27 user nova-compute[72351]:   </sysinfo>
Aug 09 02:25:27 user nova-compute[72351]:   <os>
Aug 09 02:25:27 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:25:27 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:25:27 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:25:27 user nova-compute[72351]:   </os>
Aug 09 02:25:27 user nova-compute[72351]:   <features>
Aug 09 02:25:27 user nova-compute[72351]:     <acpi/>
Aug 09 02:25:27 user nova-compute[72351]:     <apic/>
Aug 09 02:25:27 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:25:27 user nova-compute[72351]:   </features>
Aug 09 02:25:27 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:25:27 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:25:27 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:25:27 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:25:27 user nova-compute[72351]:   </clock>
Aug 09 02:25:27 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:25:27 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:25:27 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:25:27 user nova-compute[72351]:   </cpu>
Aug 09 02:25:27 user nova-compute[72351]:   <devices>
Aug 09 02:25:27 user nova-compute[72351]:     <disk type="file" device="disk">
Aug 09 02:25:27 user nova-compute[72351]:       <driver name="qemu" type="qcow2" cache="none"/>
Aug 09 02:25:27 user nova-compute[72351]:       <source file="/opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff/disk"/>
Aug 09 02:25:27 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:25:27 user nova-compute[72351]:     </disk>
Aug 09 02:25:27 user nova-compute[72351]:     <disk type="block" device="disk">
Aug 09 02:25:27 user nova-compute[72351]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:25:27 user nova-compute[72351]:       <source dev="/dev/sdc"/>
Aug 09 02:25:27 user nova-compute[72351]:       <target dev="vdb" bus="virtio"/>
Aug 09 02:25:27 user nova-compute[72351]:       <serial>2db51b4f-d032-4367-bb6f-8964563eb8fd</serial>
Aug 09 02:25:27 user nova-compute[72351]:     </disk>
Aug 09 02:25:27 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:25:27 user nova-compute[72351]:       <mac address="fa:16:3e:8b:b5:2e"/>
Aug 09 02:25:27 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:25:27 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:25:27 user nova-compute[72351]:       <target dev="tapbe4ec483-85"/>
Aug 09 02:25:27 user nova-compute[72351]:     </interface>
Aug 09 02:25:27 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:25:27 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff/console.log" append="off"/>
Aug 09 02:25:27 user nova-compute[72351]:     </serial>
Aug 09 02:25:27 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:25:27 user nova-compute[72351]:     <video>
Aug 09 02:25:27 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:25:27 user nova-compute[72351]:     </video>
Aug 09 02:25:27 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:25:27 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:25:27 user nova-compute[72351]:     </rng>
Aug 09 02:25:27 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:25:27 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:25:27 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:25:27 user nova-compute[72351]:     </memballoon>
Aug 09 02:25:27 user nova-compute[72351]:   </devices>
Aug 09 02:25:27 user nova-compute[72351]: </domain>
Aug 09 02:25:27 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:25:27 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:21:09Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestOtherB-server-634605118',display_name='tempest-ServerActionsTestOtherB-server-634605118',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(7),hidden=False,host='user',hostname='tempest-serveractionstestotherb-server-634605118',id=15,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=7,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLdtpWQKmHav2tGxZemUVMEaT9TXyD7Duhy5QV84PPI+5KrgLVkbxVzWgryocScOebEprbcJ80bjBPnYhLtQJYSe5MXXz6GOQHLb+tJCZajhrDoMbn41/bPAnB5B3oFwCw==',key_name='tempest-keypair-644333596',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:21:17Z,launched_on='user',locked=False,locked_by=None,memory_mb=2048,metadata={},migration_context=MigrationContext,new_flavor=Flavor(7),node='user',numa_topology=None,old_flavor=Flavor(6),os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='fc90e94bbcb54fb6b86a9ac45edb596a',ramdisk_id='',reservation_id='r-ozz8gdfj',resources=None,root_device_name='/dev/vda',root_gb=20,security_groups=SecurityGroupList,services=ServiceList,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',old_vm_state='active',owner_project_name='tempest-ServerActionsTestOtherB-1726991791',owner_user_name='tempest-ServerActionsTestOtherB-1726991791-project-member'},tags=<?>,task_state='resize_finish',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:25:16Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='19668b0c790d49dd830c39e3c2be29bf',uuid=75bbbead-8628-448b-9484-19bf4ab922ff,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "address": "fa:16:3e:8b:b5:2e", "network": {"id": "8231f2c2-237e-41c6-8892-720b950c1453", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherB-1917380761-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.30", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestOtherB-1917380761-network", "vif_mac": "fa:16:3e:8b:b5:2e"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fc90e94bbcb54fb6b86a9ac45edb596a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbe4ec483-85", "ovs_interfaceid": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:25:27 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Converting VIF {"id": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "address": "fa:16:3e:8b:b5:2e", "network": {"id": "8231f2c2-237e-41c6-8892-720b950c1453", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherB-1917380761-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.30", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestOtherB-1917380761-network", "vif_mac": "fa:16:3e:8b:b5:2e"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fc90e94bbcb54fb6b86a9ac45edb596a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbe4ec483-85", "ovs_interfaceid": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:25:27 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:8b:b5:2e,bridge_name='br-int',has_traffic_filtering=True,id=be4ec483-8532-45f7-bb9f-0b50952ce94b,network=Network(8231f2c2-237e-41c6-8892-720b950c1453),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapbe4ec483-85') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:25:27 user nova-compute[72351]: DEBUG os_vif [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:8b:b5:2e,bridge_name='br-int',has_traffic_filtering=True,id=be4ec483-8532-45f7-bb9f-0b50952ce94b,network=Network(8231f2c2-237e-41c6-8892-720b950c1453),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapbe4ec483-85') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:25:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:25:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:25:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapbe4ec483-85, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:25:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapbe4ec483-85, col_values=(('external_ids', {'iface-id': 'be4ec483-8532-45f7-bb9f-0b50952ce94b', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:8b:b5:2e', 'vm-uuid': '75bbbead-8628-448b-9484-19bf4ab922ff'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:25:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:25:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:27 user nova-compute[72351]: INFO os_vif [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:8b:b5:2e,bridge_name='br-int',has_traffic_filtering=True,id=be4ec483-8532-45f7-bb9f-0b50952ce94b,network=Network(8231f2c2-237e-41c6-8892-720b950c1453),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapbe4ec483-85')
Aug 09 02:25:27 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:25:27 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:25:27 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] No VIF found with MAC fa:16:3e:8b:b5:2e, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:25:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:28 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691547913.2900226, 75bbbead-8628-448b-9484-19bf4ab922ff => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:25:28 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] VM Stopped (Lifecycle Event)
Aug 09 02:25:28 user nova-compute[72351]: DEBUG nova.compute.manager [None req-d319a55f-4185-43ea-aaff-c7fb0599764b None None] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:25:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Acquiring lock "47afce95-e5f7-4703-a99f-7ef8b28c4bd2" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:25:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Lock "47afce95-e5f7-4703-a99f-7ef8b28c4bd2" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:25:28 user nova-compute[72351]: DEBUG nova.compute.manager [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Starting instance... {{(pid=72351) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2406}}
Aug 09 02:25:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:25:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:25:28 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72351) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Aug 09 02:25:28 user nova-compute[72351]: INFO nova.compute.claims [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Claim successful on node user
Aug 09 02:25:28 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:28 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:28 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:28 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:25:28 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:28 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:25:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.484s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:25:28 user nova-compute[72351]: DEBUG nova.compute.manager [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Start building networks asynchronously for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2803}}
Aug 09 02:25:29 user nova-compute[72351]: DEBUG nova.compute.manager [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Allocating IP information in the background. {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1956}}
Aug 09 02:25:29 user nova-compute[72351]: DEBUG nova.network.neutron [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] allocate_for_instance() {{(pid=72351) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Aug 09 02:25:29 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Aug 09 02:25:29 user nova-compute[72351]: DEBUG nova.compute.manager [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Start building block device mappings for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2838}}
Aug 09 02:25:29 user nova-compute[72351]: DEBUG nova.compute.manager [req-84d4fb44-fa9d-479f-89fd-b279251644a5 req-4bcfca32-800b-4af4-959b-fb0ac24b0cda service nova] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Received event network-vif-plugged-be4ec483-8532-45f7-bb9f-0b50952ce94b {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:25:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-84d4fb44-fa9d-479f-89fd-b279251644a5 req-4bcfca32-800b-4af4-959b-fb0ac24b0cda service nova] Acquiring lock "75bbbead-8628-448b-9484-19bf4ab922ff-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:25:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-84d4fb44-fa9d-479f-89fd-b279251644a5 req-4bcfca32-800b-4af4-959b-fb0ac24b0cda service nova] Lock "75bbbead-8628-448b-9484-19bf4ab922ff-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:25:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-84d4fb44-fa9d-479f-89fd-b279251644a5 req-4bcfca32-800b-4af4-959b-fb0ac24b0cda service nova] Lock "75bbbead-8628-448b-9484-19bf4ab922ff-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:25:29 user nova-compute[72351]: DEBUG nova.compute.manager [req-84d4fb44-fa9d-479f-89fd-b279251644a5 req-4bcfca32-800b-4af4-959b-fb0ac24b0cda service nova] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] No waiting events found dispatching network-vif-plugged-be4ec483-8532-45f7-bb9f-0b50952ce94b {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:25:29 user nova-compute[72351]: WARNING nova.compute.manager [req-84d4fb44-fa9d-479f-89fd-b279251644a5 req-4bcfca32-800b-4af4-959b-fb0ac24b0cda service nova] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Received unexpected event network-vif-plugged-be4ec483-8532-45f7-bb9f-0b50952ce94b for instance with vm_state active and task_state resize_finish.
Aug 09 02:25:29 user nova-compute[72351]: DEBUG nova.policy [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'ebc62bdcafdd4131be9abb7fe4ed2422', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eebc3a715ae04d48918b6c68b135935b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72351) authorize /opt/stack/nova/nova/policy.py:203}}
Aug 09 02:25:29 user nova-compute[72351]: DEBUG nova.compute.manager [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Start spawning the instance on the hypervisor. {{(pid=72351) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2612}}
Aug 09 02:25:29 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Creating instance directory {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4717}}
Aug 09 02:25:29 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Creating image(s)
Aug 09 02:25:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Acquiring lock "/opt/stack/data/nova/instances/47afce95-e5f7-4703-a99f-7ef8b28c4bd2/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:25:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Lock "/opt/stack/data/nova/instances/47afce95-e5f7-4703-a99f-7ef8b28c4bd2/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:25:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Lock "/opt/stack/data/nova/instances/47afce95-e5f7-4703-a99f-7ef8b28c4bd2/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:25:29 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:29 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.149s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Acquiring lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:25:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:25:29 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:29 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:29 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:29 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:29 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.168s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:29 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/47afce95-e5f7-4703-a99f-7ef8b28c4bd2/disk 1073741824 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:29 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/47afce95-e5f7-4703-a99f-7ef8b28c4bd2/disk 1073741824" returned: 0 in 0.055s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.227s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:25:29 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:29 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.141s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:29 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Checking if we can resize image /opt/stack/data/nova/instances/47afce95-e5f7-4703-a99f-7ef8b28c4bd2/disk. size=1073741824 {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Aug 09 02:25:29 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/47afce95-e5f7-4703-a99f-7ef8b28c4bd2/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:29 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/47afce95-e5f7-4703-a99f-7ef8b28c4bd2/disk --force-share --output=json" returned: 0 in 0.143s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:29 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Cannot resize image /opt/stack/data/nova/instances/47afce95-e5f7-4703-a99f-7ef8b28c4bd2/disk to a smaller size. {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Aug 09 02:25:29 user nova-compute[72351]: DEBUG nova.objects.instance [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Lazy-loading 'migration_context' on Instance uuid 47afce95-e5f7-4703-a99f-7ef8b28c4bd2 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:25:29 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Created local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4851}}
Aug 09 02:25:29 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Ensure instance console log exists: /opt/stack/data/nova/instances/47afce95-e5f7-4703-a99f-7ef8b28c4bd2/console.log {{(pid=72351) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4603}}
Aug 09 02:25:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:25:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:25:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:25:29 user nova-compute[72351]: DEBUG nova.network.neutron [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Successfully created port: 52e1842a-aa45-4804-9636-f0cfe1134eee {{(pid=72351) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Aug 09 02:25:30 user nova-compute[72351]: DEBUG nova.network.neutron [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Successfully updated port: 52e1842a-aa45-4804-9636-f0cfe1134eee {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:25:30 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Acquiring lock "refresh_cache-47afce95-e5f7-4703-a99f-7ef8b28c4bd2" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:25:30 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Acquired lock "refresh_cache-47afce95-e5f7-4703-a99f-7ef8b28c4bd2" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:25:30 user nova-compute[72351]: DEBUG nova.network.neutron [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:25:30 user nova-compute[72351]: DEBUG nova.network.neutron [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:25:31 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547931.0714772, 75bbbead-8628-448b-9484-19bf4ab922ff => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:25:31 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] VM Resumed (Lifecycle Event)
Aug 09 02:25:31 user nova-compute[72351]: DEBUG nova.compute.manager [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:25:31 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Instance running successfully.
Aug 09 02:25:31 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Failed to set time: agent not configured {{(pid=72351) sync_guest_time /opt/stack/nova/nova/virt/libvirt/guest.py:200}}
Aug 09 02:25:31 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-94a03f5f-b5f2-4692-ad53-2c7bbcf6a041 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] finish_migration finished successfully. {{(pid=72351) finish_migration /opt/stack/nova/nova/virt/libvirt/driver.py:11769}}
Aug 09 02:25:31 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:25:31 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: active, current task_state: resize_finish, current DB power_state: 1, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:25:31 user nova-compute[72351]: DEBUG nova.network.neutron [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Updating instance_info_cache with network_info: [{"id": "52e1842a-aa45-4804-9636-f0cfe1134eee", "address": "fa:16:3e:bd:df:05", "network": {"id": "9785f84c-cf45-464e-9127-332ce46abd11", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-87568186-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "eebc3a715ae04d48918b6c68b135935b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52e1842a-aa", "ovs_interfaceid": "52e1842a-aa45-4804-9636-f0cfe1134eee", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:25:31 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] During sync_power_state the instance has a pending task (resize_finish). Skip.
Aug 09 02:25:31 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547931.0730178, 75bbbead-8628-448b-9484-19bf4ab922ff => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:25:31 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] VM Started (Lifecycle Event)
Aug 09 02:25:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Releasing lock "refresh_cache-47afce95-e5f7-4703-a99f-7ef8b28c4bd2" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:25:31 user nova-compute[72351]: DEBUG nova.compute.manager [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Instance network_info: |[{"id": "52e1842a-aa45-4804-9636-f0cfe1134eee", "address": "fa:16:3e:bd:df:05", "network": {"id": "9785f84c-cf45-464e-9127-332ce46abd11", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-87568186-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "eebc3a715ae04d48918b6c68b135935b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52e1842a-aa", "ovs_interfaceid": "52e1842a-aa45-4804-9636-f0cfe1134eee", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1971}}
Aug 09 02:25:31 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:25:31 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Start _get_guest_xml network_info=[{"id": "52e1842a-aa45-4804-9636-f0cfe1134eee", "address": "fa:16:3e:bd:df:05", "network": {"id": "9785f84c-cf45-464e-9127-332ce46abd11", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-87568186-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "eebc3a715ae04d48918b6c68b135935b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52e1842a-aa", "ovs_interfaceid": "52e1842a-aa45-4804-9636-f0cfe1134eee", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'size': 0, 'device_type': 'disk', 'encryption_secret_uuid': None, 'guest_format': None, 'encryption_options': None, 'disk_bus': 'virtio', 'encrypted': False, 'encryption_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'image_id': '18ccda7f-31d3-4978-ab9d-4ed2f74eff94'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:25:31 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Synchronizing instance power state after lifecycle event "Started"; current vm_state: active, current task_state: resize_finish, current DB power_state: 1, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:25:31 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:25:31 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:25:31 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:25:31 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:25:31 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:25:31 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:25:31 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:25:31 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:25:31 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:25:31 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:25:31 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:25:31 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:25:31 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:25:31 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:25:31 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:25:31 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:25:31 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:25:31 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:25:31 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:25:27Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServersNegativeTestJSON-server-1699010745',display_name='tempest-ServersNegativeTestJSON-server-1699010745',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serversnegativetestjson-server-1699010745',id=26,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='eebc3a715ae04d48918b6c68b135935b',ramdisk_id='',reservation_id='r-lovurafx',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServersNegativeTestJSON-1773070288',owner_user_name='tempest-ServersNegativeTestJSON-1773070288-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:25:29Z,user_data=None,user_id='ebc62bdcafdd4131be9abb7fe4ed2422',uuid=47afce95-e5f7-4703-a99f-7ef8b28c4bd2,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "52e1842a-aa45-4804-9636-f0cfe1134eee", "address": "fa:16:3e:bd:df:05", "network": {"id": "9785f84c-cf45-464e-9127-332ce46abd11", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-87568186-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "eebc3a715ae04d48918b6c68b135935b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52e1842a-aa", "ovs_interfaceid": "52e1842a-aa45-4804-9636-f0cfe1134eee", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:25:31 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Converting VIF {"id": "52e1842a-aa45-4804-9636-f0cfe1134eee", "address": "fa:16:3e:bd:df:05", "network": {"id": "9785f84c-cf45-464e-9127-332ce46abd11", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-87568186-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "eebc3a715ae04d48918b6c68b135935b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52e1842a-aa", "ovs_interfaceid": "52e1842a-aa45-4804-9636-f0cfe1134eee", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:25:31 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:bd:df:05,bridge_name='br-int',has_traffic_filtering=True,id=52e1842a-aa45-4804-9636-f0cfe1134eee,network=Network(9785f84c-cf45-464e-9127-332ce46abd11),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap52e1842a-aa') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:25:31 user nova-compute[72351]: DEBUG nova.objects.instance [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Lazy-loading 'pci_devices' on Instance uuid 47afce95-e5f7-4703-a99f-7ef8b28c4bd2 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:25:31 user nova-compute[72351]: DEBUG nova.compute.manager [req-61ea7eda-a168-4724-b60a-7ccc4ae3b933 req-adde31a9-c20c-49c0-a652-4dbeb44a5e4c service nova] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Received event network-vif-plugged-be4ec483-8532-45f7-bb9f-0b50952ce94b {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:25:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-61ea7eda-a168-4724-b60a-7ccc4ae3b933 req-adde31a9-c20c-49c0-a652-4dbeb44a5e4c service nova] Acquiring lock "75bbbead-8628-448b-9484-19bf4ab922ff-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:25:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-61ea7eda-a168-4724-b60a-7ccc4ae3b933 req-adde31a9-c20c-49c0-a652-4dbeb44a5e4c service nova] Lock "75bbbead-8628-448b-9484-19bf4ab922ff-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:25:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-61ea7eda-a168-4724-b60a-7ccc4ae3b933 req-adde31a9-c20c-49c0-a652-4dbeb44a5e4c service nova] Lock "75bbbead-8628-448b-9484-19bf4ab922ff-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:25:31 user nova-compute[72351]: DEBUG nova.compute.manager [req-61ea7eda-a168-4724-b60a-7ccc4ae3b933 req-adde31a9-c20c-49c0-a652-4dbeb44a5e4c service nova] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] No waiting events found dispatching network-vif-plugged-be4ec483-8532-45f7-bb9f-0b50952ce94b {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:25:31 user nova-compute[72351]: WARNING nova.compute.manager [req-61ea7eda-a168-4724-b60a-7ccc4ae3b933 req-adde31a9-c20c-49c0-a652-4dbeb44a5e4c service nova] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Received unexpected event network-vif-plugged-be4ec483-8532-45f7-bb9f-0b50952ce94b for instance with vm_state active and task_state resize_finish.
Aug 09 02:25:31 user nova-compute[72351]: DEBUG nova.compute.manager [req-61ea7eda-a168-4724-b60a-7ccc4ae3b933 req-adde31a9-c20c-49c0-a652-4dbeb44a5e4c service nova] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Received event network-changed-52e1842a-aa45-4804-9636-f0cfe1134eee {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:25:31 user nova-compute[72351]: DEBUG nova.compute.manager [req-61ea7eda-a168-4724-b60a-7ccc4ae3b933 req-adde31a9-c20c-49c0-a652-4dbeb44a5e4c service nova] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Refreshing instance network info cache due to event network-changed-52e1842a-aa45-4804-9636-f0cfe1134eee. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:25:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-61ea7eda-a168-4724-b60a-7ccc4ae3b933 req-adde31a9-c20c-49c0-a652-4dbeb44a5e4c service nova] Acquiring lock "refresh_cache-47afce95-e5f7-4703-a99f-7ef8b28c4bd2" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:25:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-61ea7eda-a168-4724-b60a-7ccc4ae3b933 req-adde31a9-c20c-49c0-a652-4dbeb44a5e4c service nova] Acquired lock "refresh_cache-47afce95-e5f7-4703-a99f-7ef8b28c4bd2" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:25:31 user nova-compute[72351]: DEBUG nova.network.neutron [req-61ea7eda-a168-4724-b60a-7ccc4ae3b933 req-adde31a9-c20c-49c0-a652-4dbeb44a5e4c service nova] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Refreshing network info cache for port 52e1842a-aa45-4804-9636-f0cfe1134eee {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:25:31 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] During sync_power_state the instance has a pending task (resize_finish). Skip.
Aug 09 02:25:31 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:25:31 user nova-compute[72351]:   <uuid>47afce95-e5f7-4703-a99f-7ef8b28c4bd2</uuid>
Aug 09 02:25:31 user nova-compute[72351]:   <name>instance-0000001a</name>
Aug 09 02:25:31 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:25:31 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:25:31 user nova-compute[72351]:   <metadata>
Aug 09 02:25:31 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:25:31 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:25:31 user nova-compute[72351]:       <nova:name>tempest-ServersNegativeTestJSON-server-1699010745</nova:name>
Aug 09 02:25:31 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:25:31</nova:creationTime>
Aug 09 02:25:31 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:25:31 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:25:31 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:25:31 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:25:31 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:25:31 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:25:31 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:25:31 user nova-compute[72351]:       <nova:owner>
Aug 09 02:25:31 user nova-compute[72351]:         <nova:user uuid="ebc62bdcafdd4131be9abb7fe4ed2422">tempest-ServersNegativeTestJSON-1773070288-project-member</nova:user>
Aug 09 02:25:31 user nova-compute[72351]:         <nova:project uuid="eebc3a715ae04d48918b6c68b135935b">tempest-ServersNegativeTestJSON-1773070288</nova:project>
Aug 09 02:25:31 user nova-compute[72351]:       </nova:owner>
Aug 09 02:25:31 user nova-compute[72351]:       <nova:root type="image" uuid="18ccda7f-31d3-4978-ab9d-4ed2f74eff94"/>
Aug 09 02:25:31 user nova-compute[72351]:       <nova:ports>
Aug 09 02:25:31 user nova-compute[72351]:         <nova:port uuid="52e1842a-aa45-4804-9636-f0cfe1134eee">
Aug 09 02:25:31 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.6" ipVersion="4"/>
Aug 09 02:25:31 user nova-compute[72351]:         </nova:port>
Aug 09 02:25:31 user nova-compute[72351]:       </nova:ports>
Aug 09 02:25:31 user nova-compute[72351]:     </nova:instance>
Aug 09 02:25:31 user nova-compute[72351]:   </metadata>
Aug 09 02:25:31 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:25:31 user nova-compute[72351]:     <system>
Aug 09 02:25:31 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:25:31 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:25:31 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:25:31 user nova-compute[72351]:       <entry name="serial">47afce95-e5f7-4703-a99f-7ef8b28c4bd2</entry>
Aug 09 02:25:31 user nova-compute[72351]:       <entry name="uuid">47afce95-e5f7-4703-a99f-7ef8b28c4bd2</entry>
Aug 09 02:25:31 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:25:31 user nova-compute[72351]:     </system>
Aug 09 02:25:31 user nova-compute[72351]:   </sysinfo>
Aug 09 02:25:31 user nova-compute[72351]:   <os>
Aug 09 02:25:31 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:25:31 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:25:31 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:25:31 user nova-compute[72351]:   </os>
Aug 09 02:25:31 user nova-compute[72351]:   <features>
Aug 09 02:25:31 user nova-compute[72351]:     <acpi/>
Aug 09 02:25:31 user nova-compute[72351]:     <apic/>
Aug 09 02:25:31 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:25:31 user nova-compute[72351]:   </features>
Aug 09 02:25:31 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:25:31 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:25:31 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:25:31 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:25:31 user nova-compute[72351]:   </clock>
Aug 09 02:25:31 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:25:31 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:25:31 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:25:31 user nova-compute[72351]:   </cpu>
Aug 09 02:25:31 user nova-compute[72351]:   <devices>
Aug 09 02:25:31 user nova-compute[72351]:     <disk type="file" device="disk">
Aug 09 02:25:31 user nova-compute[72351]:       <driver name="qemu" type="qcow2" cache="none"/>
Aug 09 02:25:31 user nova-compute[72351]:       <source file="/opt/stack/data/nova/instances/47afce95-e5f7-4703-a99f-7ef8b28c4bd2/disk"/>
Aug 09 02:25:31 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:25:31 user nova-compute[72351]:     </disk>
Aug 09 02:25:31 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:25:31 user nova-compute[72351]:       <mac address="fa:16:3e:bd:df:05"/>
Aug 09 02:25:31 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:25:31 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:25:31 user nova-compute[72351]:       <target dev="tap52e1842a-aa"/>
Aug 09 02:25:31 user nova-compute[72351]:     </interface>
Aug 09 02:25:31 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:25:31 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/47afce95-e5f7-4703-a99f-7ef8b28c4bd2/console.log" append="off"/>
Aug 09 02:25:31 user nova-compute[72351]:     </serial>
Aug 09 02:25:31 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:25:31 user nova-compute[72351]:     <video>
Aug 09 02:25:31 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:25:31 user nova-compute[72351]:     </video>
Aug 09 02:25:31 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:25:31 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:25:31 user nova-compute[72351]:     </rng>
Aug 09 02:25:31 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:25:31 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:25:31 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:25:31 user nova-compute[72351]:     </memballoon>
Aug 09 02:25:31 user nova-compute[72351]:   </devices>
Aug 09 02:25:31 user nova-compute[72351]: </domain>
Aug 09 02:25:31 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:25:31 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:25:27Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServersNegativeTestJSON-server-1699010745',display_name='tempest-ServersNegativeTestJSON-server-1699010745',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serversnegativetestjson-server-1699010745',id=26,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='eebc3a715ae04d48918b6c68b135935b',ramdisk_id='',reservation_id='r-lovurafx',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServersNegativeTestJSON-1773070288',owner_user_name='tempest-ServersNegativeTestJSON-1773070288-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:25:29Z,user_data=None,user_id='ebc62bdcafdd4131be9abb7fe4ed2422',uuid=47afce95-e5f7-4703-a99f-7ef8b28c4bd2,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "52e1842a-aa45-4804-9636-f0cfe1134eee", "address": "fa:16:3e:bd:df:05", "network": {"id": "9785f84c-cf45-464e-9127-332ce46abd11", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-87568186-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "eebc3a715ae04d48918b6c68b135935b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52e1842a-aa", "ovs_interfaceid": "52e1842a-aa45-4804-9636-f0cfe1134eee", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:25:31 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Converting VIF {"id": "52e1842a-aa45-4804-9636-f0cfe1134eee", "address": "fa:16:3e:bd:df:05", "network": {"id": "9785f84c-cf45-464e-9127-332ce46abd11", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-87568186-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "eebc3a715ae04d48918b6c68b135935b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52e1842a-aa", "ovs_interfaceid": "52e1842a-aa45-4804-9636-f0cfe1134eee", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:25:31 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:bd:df:05,bridge_name='br-int',has_traffic_filtering=True,id=52e1842a-aa45-4804-9636-f0cfe1134eee,network=Network(9785f84c-cf45-464e-9127-332ce46abd11),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap52e1842a-aa') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:25:31 user nova-compute[72351]: DEBUG os_vif [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:bd:df:05,bridge_name='br-int',has_traffic_filtering=True,id=52e1842a-aa45-4804-9636-f0cfe1134eee,network=Network(9785f84c-cf45-464e-9127-332ce46abd11),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap52e1842a-aa') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:25:31 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:31 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:25:31 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:25:31 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:31 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap52e1842a-aa, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:25:31 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap52e1842a-aa, col_values=(('external_ids', {'iface-id': '52e1842a-aa45-4804-9636-f0cfe1134eee', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:bd:df:05', 'vm-uuid': '47afce95-e5f7-4703-a99f-7ef8b28c4bd2'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:25:31 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:31 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:25:31 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:31 user nova-compute[72351]: INFO os_vif [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:bd:df:05,bridge_name='br-int',has_traffic_filtering=True,id=52e1842a-aa45-4804-9636-f0cfe1134eee,network=Network(9785f84c-cf45-464e-9127-332ce46abd11),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap52e1842a-aa')
Aug 09 02:25:31 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:25:31 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] No VIF found with MAC fa:16:3e:bd:df:05, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:25:31 user nova-compute[72351]: DEBUG nova.network.neutron [req-61ea7eda-a168-4724-b60a-7ccc4ae3b933 req-adde31a9-c20c-49c0-a652-4dbeb44a5e4c service nova] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Updated VIF entry in instance network info cache for port 52e1842a-aa45-4804-9636-f0cfe1134eee. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:25:31 user nova-compute[72351]: DEBUG nova.network.neutron [req-61ea7eda-a168-4724-b60a-7ccc4ae3b933 req-adde31a9-c20c-49c0-a652-4dbeb44a5e4c service nova] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Updating instance_info_cache with network_info: [{"id": "52e1842a-aa45-4804-9636-f0cfe1134eee", "address": "fa:16:3e:bd:df:05", "network": {"id": "9785f84c-cf45-464e-9127-332ce46abd11", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-87568186-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "eebc3a715ae04d48918b6c68b135935b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52e1842a-aa", "ovs_interfaceid": "52e1842a-aa45-4804-9636-f0cfe1134eee", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:25:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-61ea7eda-a168-4724-b60a-7ccc4ae3b933 req-adde31a9-c20c-49c0-a652-4dbeb44a5e4c service nova] Releasing lock "refresh_cache-47afce95-e5f7-4703-a99f-7ef8b28c4bd2" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:25:32 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:32 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:32 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:32 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:32 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:33 user nova-compute[72351]: DEBUG nova.compute.manager [req-6ee45c72-6a82-4ebc-9d66-2d3ba7555516 req-5907a594-dafd-4c69-a0ff-6d0b04e55a74 service nova] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Received event network-vif-plugged-52e1842a-aa45-4804-9636-f0cfe1134eee {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:25:33 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-6ee45c72-6a82-4ebc-9d66-2d3ba7555516 req-5907a594-dafd-4c69-a0ff-6d0b04e55a74 service nova] Acquiring lock "47afce95-e5f7-4703-a99f-7ef8b28c4bd2-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:25:33 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-6ee45c72-6a82-4ebc-9d66-2d3ba7555516 req-5907a594-dafd-4c69-a0ff-6d0b04e55a74 service nova] Lock "47afce95-e5f7-4703-a99f-7ef8b28c4bd2-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:25:33 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-6ee45c72-6a82-4ebc-9d66-2d3ba7555516 req-5907a594-dafd-4c69-a0ff-6d0b04e55a74 service nova] Lock "47afce95-e5f7-4703-a99f-7ef8b28c4bd2-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:25:33 user nova-compute[72351]: DEBUG nova.compute.manager [req-6ee45c72-6a82-4ebc-9d66-2d3ba7555516 req-5907a594-dafd-4c69-a0ff-6d0b04e55a74 service nova] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] No waiting events found dispatching network-vif-plugged-52e1842a-aa45-4804-9636-f0cfe1134eee {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:25:33 user nova-compute[72351]: WARNING nova.compute.manager [req-6ee45c72-6a82-4ebc-9d66-2d3ba7555516 req-5907a594-dafd-4c69-a0ff-6d0b04e55a74 service nova] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Received unexpected event network-vif-plugged-52e1842a-aa45-4804-9636-f0cfe1134eee for instance with vm_state building and task_state spawning.
Aug 09 02:25:33 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:33 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:33 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:33 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:35 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547935.1313975, 47afce95-e5f7-4703-a99f-7ef8b28c4bd2 => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:25:35 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] VM Resumed (Lifecycle Event)
Aug 09 02:25:35 user nova-compute[72351]: DEBUG nova.compute.manager [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:25:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Guest created on hypervisor {{(pid=72351) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4411}}
Aug 09 02:25:35 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Instance spawned successfully.
Aug 09 02:25:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:894}}
Aug 09 02:25:35 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:25:35 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:25:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Found default for hw_cdrom_bus of ide {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:25:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Found default for hw_disk_bus of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:25:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Found default for hw_input_bus of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:25:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Found default for hw_pointer_model of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:25:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Found default for hw_video_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:25:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Found default for hw_vif_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:25:35 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:25:35 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547935.1351516, 47afce95-e5f7-4703-a99f-7ef8b28c4bd2 => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:25:35 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] VM Started (Lifecycle Event)
Aug 09 02:25:35 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:25:35 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:25:35 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:25:35 user nova-compute[72351]: INFO nova.compute.manager [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Took 6.02 seconds to spawn the instance on the hypervisor.
Aug 09 02:25:35 user nova-compute[72351]: DEBUG nova.compute.manager [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:25:35 user nova-compute[72351]: DEBUG nova.compute.manager [req-684dd782-9548-41df-84f4-7cb1c2210e6e req-8c785b60-901c-4483-a394-665a4ecc0b6a service nova] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Received event network-vif-plugged-52e1842a-aa45-4804-9636-f0cfe1134eee {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:25:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-684dd782-9548-41df-84f4-7cb1c2210e6e req-8c785b60-901c-4483-a394-665a4ecc0b6a service nova] Acquiring lock "47afce95-e5f7-4703-a99f-7ef8b28c4bd2-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:25:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-684dd782-9548-41df-84f4-7cb1c2210e6e req-8c785b60-901c-4483-a394-665a4ecc0b6a service nova] Lock "47afce95-e5f7-4703-a99f-7ef8b28c4bd2-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:25:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-684dd782-9548-41df-84f4-7cb1c2210e6e req-8c785b60-901c-4483-a394-665a4ecc0b6a service nova] Lock "47afce95-e5f7-4703-a99f-7ef8b28c4bd2-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:25:35 user nova-compute[72351]: DEBUG nova.compute.manager [req-684dd782-9548-41df-84f4-7cb1c2210e6e req-8c785b60-901c-4483-a394-665a4ecc0b6a service nova] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] No waiting events found dispatching network-vif-plugged-52e1842a-aa45-4804-9636-f0cfe1134eee {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:25:35 user nova-compute[72351]: WARNING nova.compute.manager [req-684dd782-9548-41df-84f4-7cb1c2210e6e req-8c785b60-901c-4483-a394-665a4ecc0b6a service nova] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Received unexpected event network-vif-plugged-52e1842a-aa45-4804-9636-f0cfe1134eee for instance with vm_state active and task_state None.
Aug 09 02:25:35 user nova-compute[72351]: INFO nova.compute.manager [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Took 6.90 seconds to build instance.
Aug 09 02:25:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-23629dae-f131-43aa-9ca0-a7d12908c143 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Lock "47afce95-e5f7-4703-a99f-7ef8b28c4bd2" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 6.992s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:25:36 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:37 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:41 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:42 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:42 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:46 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:47 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:48 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:51 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:51 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:52 user nova-compute[72351]: DEBUG nova.compute.manager [req-f10c0a93-f0f5-4a0d-b7b1-12144ce9b9fb req-a305e816-e6cf-4fd7-8719-a8e25d6f7edf service nova] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Received event network-changed-f0429a54-9d4b-4bf9-be97-1dafb19ebc0b {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:25:52 user nova-compute[72351]: DEBUG nova.compute.manager [req-f10c0a93-f0f5-4a0d-b7b1-12144ce9b9fb req-a305e816-e6cf-4fd7-8719-a8e25d6f7edf service nova] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Refreshing instance network info cache due to event network-changed-f0429a54-9d4b-4bf9-be97-1dafb19ebc0b. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:25:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-f10c0a93-f0f5-4a0d-b7b1-12144ce9b9fb req-a305e816-e6cf-4fd7-8719-a8e25d6f7edf service nova] Acquiring lock "refresh_cache-5c71284c-df41-4990-9a46-7986982bf7e2" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:25:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-f10c0a93-f0f5-4a0d-b7b1-12144ce9b9fb req-a305e816-e6cf-4fd7-8719-a8e25d6f7edf service nova] Acquired lock "refresh_cache-5c71284c-df41-4990-9a46-7986982bf7e2" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:25:52 user nova-compute[72351]: DEBUG nova.network.neutron [req-f10c0a93-f0f5-4a0d-b7b1-12144ce9b9fb req-a305e816-e6cf-4fd7-8719-a8e25d6f7edf service nova] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Refreshing network info cache for port f0429a54-9d4b-4bf9-be97-1dafb19ebc0b {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:25:52 user nova-compute[72351]: DEBUG nova.network.neutron [req-f10c0a93-f0f5-4a0d-b7b1-12144ce9b9fb req-a305e816-e6cf-4fd7-8719-a8e25d6f7edf service nova] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Updated VIF entry in instance network info cache for port f0429a54-9d4b-4bf9-be97-1dafb19ebc0b. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:25:52 user nova-compute[72351]: DEBUG nova.network.neutron [req-f10c0a93-f0f5-4a0d-b7b1-12144ce9b9fb req-a305e816-e6cf-4fd7-8719-a8e25d6f7edf service nova] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Updating instance_info_cache with network_info: [{"id": "f0429a54-9d4b-4bf9-be97-1dafb19ebc0b", "address": "fa:16:3e:78:c7:55", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.236", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf0429a54-9d", "ovs_interfaceid": "f0429a54-9d4b-4bf9-be97-1dafb19ebc0b", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:25:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-f10c0a93-f0f5-4a0d-b7b1-12144ce9b9fb req-a305e816-e6cf-4fd7-8719-a8e25d6f7edf service nova] Releasing lock "refresh_cache-5c71284c-df41-4990-9a46-7986982bf7e2" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:25:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c6b9dfcd-b044-4ac8-bfa2-5261ebe7224c tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquiring lock "5c71284c-df41-4990-9a46-7986982bf7e2" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:25:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c6b9dfcd-b044-4ac8-bfa2-5261ebe7224c tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "5c71284c-df41-4990-9a46-7986982bf7e2" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:25:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c6b9dfcd-b044-4ac8-bfa2-5261ebe7224c tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquiring lock "5c71284c-df41-4990-9a46-7986982bf7e2-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:25:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c6b9dfcd-b044-4ac8-bfa2-5261ebe7224c tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "5c71284c-df41-4990-9a46-7986982bf7e2-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:25:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c6b9dfcd-b044-4ac8-bfa2-5261ebe7224c tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "5c71284c-df41-4990-9a46-7986982bf7e2-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:25:52 user nova-compute[72351]: INFO nova.compute.manager [None req-c6b9dfcd-b044-4ac8-bfa2-5261ebe7224c tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Terminating instance
Aug 09 02:25:52 user nova-compute[72351]: DEBUG nova.compute.manager [None req-c6b9dfcd-b044-4ac8-bfa2-5261ebe7224c tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Start destroying the instance on the hypervisor. {{(pid=72351) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3124}}
Aug 09 02:25:52 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:52 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:52 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:53 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Instance destroyed successfully.
Aug 09 02:25:53 user nova-compute[72351]: DEBUG nova.objects.instance [None req-c6b9dfcd-b044-4ac8-bfa2-5261ebe7224c tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lazy-loading 'resources' on Instance uuid 5c71284c-df41-4990-9a46-7986982bf7e2 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:25:53 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-c6b9dfcd-b044-4ac8-bfa2-5261ebe7224c tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:23:59Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-image-snapshot-server-291879344',display_name='tempest-TestVolumeBootPattern-image-snapshot-server-291879344',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testvolumebootpattern-image-snapshot-server-291879344',id=21,image_ref='7869c8d4-caa3-435c-a4e4-e01eb5198e6f',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMigRZU/OzbuGai7a1EcqwThXhvudiR9/Q8Da5xVMuKq+4DVrTM+XekLmyXUIzTgyNFsg9BxRHjyOIIoT5ZbBndJV38K0BzyxRl30KiADrnKDbDnCylyfZZ9qqf+nevzqg==',key_name='tempest-keypair-2017099597',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:24:23Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='a4c6bbfd8d904ba1ac69ba5da4564513',ramdisk_id='',reservation_id='r-caoazglt',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_bdm_v2='True',image_boot_roles='reader,member',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_project_name='tempest-TestVolumeBootPattern-350743460',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',image_owner_user_name='tempest-TestVolumeBootPattern-350743460-project-member',image_root_device_name='/dev/vda',owner_project_name='tempest-TestVolumeBootPattern-350743460',owner_user_name='tempest-TestVolumeBootPattern-350743460-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:24:23Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='07d8d5a634274df19c2db769d2d27b29',uuid=5c71284c-df41-4990-9a46-7986982bf7e2,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "f0429a54-9d4b-4bf9-be97-1dafb19ebc0b", "address": "fa:16:3e:78:c7:55", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.236", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf0429a54-9d", "ovs_interfaceid": "f0429a54-9d4b-4bf9-be97-1dafb19ebc0b", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:25:53 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-c6b9dfcd-b044-4ac8-bfa2-5261ebe7224c tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Converting VIF {"id": "f0429a54-9d4b-4bf9-be97-1dafb19ebc0b", "address": "fa:16:3e:78:c7:55", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.236", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf0429a54-9d", "ovs_interfaceid": "f0429a54-9d4b-4bf9-be97-1dafb19ebc0b", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:25:53 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-c6b9dfcd-b044-4ac8-bfa2-5261ebe7224c tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:78:c7:55,bridge_name='br-int',has_traffic_filtering=True,id=f0429a54-9d4b-4bf9-be97-1dafb19ebc0b,network=Network(5ef428cf-e408-490c-8f60-70934f760f4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf0429a54-9d') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:25:53 user nova-compute[72351]: DEBUG os_vif [None req-c6b9dfcd-b044-4ac8-bfa2-5261ebe7224c tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:78:c7:55,bridge_name='br-int',has_traffic_filtering=True,id=f0429a54-9d4b-4bf9-be97-1dafb19ebc0b,network=Network(5ef428cf-e408-490c-8f60-70934f760f4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf0429a54-9d') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:25:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapf0429a54-9d, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:25:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:53 user nova-compute[72351]: INFO os_vif [None req-c6b9dfcd-b044-4ac8-bfa2-5261ebe7224c tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:78:c7:55,bridge_name='br-int',has_traffic_filtering=True,id=f0429a54-9d4b-4bf9-be97-1dafb19ebc0b,network=Network(5ef428cf-e408-490c-8f60-70934f760f4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf0429a54-9d')
Aug 09 02:25:53 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-c6b9dfcd-b044-4ac8-bfa2-5261ebe7224c tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 02:25:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c6b9dfcd-b044-4ac8-bfa2-5261ebe7224c tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'lfCTruNe', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb'}, None), 'kwargs': True}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:25:53 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-c6b9dfcd-b044-4ac8-bfa2-5261ebe7224c tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:25:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-c6b9dfcd-b044-4ac8-bfa2-5261ebe7224c tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:25:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-c6b9dfcd-b044-4ac8-bfa2-5261ebe7224c tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:25:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c6b9dfcd-b044-4ac8-bfa2-5261ebe7224c tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 02:25:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:53 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e41d8937-fb28-485f-8869-8dbe3fb208f7]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:25:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.026s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:53 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[63161e6a-9105-4203-a478-d5a0812bc124]: (4, ('tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)\ntcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065 (non-flash)\ntcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:25:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c6b9dfcd-b044-4ac8-bfa2-5261ebe7224c tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)
Aug 09 02:25:53 user nova-compute[72351]: tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065 (non-flash)
Aug 09 02:25:53 user nova-compute[72351]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:25:53 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:25:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c6b9dfcd-b044-4ac8-bfa2-5261ebe7224c tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsi session list stdout=tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)
Aug 09 02:25:53 user nova-compute[72351]: tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065 (non-flash)
Aug 09 02:25:53 user nova-compute[72351]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:25:53 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:25:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c6b9dfcd-b044-4ac8-bfa2-5261ebe7224c tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fd00b3d48b0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065'): ({'sdb'}, set())}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 02:25:53 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-c6b9dfcd-b044-4ac8-bfa2-5261ebe7224c tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Removing single pathed devices sdb {{(pid=72351) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Aug 09 02:25:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:53 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[20d9678d-750a-4651-8bf2-0d8815e31af9]: (4, ('path checker states:\nup                  3\n\npaths: 0\nbusy: False\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:25:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:53 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[34fbfe93-6e38-4b6e-a43d-5bd7556a9b84]: (4, ('ok\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:25:53 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-c6b9dfcd-b044-4ac8-bfa2-5261ebe7224c tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Flushing IO for device /dev/sdb {{(pid=72351) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Aug 09 02:25:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.014s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:53 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[dc370165-5bbc-48d5-aaf9-60bc08b14e48]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:25:53 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-c6b9dfcd-b044-4ac8-bfa2-5261ebe7224c tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=72351) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Aug 09 02:25:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.053s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:53 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[030c03dc-cc5e-473d-bc95-5b9187cbc92a]: (4, ('1', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:25:53 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-c6b9dfcd-b044-4ac8-bfa2-5261ebe7224c tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Aug 09 02:25:53 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-c6b9dfcd-b044-4ac8-bfa2-5261ebe7224c tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] SCSI volumes sdb have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Aug 09 02:25:53 user nova-compute[72351]: DEBUG os_brick.privileged.rootwrap [-] Unlinking ('/dev/disk/by-id/scsi-1SCST_BIO56118318-4cd24d0533cd5379', '/dev/disk/by-id/scsi-SSCST_BIO_Storage_56118318', '/dev/disk/by-id/scsi-23463643234643035') {{(pid=83366) unlink_root /usr/local/lib/python3.10/dist-packages/os_brick/privileged/rootwrap.py:219}}
Aug 09 02:25:53 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[b702651b-5eb6-4065-b024-c86f78cffb96]: (4, None) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:25:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c6b9dfcd-b044-4ac8-bfa2-5261ebe7224c tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065')] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 02:25:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:53 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[227a3168-43a8-432f-a3a1-e4ac45d5f6ec]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:25:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c6b9dfcd-b044-4ac8-bfa2-5261ebe7224c tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:25:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065 -p 172.16.0.220:3260 --logout {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065 -p 172.16.0.220:3260 --logout" returned: 0 in 0.069s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:53 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[f3a8f06d-14d8-42b4-b74e-dfe7dfe62975]: (4, ('Logging out of session [sid: 19, target: iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065, portal: 172.16.0.220,3260]\nLogout of [sid: 19, target: iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:25:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c6b9dfcd-b044-4ac8-bfa2-5261ebe7224c tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 19, target: iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065, portal: 172.16.0.220,3260]
Aug 09 02:25:53 user nova-compute[72351]: Logout of [sid: 19, target: iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065, portal: 172.16.0.220,3260] successful.
Aug 09 02:25:53 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:25:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065 -p 172.16.0.220:3260 --op delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c67f9b74-7547-4996-abf6-678dac1ff065 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:53 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a345551f-d4ae-4ab1-9137-961b1cb1c315]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:25:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c6b9dfcd-b044-4ac8-bfa2-5261ebe7224c tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:25:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-c6b9dfcd-b044-4ac8-bfa2-5261ebe7224c tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.307s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:25:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c6b9dfcd-b044-4ac8-bfa2-5261ebe7224c tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] <== disconnect_volume: return (310ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:25:53 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-c6b9dfcd-b044-4ac8-bfa2-5261ebe7224c tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 02:25:53 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-c6b9dfcd-b044-4ac8-bfa2-5261ebe7224c tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Deleting instance files /opt/stack/data/nova/instances/5c71284c-df41-4990-9a46-7986982bf7e2_del
Aug 09 02:25:53 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-c6b9dfcd-b044-4ac8-bfa2-5261ebe7224c tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Deletion of /opt/stack/data/nova/instances/5c71284c-df41-4990-9a46-7986982bf7e2_del complete
Aug 09 02:25:54 user nova-compute[72351]: INFO nova.compute.manager [None req-c6b9dfcd-b044-4ac8-bfa2-5261ebe7224c tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Took 1.27 seconds to destroy the instance on the hypervisor.
Aug 09 02:25:54 user nova-compute[72351]: DEBUG oslo.service.loopingcall [None req-c6b9dfcd-b044-4ac8-bfa2-5261ebe7224c tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72351) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Aug 09 02:25:54 user nova-compute[72351]: DEBUG nova.compute.manager [-] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Deallocating network for instance {{(pid=72351) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2263}}
Aug 09 02:25:54 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] deallocate_for_instance() {{(pid=72351) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1795}}
Aug 09 02:25:54 user nova-compute[72351]: DEBUG nova.compute.manager [req-671b0ca6-dd0e-40bb-af6a-2e10fd094004 req-0aa90819-ae31-4d8a-8a8d-feb049b73d83 service nova] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Received event network-vif-unplugged-f0429a54-9d4b-4bf9-be97-1dafb19ebc0b {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:25:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-671b0ca6-dd0e-40bb-af6a-2e10fd094004 req-0aa90819-ae31-4d8a-8a8d-feb049b73d83 service nova] Acquiring lock "5c71284c-df41-4990-9a46-7986982bf7e2-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:25:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-671b0ca6-dd0e-40bb-af6a-2e10fd094004 req-0aa90819-ae31-4d8a-8a8d-feb049b73d83 service nova] Lock "5c71284c-df41-4990-9a46-7986982bf7e2-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:25:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-671b0ca6-dd0e-40bb-af6a-2e10fd094004 req-0aa90819-ae31-4d8a-8a8d-feb049b73d83 service nova] Lock "5c71284c-df41-4990-9a46-7986982bf7e2-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:25:54 user nova-compute[72351]: DEBUG nova.compute.manager [req-671b0ca6-dd0e-40bb-af6a-2e10fd094004 req-0aa90819-ae31-4d8a-8a8d-feb049b73d83 service nova] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] No waiting events found dispatching network-vif-unplugged-f0429a54-9d4b-4bf9-be97-1dafb19ebc0b {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:25:54 user nova-compute[72351]: DEBUG nova.compute.manager [req-671b0ca6-dd0e-40bb-af6a-2e10fd094004 req-0aa90819-ae31-4d8a-8a8d-feb049b73d83 service nova] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Received event network-vif-unplugged-f0429a54-9d4b-4bf9-be97-1dafb19ebc0b for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:25:54 user nova-compute[72351]: DEBUG nova.compute.manager [req-671b0ca6-dd0e-40bb-af6a-2e10fd094004 req-0aa90819-ae31-4d8a-8a8d-feb049b73d83 service nova] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Received event network-vif-plugged-f0429a54-9d4b-4bf9-be97-1dafb19ebc0b {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:25:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-671b0ca6-dd0e-40bb-af6a-2e10fd094004 req-0aa90819-ae31-4d8a-8a8d-feb049b73d83 service nova] Acquiring lock "5c71284c-df41-4990-9a46-7986982bf7e2-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:25:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-671b0ca6-dd0e-40bb-af6a-2e10fd094004 req-0aa90819-ae31-4d8a-8a8d-feb049b73d83 service nova] Lock "5c71284c-df41-4990-9a46-7986982bf7e2-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:25:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-671b0ca6-dd0e-40bb-af6a-2e10fd094004 req-0aa90819-ae31-4d8a-8a8d-feb049b73d83 service nova] Lock "5c71284c-df41-4990-9a46-7986982bf7e2-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:25:54 user nova-compute[72351]: DEBUG nova.compute.manager [req-671b0ca6-dd0e-40bb-af6a-2e10fd094004 req-0aa90819-ae31-4d8a-8a8d-feb049b73d83 service nova] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] No waiting events found dispatching network-vif-plugged-f0429a54-9d4b-4bf9-be97-1dafb19ebc0b {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:25:54 user nova-compute[72351]: WARNING nova.compute.manager [req-671b0ca6-dd0e-40bb-af6a-2e10fd094004 req-0aa90819-ae31-4d8a-8a8d-feb049b73d83 service nova] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Received unexpected event network-vif-plugged-f0429a54-9d4b-4bf9-be97-1dafb19ebc0b for instance with vm_state active and task_state deleting.
Aug 09 02:25:54 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:25:54 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Took 0.83 seconds to deallocate network for instance.
Aug 09 02:25:54 user nova-compute[72351]: DEBUG nova.compute.manager [req-2142fcf5-ddb7-4cb0-9c72-94a1b7b3f922 req-9c55d0fe-548e-4e63-b988-3fac11d8ab63 service nova] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Received event network-vif-deleted-f0429a54-9d4b-4bf9-be97-1dafb19ebc0b {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:25:54 user nova-compute[72351]: INFO nova.compute.manager [req-2142fcf5-ddb7-4cb0-9c72-94a1b7b3f922 req-9c55d0fe-548e-4e63-b988-3fac11d8ab63 service nova] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Neutron deleted interface f0429a54-9d4b-4bf9-be97-1dafb19ebc0b; detaching it from the instance and deleting it from the info cache
Aug 09 02:25:54 user nova-compute[72351]: DEBUG nova.network.neutron [req-2142fcf5-ddb7-4cb0-9c72-94a1b7b3f922 req-9c55d0fe-548e-4e63-b988-3fac11d8ab63 service nova] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:25:54 user nova-compute[72351]: DEBUG nova.compute.manager [req-2142fcf5-ddb7-4cb0-9c72-94a1b7b3f922 req-9c55d0fe-548e-4e63-b988-3fac11d8ab63 service nova] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Detach interface failed, port_id=f0429a54-9d4b-4bf9-be97-1dafb19ebc0b, reason: Instance 5c71284c-df41-4990-9a46-7986982bf7e2 could not be found. {{(pid=72351) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10866}}
Aug 09 02:25:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Acquiring lock "16c1e558-baa1-4c2d-b13b-50585f74d587" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:25:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Lock "16c1e558-baa1-4c2d-b13b-50585f74d587" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:25:55 user nova-compute[72351]: DEBUG nova.compute.manager [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Starting instance... {{(pid=72351) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2406}}
Aug 09 02:25:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:25:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:25:55 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72351) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Aug 09 02:25:55 user nova-compute[72351]: INFO nova.compute.claims [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Claim successful on node user
Aug 09 02:25:55 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:25:55 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:25:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.599s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:25:55 user nova-compute[72351]: DEBUG nova.compute.manager [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Start building networks asynchronously for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2803}}
Aug 09 02:25:55 user nova-compute[72351]: DEBUG nova.compute.manager [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Allocating IP information in the background. {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1956}}
Aug 09 02:25:55 user nova-compute[72351]: DEBUG nova.network.neutron [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] allocate_for_instance() {{(pid=72351) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Aug 09 02:25:55 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Aug 09 02:25:55 user nova-compute[72351]: DEBUG nova.compute.manager [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Start building block device mappings for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2838}}
Aug 09 02:25:55 user nova-compute[72351]: INFO nova.compute.manager [None req-c6b9dfcd-b044-4ac8-bfa2-5261ebe7224c tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Took 1.01 seconds to detach 1 volumes for instance.
Aug 09 02:25:55 user nova-compute[72351]: DEBUG nova.compute.manager [None req-c6b9dfcd-b044-4ac8-bfa2-5261ebe7224c tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Deleting volume: c67f9b74-7547-4996-abf6-678dac1ff065 {{(pid=72351) _cleanup_volumes /opt/stack/nova/nova/compute/manager.py:3221}}
Aug 09 02:25:55 user nova-compute[72351]: DEBUG nova.policy [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'fd71797160d14dfbba2c65eca2e2f694', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de4a7655f2604c8a9c7964cfd298be37', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72351) authorize /opt/stack/nova/nova/policy.py:203}}
Aug 09 02:25:55 user nova-compute[72351]: DEBUG nova.compute.manager [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Start spawning the instance on the hypervisor. {{(pid=72351) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2612}}
Aug 09 02:25:55 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Creating instance directory {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4717}}
Aug 09 02:25:55 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Creating image(s)
Aug 09 02:25:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Acquiring lock "/opt/stack/data/nova/instances/16c1e558-baa1-4c2d-b13b-50585f74d587/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:25:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Lock "/opt/stack/data/nova/instances/16c1e558-baa1-4c2d-b13b-50585f74d587/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:25:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Lock "/opt/stack/data/nova/instances/16c1e558-baa1-4c2d-b13b-50585f74d587/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:25:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:56 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c6b9dfcd-b044-4ac8-bfa2-5261ebe7224c tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:25:56 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c6b9dfcd-b044-4ac8-bfa2-5261ebe7224c tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:25:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.140s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:56 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Acquiring lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:25:56 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:25:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.150s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/16c1e558-baa1-4c2d-b13b-50585f74d587/disk 1073741824 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/16c1e558-baa1-4c2d-b13b-50585f74d587/disk 1073741824" returned: 0 in 0.062s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:56 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.219s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:25:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.134s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:56 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Checking if we can resize image /opt/stack/data/nova/instances/16c1e558-baa1-4c2d-b13b-50585f74d587/disk. size=1073741824 {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Aug 09 02:25:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/16c1e558-baa1-4c2d-b13b-50585f74d587/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:56 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-c6b9dfcd-b044-4ac8-bfa2-5261ebe7224c tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:25:56 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-c6b9dfcd-b044-4ac8-bfa2-5261ebe7224c tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:25:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/16c1e558-baa1-4c2d-b13b-50585f74d587/disk --force-share --output=json" returned: 0 in 0.130s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:56 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Cannot resize image /opt/stack/data/nova/instances/16c1e558-baa1-4c2d-b13b-50585f74d587/disk to a smaller size. {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Aug 09 02:25:56 user nova-compute[72351]: DEBUG nova.objects.instance [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Lazy-loading 'migration_context' on Instance uuid 16c1e558-baa1-4c2d-b13b-50585f74d587 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:25:56 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c6b9dfcd-b044-4ac8-bfa2-5261ebe7224c tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.589s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:25:56 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Created local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4851}}
Aug 09 02:25:56 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Ensure instance console log exists: /opt/stack/data/nova/instances/16c1e558-baa1-4c2d-b13b-50585f74d587/console.log {{(pid=72351) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4603}}
Aug 09 02:25:56 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:25:56 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:25:56 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:25:56 user nova-compute[72351]: INFO nova.scheduler.client.report [None req-c6b9dfcd-b044-4ac8-bfa2-5261ebe7224c tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Deleted allocations for instance 5c71284c-df41-4990-9a46-7986982bf7e2
Aug 09 02:25:56 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c6b9dfcd-b044-4ac8-bfa2-5261ebe7224c tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "5c71284c-df41-4990-9a46-7986982bf7e2" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 4.012s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:25:56 user nova-compute[72351]: DEBUG nova.network.neutron [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Successfully created port: fb1b25f9-3a24-43df-805c-9fffe7cd74f7 {{(pid=72351) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Aug 09 02:25:57 user nova-compute[72351]: DEBUG nova.network.neutron [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Successfully updated port: fb1b25f9-3a24-43df-805c-9fffe7cd74f7 {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:25:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Acquiring lock "refresh_cache-16c1e558-baa1-4c2d-b13b-50585f74d587" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:25:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Acquired lock "refresh_cache-16c1e558-baa1-4c2d-b13b-50585f74d587" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:25:57 user nova-compute[72351]: DEBUG nova.network.neutron [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:25:57 user nova-compute[72351]: DEBUG nova.compute.manager [req-230b2e85-fff9-4e2a-913e-7748905952ec req-6f8260a9-5fe8-4b93-bf62-735e8f24d40d service nova] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Received event network-changed-fb1b25f9-3a24-43df-805c-9fffe7cd74f7 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:25:57 user nova-compute[72351]: DEBUG nova.compute.manager [req-230b2e85-fff9-4e2a-913e-7748905952ec req-6f8260a9-5fe8-4b93-bf62-735e8f24d40d service nova] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Refreshing instance network info cache due to event network-changed-fb1b25f9-3a24-43df-805c-9fffe7cd74f7. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:25:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-230b2e85-fff9-4e2a-913e-7748905952ec req-6f8260a9-5fe8-4b93-bf62-735e8f24d40d service nova] Acquiring lock "refresh_cache-16c1e558-baa1-4c2d-b13b-50585f74d587" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:25:57 user nova-compute[72351]: DEBUG nova.network.neutron [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:25:57 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:25:57 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG nova.compute.manager [req-d6e4b215-deb0-48a6-bbf6-c73ace4e1b95 req-1e7dcc5a-a046-4233-9ad8-0d02d5e66456 service nova] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Received event network-changed-7f4f4044-2549-41b0-b1b8-9154419a8b54 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG nova.compute.manager [req-d6e4b215-deb0-48a6-bbf6-c73ace4e1b95 req-1e7dcc5a-a046-4233-9ad8-0d02d5e66456 service nova] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Refreshing instance network info cache due to event network-changed-7f4f4044-2549-41b0-b1b8-9154419a8b54. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d6e4b215-deb0-48a6-bbf6-c73ace4e1b95 req-1e7dcc5a-a046-4233-9ad8-0d02d5e66456 service nova] Acquiring lock "refresh_cache-433e6729-fdd4-4467-971c-08ca962d887d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d6e4b215-deb0-48a6-bbf6-c73ace4e1b95 req-1e7dcc5a-a046-4233-9ad8-0d02d5e66456 service nova] Acquired lock "refresh_cache-433e6729-fdd4-4467-971c-08ca962d887d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG nova.network.neutron [req-d6e4b215-deb0-48a6-bbf6-c73ace4e1b95 req-1e7dcc5a-a046-4233-9ad8-0d02d5e66456 service nova] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Refreshing network info cache for port 7f4f4044-2549-41b0-b1b8-9154419a8b54 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG nova.network.neutron [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Updating instance_info_cache with network_info: [{"id": "fb1b25f9-3a24-43df-805c-9fffe7cd74f7", "address": "fa:16:3e:66:3c:d2", "network": {"id": "39ddd1d4-5757-425b-92e8-a6a5d8ff29db", "bridge": "br-int", "label": "tempest-TestServerVolumeAttachmentScenario-2122827433-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "de4a7655f2604c8a9c7964cfd298be37", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfb1b25f9-3a", "ovs_interfaceid": "fb1b25f9-3a24-43df-805c-9fffe7cd74f7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Releasing lock "refresh_cache-16c1e558-baa1-4c2d-b13b-50585f74d587" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG nova.compute.manager [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Instance network_info: |[{"id": "fb1b25f9-3a24-43df-805c-9fffe7cd74f7", "address": "fa:16:3e:66:3c:d2", "network": {"id": "39ddd1d4-5757-425b-92e8-a6a5d8ff29db", "bridge": "br-int", "label": "tempest-TestServerVolumeAttachmentScenario-2122827433-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "de4a7655f2604c8a9c7964cfd298be37", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfb1b25f9-3a", "ovs_interfaceid": "fb1b25f9-3a24-43df-805c-9fffe7cd74f7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1971}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-230b2e85-fff9-4e2a-913e-7748905952ec req-6f8260a9-5fe8-4b93-bf62-735e8f24d40d service nova] Acquired lock "refresh_cache-16c1e558-baa1-4c2d-b13b-50585f74d587" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG nova.network.neutron [req-230b2e85-fff9-4e2a-913e-7748905952ec req-6f8260a9-5fe8-4b93-bf62-735e8f24d40d service nova] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Refreshing network info cache for port fb1b25f9-3a24-43df-805c-9fffe7cd74f7 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Start _get_guest_xml network_info=[{"id": "fb1b25f9-3a24-43df-805c-9fffe7cd74f7", "address": "fa:16:3e:66:3c:d2", "network": {"id": "39ddd1d4-5757-425b-92e8-a6a5d8ff29db", "bridge": "br-int", "label": "tempest-TestServerVolumeAttachmentScenario-2122827433-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "de4a7655f2604c8a9c7964cfd298be37", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfb1b25f9-3a", "ovs_interfaceid": "fb1b25f9-3a24-43df-805c-9fffe7cd74f7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'size': 0, 'device_type': 'disk', 'encryption_secret_uuid': None, 'guest_format': None, 'encryption_options': None, 'disk_bus': 'virtio', 'encrypted': False, 'encryption_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'image_id': '18ccda7f-31d3-4978-ab9d-4ed2f74eff94'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:25:58 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:25:58 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:25:58 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:25:54Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestServerVolumeAttachmentScenario-server-360751817',display_name='tempest-TestServerVolumeAttachmentScenario-server-360751817',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testservervolumeattachmentscenario-server-360751817',id=27,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBO5LKDWUPul31VlDBgVZo5WIbldUiPko7qY2o+u8Xw9ekIjzjjzrKBVPG8SI+m7Xh+/4dSwSjsie+3WOYIYW29h/ArzglDh0AzoC+h3SECFUWbPsf8al1tugMF9vK3vduA==',key_name='tempest-keypair-148936292',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='de4a7655f2604c8a9c7964cfd298be37',ramdisk_id='',reservation_id='r-s9d31yuy',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestServerVolumeAttachmentScenario-1098806376',owner_user_name='tempest-TestServerVolumeAttachmentScenario-1098806376-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:25:56Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='fd71797160d14dfbba2c65eca2e2f694',uuid=16c1e558-baa1-4c2d-b13b-50585f74d587,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "fb1b25f9-3a24-43df-805c-9fffe7cd74f7", "address": "fa:16:3e:66:3c:d2", "network": {"id": "39ddd1d4-5757-425b-92e8-a6a5d8ff29db", "bridge": "br-int", "label": "tempest-TestServerVolumeAttachmentScenario-2122827433-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "de4a7655f2604c8a9c7964cfd298be37", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfb1b25f9-3a", "ovs_interfaceid": "fb1b25f9-3a24-43df-805c-9fffe7cd74f7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Converting VIF {"id": "fb1b25f9-3a24-43df-805c-9fffe7cd74f7", "address": "fa:16:3e:66:3c:d2", "network": {"id": "39ddd1d4-5757-425b-92e8-a6a5d8ff29db", "bridge": "br-int", "label": "tempest-TestServerVolumeAttachmentScenario-2122827433-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "de4a7655f2604c8a9c7964cfd298be37", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfb1b25f9-3a", "ovs_interfaceid": "fb1b25f9-3a24-43df-805c-9fffe7cd74f7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:66:3c:d2,bridge_name='br-int',has_traffic_filtering=True,id=fb1b25f9-3a24-43df-805c-9fffe7cd74f7,network=Network(39ddd1d4-5757-425b-92e8-a6a5d8ff29db),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfb1b25f9-3a') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG nova.objects.instance [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Lazy-loading 'pci_devices' on Instance uuid 16c1e558-baa1-4c2d-b13b-50585f74d587 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:25:58 user nova-compute[72351]:   <uuid>16c1e558-baa1-4c2d-b13b-50585f74d587</uuid>
Aug 09 02:25:58 user nova-compute[72351]:   <name>instance-0000001b</name>
Aug 09 02:25:58 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:25:58 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:25:58 user nova-compute[72351]:   <metadata>
Aug 09 02:25:58 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:25:58 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:25:58 user nova-compute[72351]:       <nova:name>tempest-TestServerVolumeAttachmentScenario-server-360751817</nova:name>
Aug 09 02:25:58 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:25:58</nova:creationTime>
Aug 09 02:25:58 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:25:58 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:25:58 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:25:58 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:25:58 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:25:58 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:25:58 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:25:58 user nova-compute[72351]:       <nova:owner>
Aug 09 02:25:58 user nova-compute[72351]:         <nova:user uuid="fd71797160d14dfbba2c65eca2e2f694">tempest-TestServerVolumeAttachmentScenario-1098806376-project-member</nova:user>
Aug 09 02:25:58 user nova-compute[72351]:         <nova:project uuid="de4a7655f2604c8a9c7964cfd298be37">tempest-TestServerVolumeAttachmentScenario-1098806376</nova:project>
Aug 09 02:25:58 user nova-compute[72351]:       </nova:owner>
Aug 09 02:25:58 user nova-compute[72351]:       <nova:root type="image" uuid="18ccda7f-31d3-4978-ab9d-4ed2f74eff94"/>
Aug 09 02:25:58 user nova-compute[72351]:       <nova:ports>
Aug 09 02:25:58 user nova-compute[72351]:         <nova:port uuid="fb1b25f9-3a24-43df-805c-9fffe7cd74f7">
Aug 09 02:25:58 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.12" ipVersion="4"/>
Aug 09 02:25:58 user nova-compute[72351]:         </nova:port>
Aug 09 02:25:58 user nova-compute[72351]:       </nova:ports>
Aug 09 02:25:58 user nova-compute[72351]:     </nova:instance>
Aug 09 02:25:58 user nova-compute[72351]:   </metadata>
Aug 09 02:25:58 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:25:58 user nova-compute[72351]:     <system>
Aug 09 02:25:58 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:25:58 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:25:58 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:25:58 user nova-compute[72351]:       <entry name="serial">16c1e558-baa1-4c2d-b13b-50585f74d587</entry>
Aug 09 02:25:58 user nova-compute[72351]:       <entry name="uuid">16c1e558-baa1-4c2d-b13b-50585f74d587</entry>
Aug 09 02:25:58 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:25:58 user nova-compute[72351]:     </system>
Aug 09 02:25:58 user nova-compute[72351]:   </sysinfo>
Aug 09 02:25:58 user nova-compute[72351]:   <os>
Aug 09 02:25:58 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:25:58 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:25:58 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:25:58 user nova-compute[72351]:   </os>
Aug 09 02:25:58 user nova-compute[72351]:   <features>
Aug 09 02:25:58 user nova-compute[72351]:     <acpi/>
Aug 09 02:25:58 user nova-compute[72351]:     <apic/>
Aug 09 02:25:58 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:25:58 user nova-compute[72351]:   </features>
Aug 09 02:25:58 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:25:58 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:25:58 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:25:58 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:25:58 user nova-compute[72351]:   </clock>
Aug 09 02:25:58 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:25:58 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:25:58 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:25:58 user nova-compute[72351]:   </cpu>
Aug 09 02:25:58 user nova-compute[72351]:   <devices>
Aug 09 02:25:58 user nova-compute[72351]:     <disk type="file" device="disk">
Aug 09 02:25:58 user nova-compute[72351]:       <driver name="qemu" type="qcow2" cache="none"/>
Aug 09 02:25:58 user nova-compute[72351]:       <source file="/opt/stack/data/nova/instances/16c1e558-baa1-4c2d-b13b-50585f74d587/disk"/>
Aug 09 02:25:58 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:25:58 user nova-compute[72351]:     </disk>
Aug 09 02:25:58 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:25:58 user nova-compute[72351]:       <mac address="fa:16:3e:66:3c:d2"/>
Aug 09 02:25:58 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:25:58 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:25:58 user nova-compute[72351]:       <target dev="tapfb1b25f9-3a"/>
Aug 09 02:25:58 user nova-compute[72351]:     </interface>
Aug 09 02:25:58 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:25:58 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/16c1e558-baa1-4c2d-b13b-50585f74d587/console.log" append="off"/>
Aug 09 02:25:58 user nova-compute[72351]:     </serial>
Aug 09 02:25:58 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:25:58 user nova-compute[72351]:     <video>
Aug 09 02:25:58 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:25:58 user nova-compute[72351]:     </video>
Aug 09 02:25:58 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:25:58 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:25:58 user nova-compute[72351]:     </rng>
Aug 09 02:25:58 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:25:58 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:25:58 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:25:58 user nova-compute[72351]:     </memballoon>
Aug 09 02:25:58 user nova-compute[72351]:   </devices>
Aug 09 02:25:58 user nova-compute[72351]: </domain>
Aug 09 02:25:58 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:25:54Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestServerVolumeAttachmentScenario-server-360751817',display_name='tempest-TestServerVolumeAttachmentScenario-server-360751817',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testservervolumeattachmentscenario-server-360751817',id=27,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBO5LKDWUPul31VlDBgVZo5WIbldUiPko7qY2o+u8Xw9ekIjzjjzrKBVPG8SI+m7Xh+/4dSwSjsie+3WOYIYW29h/ArzglDh0AzoC+h3SECFUWbPsf8al1tugMF9vK3vduA==',key_name='tempest-keypair-148936292',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='de4a7655f2604c8a9c7964cfd298be37',ramdisk_id='',reservation_id='r-s9d31yuy',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestServerVolumeAttachmentScenario-1098806376',owner_user_name='tempest-TestServerVolumeAttachmentScenario-1098806376-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:25:56Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='fd71797160d14dfbba2c65eca2e2f694',uuid=16c1e558-baa1-4c2d-b13b-50585f74d587,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "fb1b25f9-3a24-43df-805c-9fffe7cd74f7", "address": "fa:16:3e:66:3c:d2", "network": {"id": "39ddd1d4-5757-425b-92e8-a6a5d8ff29db", "bridge": "br-int", "label": "tempest-TestServerVolumeAttachmentScenario-2122827433-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "de4a7655f2604c8a9c7964cfd298be37", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfb1b25f9-3a", "ovs_interfaceid": "fb1b25f9-3a24-43df-805c-9fffe7cd74f7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Converting VIF {"id": "fb1b25f9-3a24-43df-805c-9fffe7cd74f7", "address": "fa:16:3e:66:3c:d2", "network": {"id": "39ddd1d4-5757-425b-92e8-a6a5d8ff29db", "bridge": "br-int", "label": "tempest-TestServerVolumeAttachmentScenario-2122827433-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "de4a7655f2604c8a9c7964cfd298be37", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfb1b25f9-3a", "ovs_interfaceid": "fb1b25f9-3a24-43df-805c-9fffe7cd74f7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:66:3c:d2,bridge_name='br-int',has_traffic_filtering=True,id=fb1b25f9-3a24-43df-805c-9fffe7cd74f7,network=Network(39ddd1d4-5757-425b-92e8-a6a5d8ff29db),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfb1b25f9-3a') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG os_vif [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:66:3c:d2,bridge_name='br-int',has_traffic_filtering=True,id=fb1b25f9-3a24-43df-805c-9fffe7cd74f7,network=Network(39ddd1d4-5757-425b-92e8-a6a5d8ff29db),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfb1b25f9-3a') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapfb1b25f9-3a, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapfb1b25f9-3a, col_values=(('external_ids', {'iface-id': 'fb1b25f9-3a24-43df-805c-9fffe7cd74f7', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:66:3c:d2', 'vm-uuid': '16c1e558-baa1-4c2d-b13b-50585f74d587'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:58 user nova-compute[72351]: INFO os_vif [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:66:3c:d2,bridge_name='br-int',has_traffic_filtering=True,id=fb1b25f9-3a24-43df-805c-9fffe7cd74f7,network=Network(39ddd1d4-5757-425b-92e8-a6a5d8ff29db),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfb1b25f9-3a')
Aug 09 02:25:58 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] No VIF found with MAC fa:16:3e:66:3c:d2, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-118075b0-4332-4c1e-a492-4c5c41df6585 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Acquiring lock "433e6729-fdd4-4467-971c-08ca962d887d" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-118075b0-4332-4c1e-a492-4c5c41df6585 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lock "433e6729-fdd4-4467-971c-08ca962d887d" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG nova.objects.instance [None req-118075b0-4332-4c1e-a492-4c5c41df6585 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lazy-loading 'flavor' on Instance uuid 433e6729-fdd4-4467-971c-08ca962d887d {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:25:58 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-118075b0-4332-4c1e-a492-4c5c41df6585 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Ignoring supplied device name: /dev/vdb
Aug 09 02:25:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-118075b0-4332-4c1e-a492-4c5c41df6585 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lock "433e6729-fdd4-4467-971c-08ca962d887d" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.082s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-118075b0-4332-4c1e-a492-4c5c41df6585 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Acquiring lock "433e6729-fdd4-4467-971c-08ca962d887d" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-118075b0-4332-4c1e-a492-4c5c41df6585 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lock "433e6729-fdd4-4467-971c-08ca962d887d" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:25:58 user nova-compute[72351]: INFO nova.compute.manager [None req-118075b0-4332-4c1e-a492-4c5c41df6585 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Attaching volume f19fe8f8-5857-44ff-bf31-b2f4a7671a6b to /dev/vdb
Aug 09 02:25:58 user nova-compute[72351]: DEBUG nova.network.neutron [req-230b2e85-fff9-4e2a-913e-7748905952ec req-6f8260a9-5fe8-4b93-bf62-735e8f24d40d service nova] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Updated VIF entry in instance network info cache for port fb1b25f9-3a24-43df-805c-9fffe7cd74f7. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG nova.network.neutron [req-230b2e85-fff9-4e2a-913e-7748905952ec req-6f8260a9-5fe8-4b93-bf62-735e8f24d40d service nova] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Updating instance_info_cache with network_info: [{"id": "fb1b25f9-3a24-43df-805c-9fffe7cd74f7", "address": "fa:16:3e:66:3c:d2", "network": {"id": "39ddd1d4-5757-425b-92e8-a6a5d8ff29db", "bridge": "br-int", "label": "tempest-TestServerVolumeAttachmentScenario-2122827433-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "de4a7655f2604c8a9c7964cfd298be37", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfb1b25f9-3a", "ovs_interfaceid": "fb1b25f9-3a24-43df-805c-9fffe7cd74f7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG nova.network.neutron [req-d6e4b215-deb0-48a6-bbf6-c73ace4e1b95 req-1e7dcc5a-a046-4233-9ad8-0d02d5e66456 service nova] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Updated VIF entry in instance network info cache for port 7f4f4044-2549-41b0-b1b8-9154419a8b54. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG nova.network.neutron [req-d6e4b215-deb0-48a6-bbf6-c73ace4e1b95 req-1e7dcc5a-a046-4233-9ad8-0d02d5e66456 service nova] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Updating instance_info_cache with network_info: [{"id": "7f4f4044-2549-41b0-b1b8-9154419a8b54", "address": "fa:16:3e:04:53:28", "network": {"id": "96018a29-3b2e-44de-b2c4-67abc92e3a30", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-924549970-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.25", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "1c8ff52319444079b754f79843c67b2b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7f4f4044-25", "ovs_interfaceid": "7f4f4044-2549-41b0-b1b8-9154419a8b54", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-230b2e85-fff9-4e2a-913e-7748905952ec req-6f8260a9-5fe8-4b93-bf62-735e8f24d40d service nova] Releasing lock "refresh_cache-16c1e558-baa1-4c2d-b13b-50585f74d587" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d6e4b215-deb0-48a6-bbf6-c73ace4e1b95 req-1e7dcc5a-a046-4233-9ad8-0d02d5e66456 service nova] Releasing lock "refresh_cache-433e6729-fdd4-4467-971c-08ca962d887d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG os_brick.utils [None req-118075b0-4332-4c1e-a492-4c5c41df6585 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.012s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[8b98a3aa-7945-4b46-94d1-5439138857dd]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:a52269602ff4\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.015s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e177d9fe-5e86-4955-9b81-ea60b7c3b041]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[b9ee1919-5b33-44de-b721-35095628c1bb]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[7bdef45f-c59f-41b2-9a78-0ccb10d9c2cd]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-118075b0-4332-4c1e-a492-4c5c41df6585 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Running cmd (subprocess): nvme version {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-118075b0-4332-4c1e-a492-4c5c41df6585 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] 'nvme version' failed. Not Retrying. {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.nvmeof [None req-118075b0-4332-4c1e-a492-4c5c41df6585 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] nvme not present on system {{(pid=72351) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:25:58 user nova-compute[72351]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Aug 09 02:25:58 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[10ddb14c-c582-4d81-a584-b558ec34161d]: (4, '') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-118075b0-4332-4c1e-a492-4c5c41df6585 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72351) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-118075b0-4332-4c1e-a492-4c5c41df6585 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-118075b0-4332-4c1e-a492-4c5c41df6585 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] LIGHTOS: no hostnqn found. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG os_brick.utils [None req-118075b0-4332-4c1e-a492-4c5c41df6585 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] <== get_connector_properties: return (111ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:a52269602ff4', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG nova.virt.block_device [None req-118075b0-4332-4c1e-a492-4c5c41df6585 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Updating existing volume attachment record: 844fc477-8700-4494-9ebc-7f4f361f6fce {{(pid=72351) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72351) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10461}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:25:58 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Auditing locally available compute resources for user (node: user) {{(pid=72351) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Aug 09 02:25:59 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:59 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:59 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:25:59 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:00 user nova-compute[72351]: DEBUG nova.compute.manager [req-4f3ca917-d470-41d0-9840-c846e3e3922a req-2085803e-3264-411c-b080-7e43b4afa5ef service nova] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Received event network-vif-plugged-fb1b25f9-3a24-43df-805c-9fffe7cd74f7 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:26:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-4f3ca917-d470-41d0-9840-c846e3e3922a req-2085803e-3264-411c-b080-7e43b4afa5ef service nova] Acquiring lock "16c1e558-baa1-4c2d-b13b-50585f74d587-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:26:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-4f3ca917-d470-41d0-9840-c846e3e3922a req-2085803e-3264-411c-b080-7e43b4afa5ef service nova] Lock "16c1e558-baa1-4c2d-b13b-50585f74d587-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:26:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-4f3ca917-d470-41d0-9840-c846e3e3922a req-2085803e-3264-411c-b080-7e43b4afa5ef service nova] Lock "16c1e558-baa1-4c2d-b13b-50585f74d587-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:26:00 user nova-compute[72351]: DEBUG nova.compute.manager [req-4f3ca917-d470-41d0-9840-c846e3e3922a req-2085803e-3264-411c-b080-7e43b4afa5ef service nova] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] No waiting events found dispatching network-vif-plugged-fb1b25f9-3a24-43df-805c-9fffe7cd74f7 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:26:00 user nova-compute[72351]: WARNING nova.compute.manager [req-4f3ca917-d470-41d0-9840-c846e3e3922a req-2085803e-3264-411c-b080-7e43b4afa5ef service nova] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Received unexpected event network-vif-plugged-fb1b25f9-3a24-43df-805c-9fffe7cd74f7 for instance with vm_state building and task_state spawning.
Aug 09 02:26:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:00 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/0825f13f-c923-417d-8fa0-f4e61616e4db/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/0825f13f-c923-417d-8fa0-f4e61616e4db/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/0825f13f-c923-417d-8fa0-f4e61616e4db/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/0825f13f-c923-417d-8fa0-f4e61616e4db/disk --force-share --output=json" returned: 0 in 0.180s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/47afce95-e5f7-4703-a99f-7ef8b28c4bd2/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/47afce95-e5f7-4703-a99f-7ef8b28c4bd2/disk --force-share --output=json" returned: 0 in 0.145s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/47afce95-e5f7-4703-a99f-7ef8b28c4bd2/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/47afce95-e5f7-4703-a99f-7ef8b28c4bd2/disk --force-share --output=json" returned: 0 in 0.142s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.rescue --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.rescue --force-share --output=json" returned: 0 in 0.140s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.rescue --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.rescue --force-share --output=json" returned: 0 in 0.134s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json" returned: 0 in 0.132s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json" returned: 0 in 0.132s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f457e6ea-8d06-46c6-be2e-f650e28a3b69/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:01 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547961.8788288, 16c1e558-baa1-4c2d-b13b-50585f74d587 => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:26:01 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] VM Resumed (Lifecycle Event)
Aug 09 02:26:01 user nova-compute[72351]: DEBUG nova.compute.manager [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:26:01 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Guest created on hypervisor {{(pid=72351) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4411}}
Aug 09 02:26:01 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Instance spawned successfully.
Aug 09 02:26:01 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:894}}
Aug 09 02:26:01 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:26:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f457e6ea-8d06-46c6-be2e-f650e28a3b69/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f457e6ea-8d06-46c6-be2e-f650e28a3b69/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:01 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:26:01 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Found default for hw_cdrom_bus of ide {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:26:01 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Found default for hw_disk_bus of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:26:01 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Found default for hw_input_bus of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:26:01 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Found default for hw_pointer_model of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:26:01 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Found default for hw_video_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:26:01 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Found default for hw_vif_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:26:01 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:26:01 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547961.882232, 16c1e558-baa1-4c2d-b13b-50585f74d587 => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:26:01 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] VM Started (Lifecycle Event)
Aug 09 02:26:02 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:26:02 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:26:02 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:26:02 user nova-compute[72351]: INFO nova.compute.manager [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Took 6.12 seconds to spawn the instance on the hypervisor.
Aug 09 02:26:02 user nova-compute[72351]: DEBUG nova.compute.manager [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:26:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f457e6ea-8d06-46c6-be2e-f650e28a3b69/disk --force-share --output=json" returned: 0 in 0.161s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:02 user nova-compute[72351]: DEBUG nova.compute.manager [req-75178d62-4824-4c27-92e5-f60e159d6773 req-f9587843-6cde-4348-88d3-6302712456e8 service nova] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Received event network-vif-plugged-fb1b25f9-3a24-43df-805c-9fffe7cd74f7 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:26:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-75178d62-4824-4c27-92e5-f60e159d6773 req-f9587843-6cde-4348-88d3-6302712456e8 service nova] Acquiring lock "16c1e558-baa1-4c2d-b13b-50585f74d587-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:26:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-75178d62-4824-4c27-92e5-f60e159d6773 req-f9587843-6cde-4348-88d3-6302712456e8 service nova] Lock "16c1e558-baa1-4c2d-b13b-50585f74d587-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:26:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-75178d62-4824-4c27-92e5-f60e159d6773 req-f9587843-6cde-4348-88d3-6302712456e8 service nova] Lock "16c1e558-baa1-4c2d-b13b-50585f74d587-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:26:02 user nova-compute[72351]: DEBUG nova.compute.manager [req-75178d62-4824-4c27-92e5-f60e159d6773 req-f9587843-6cde-4348-88d3-6302712456e8 service nova] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] No waiting events found dispatching network-vif-plugged-fb1b25f9-3a24-43df-805c-9fffe7cd74f7 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:26:02 user nova-compute[72351]: WARNING nova.compute.manager [req-75178d62-4824-4c27-92e5-f60e159d6773 req-f9587843-6cde-4348-88d3-6302712456e8 service nova] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Received unexpected event network-vif-plugged-fb1b25f9-3a24-43df-805c-9fffe7cd74f7 for instance with vm_state building and task_state spawning.
Aug 09 02:26:02 user nova-compute[72351]: INFO nova.compute.manager [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Took 7.08 seconds to build instance.
Aug 09 02:26:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-0dcbc49c-4782-4341-9661-b7a4db94d49f tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Lock "16c1e558-baa1-4c2d-b13b-50585f74d587" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 7.186s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:26:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json" returned: 0 in 0.155s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json" returned: 0 in 0.176s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/433e6729-fdd4-4467-971c-08ca962d887d/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/433e6729-fdd4-4467-971c-08ca962d887d/disk --force-share --output=json" returned: 0 in 0.165s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/433e6729-fdd4-4467-971c-08ca962d887d/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/433e6729-fdd4-4467-971c-08ca962d887d/disk --force-share --output=json" returned: 0 in 0.148s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/16c1e558-baa1-4c2d-b13b-50585f74d587/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/16c1e558-baa1-4c2d-b13b-50585f74d587/disk --force-share --output=json" returned: 0 in 0.142s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/16c1e558-baa1-4c2d-b13b-50585f74d587/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:02 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/16c1e558-baa1-4c2d-b13b-50585f74d587/disk --force-share --output=json" returned: 0 in 0.154s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json" returned: 0 in 0.156s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json" returned: 0 in 0.198s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:03 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdd (vda) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:26:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff/disk --force-share --output=json" returned: 0 in 0.168s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff/disk --force-share --output=json" returned: 0 in 0.145s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:03 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdc (vdb) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:26:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/831c864a-75d9-42b3-b952-749f4201e40e/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/831c864a-75d9-42b3-b952-749f4201e40e/disk --force-share --output=json" returned: 0 in 0.170s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/831c864a-75d9-42b3-b952-749f4201e40e/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/831c864a-75d9-42b3-b952-749f4201e40e/disk --force-share --output=json" returned: 0 in 0.143s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:04 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:26:04 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:26:04 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Hypervisor/Node resource view: name=user free_ram=6913MB free_disk=25.324199676513672GB free_vcpus=0 pci_devices=[{"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72351) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Aug 09 02:26:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:26:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:26:04 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Applying migration context for instance 75bbbead-8628-448b-9484-19bf4ab922ff as it has an incoming, in-progress migration 161f097f-81e6-4201-8af0-05f72c1ba9e0. Migration status is finished {{(pid=72351) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1016}}
Aug 09 02:26:04 user nova-compute[72351]: INFO nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Updating resource usage from migration 161f097f-81e6-4201-8af0-05f72c1ba9e0
Aug 09 02:26:04 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:26:04 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance eb86cb49-2018-42c2-bd97-c662b829775d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:26:04 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 1d772c10-1c8a-4101-9a61-f8a60bb19c66 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:26:04 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:26:04 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance a7f3d076-864b-4b68-9207-214ce2c21444 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:26:04 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 433e6729-fdd4-4467-971c-08ca962d887d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:26:04 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 831c864a-75d9-42b3-b952-749f4201e40e actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:26:04 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 0825f13f-c923-417d-8fa0-f4e61616e4db actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:26:04 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance f457e6ea-8d06-46c6-be2e-f650e28a3b69 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:26:04 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Migration 161f097f-81e6-4201-8af0-05f72c1ba9e0 is active on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1709}}
Aug 09 02:26:04 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 75bbbead-8628-448b-9484-19bf4ab922ff actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 20, 'MEMORY_MB': 2048, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:26:04 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 47afce95-e5f7-4703-a99f-7ef8b28c4bd2 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:26:04 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 16c1e558-baa1-4c2d-b13b-50585f74d587 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:26:04 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Total usable vcpus: 12, total allocated vcpus: 13 {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Aug 09 02:26:04 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Final resource view: name=user phys_ram=16011MB used_ram=8704MB phys_disk=40GB used_disk=31GB total_vcpus=12 used_vcpus=13 pci_stats=[] {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Aug 09 02:26:05 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:26:05 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:26:05 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Compute_service record updated for user:user {{(pid=72351) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Aug 09 02:26:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.671s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:26:06 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:26:06 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:26:06 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Starting heal instance info cache {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9842}}
Aug 09 02:26:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "refresh_cache-eb86cb49-2018-42c2-bd97-c662b829775d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:26:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquired lock "refresh_cache-eb86cb49-2018-42c2-bd97-c662b829775d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:26:06 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Forcefully refreshing network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1996}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Updating instance_info_cache with network_info: [{"id": "5f97db70-9e5b-4d42-b765-b832fb522f41", "address": "fa:16:3e:dc:da:c7", "network": {"id": "49e8751c-e119-430f-9110-3dc6069b1589", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-899275135-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3aea1e66805146f7a8cd1d97e64a24af", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5f97db70-9e", "ovs_interfaceid": "5f97db70-9e5b-4d42-b765-b832fb522f41", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Releasing lock "refresh_cache-eb86cb49-2018-42c2-bd97-c662b829775d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Updated the network info_cache for instance {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9913}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-118075b0-4332-4c1e-a492-4c5c41df6585 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-118075b0-4332-4c1e-a492-4c5c41df6585 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'HGSEFVOU', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:26:07 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-118075b0-4332-4c1e-a492-4c5c41df6585 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:26:07 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-118075b0-4332-4c1e-a492-4c5c41df6585 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-118075b0-4332-4c1e-a492-4c5c41df6585 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:26:07 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-118075b0-4332-4c1e-a492-4c5c41df6585 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:26:07 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-118075b0-4332-4c1e-a492-4c5c41df6585 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-118075b0-4332-4c1e-a492-4c5c41df6585 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:26:07 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-118075b0-4332-4c1e-a492-4c5c41df6585 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 02:26:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b -p 172.16.0.220:3260" returned: 21 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[cccfdcc9-1c35-41ae-9e12-b120de17f83a]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-118075b0-4332-4c1e-a492-4c5c41df6585 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Aug 09 02:26:07 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b -p 172.16.0.220:3260 --interface default --op new {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[33219224-8fa7-40cf-9594-d3782deee5a1]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b] added\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-118075b0-4332-4c1e-a492-4c5c41df6585 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b] added
Aug 09 02:26:07 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[1e5af933-9a1b-4f0b-a7ca-779415769cc5]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-118075b0-4332-4c1e-a492-4c5c41df6585 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[621b6ffa-41e3-4e89-b78a-6ea94f68e211]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-118075b0-4332-4c1e-a492-4c5c41df6585 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b -p 172.16.0.220:3260 --op update -n node.session.auth.username -v HGSEFVOU {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b -p 172.16.0.220:3260 --op update -n node.session.auth.username -v HGSEFVOU" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[9ee812a2-d336-4de5-8a75-cc57efcfd960]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-118075b0-4332-4c1e-a492-4c5c41df6585 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'HGSEFVOU'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e55fb26c-9fcd-4b66-a413-7eabb6287b75]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-118075b0-4332-4c1e-a492-4c5c41df6585 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[0130d2da-a90e-45a6-8966-7915a33a223d]: (4, ('tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)\ntcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-118075b0-4332-4c1e-a492-4c5c41df6585 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)
Aug 09 02:26:07 user nova-compute[72351]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:26:07 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-118075b0-4332-4c1e-a492-4c5c41df6585 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] iscsi session list stdout=tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)
Aug 09 02:26:07 user nova-compute[72351]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:26:07 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b -p 172.16.0.220:3260 --login {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b -p 172.16.0.220:3260 --login" returned: 0 in 0.059s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[0cff52a2-dd09-4f10-b7ce-eac5ca281650]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-118075b0-4332-4c1e-a492-4c5c41df6585 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b, portal: 172.16.0.220,3260]
Aug 09 02:26:07 user nova-compute[72351]: Login to [iface: default, target: iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b, portal: 172.16.0.220,3260] successful.
Aug 09 02:26:07 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ab0f7bc6-d92f-4df8-97b7-fd1fc8d1a8e3]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-118075b0-4332-4c1e-a492-4c5c41df6585 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.027s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[19e6f3e6-99e8-4947-b1a3-598073b7cffe]: (4, ('tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)\ntcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)\ntcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-118075b0-4332-4c1e-a492-4c5c41df6585 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)
Aug 09 02:26:07 user nova-compute[72351]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:26:07 user nova-compute[72351]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b (non-flash)
Aug 09 02:26:07 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-118075b0-4332-4c1e-a492-4c5c41df6585 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] iscsi session list stdout=tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)
Aug 09 02:26:07 user nova-compute[72351]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:26:07 user nova-compute[72351]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b (non-flash)
Aug 09 02:26:07 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-118075b0-4332-4c1e-a492-4c5c41df6585 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.278s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-118075b0-4332-4c1e-a492-4c5c41df6585 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-118075b0-4332-4c1e-a492-4c5c41df6585 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] HCTL ('33', '-', '-', 0) found on session 22 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-118075b0-4332-4c1e-a492-4c5c41df6585 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[be86fe4d-4588-4ee8-a6c8-0f6ddb0cc2d6]: (4, ('- - 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-118075b0-4332-4c1e-a492-4c5c41df6585 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Searching for a device in session 22 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-118075b0-4332-4c1e-a492-4c5c41df6585 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Connected to sdb using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'HGSEFVOU', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 02:26:07 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c852f29a-fb98-41e3-a5e9-95b324b4c994 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Acquiring lock "585c81a5-ac24-4d12-a8a6-1e7b12a2d53a" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:26:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c852f29a-fb98-41e3-a5e9-95b324b4c994 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "585c81a5-ac24-4d12-a8a6-1e7b12a2d53a" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:26:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:08 user nova-compute[72351]: DEBUG nova.objects.instance [None req-c852f29a-fb98-41e3-a5e9-95b324b4c994 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lazy-loading 'flavor' on Instance uuid 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:26:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c852f29a-fb98-41e3-a5e9-95b324b4c994 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "585c81a5-ac24-4d12-a8a6-1e7b12a2d53a" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.090s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:26:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c852f29a-fb98-41e3-a5e9-95b324b4c994 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Acquiring lock "585c81a5-ac24-4d12-a8a6-1e7b12a2d53a" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:26:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c852f29a-fb98-41e3-a5e9-95b324b4c994 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "585c81a5-ac24-4d12-a8a6-1e7b12a2d53a" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:26:08 user nova-compute[72351]: INFO nova.compute.manager [None req-c852f29a-fb98-41e3-a5e9-95b324b4c994 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Attaching volume 40cdcbc5-17b6-490e-8989-3d6ab80bdd80 to /dev/vdb
Aug 09 02:26:08 user nova-compute[72351]: DEBUG os_brick.utils [None req-c852f29a-fb98-41e3-a5e9-95b324b4c994 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:26:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.013s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:08 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[b1a58f8d-f09a-4887-a165-ea3eb14a4b8d]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:a52269602ff4\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.016s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:08 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[0a0632d9-0a9d-401f-97fe-70470404d34a]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:08 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[5bab56fb-845d-4b96-92a4-f56d50b55f24]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:08 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[71bd6f15-3398-4eb1-a081-4a57f6d0c23d]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-c852f29a-fb98-41e3-a5e9-95b324b4c994 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Running cmd (subprocess): nvme version {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-c852f29a-fb98-41e3-a5e9-95b324b4c994 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] 'nvme version' failed. Not Retrying. {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:26:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.nvmeof [None req-c852f29a-fb98-41e3-a5e9-95b324b4c994 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] nvme not present on system {{(pid=72351) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Aug 09 02:26:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:08 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691547953.5606096, 5c71284c-df41-4990-9a46-7986982bf7e2 => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:26:08 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] VM Stopped (Lifecycle Event)
Aug 09 02:26:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:26:08 user nova-compute[72351]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Aug 09 02:26:08 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[92abf43b-7c97-4589-9e57-9cfda79bdbce]: (4, '') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-c852f29a-fb98-41e3-a5e9-95b324b4c994 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72351) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Aug 09 02:26:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-c852f29a-fb98-41e3-a5e9-95b324b4c994 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Aug 09 02:26:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-c852f29a-fb98-41e3-a5e9-95b324b4c994 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] LIGHTOS: no hostnqn found. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Aug 09 02:26:08 user nova-compute[72351]: DEBUG os_brick.utils [None req-c852f29a-fb98-41e3-a5e9-95b324b4c994 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] <== get_connector_properties: return (123ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:a52269602ff4', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:26:08 user nova-compute[72351]: DEBUG nova.virt.block_device [None req-c852f29a-fb98-41e3-a5e9-95b324b4c994 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Updating existing volume attachment record: 402920de-47e0-4438-afb7-df610d2c0810 {{(pid=72351) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Aug 09 02:26:08 user nova-compute[72351]: DEBUG nova.compute.manager [None req-d3a8d6b8-4812-4c63-87e6-6b3c7dabbb92 None None] [instance: 5c71284c-df41-4990-9a46-7986982bf7e2] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:26:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-118075b0-4332-4c1e-a492-4c5c41df6585 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.315s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:26:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-118075b0-4332-4c1e-a492-4c5c41df6585 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] <== connect_volume: return (1317ms) {'type': 'block', 'scsi_wwn': '23036643931383665', 'path': '/dev/sdb'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:26:08 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-118075b0-4332-4c1e-a492-4c5c41df6585 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23036643931383665', 'path': '/dev/sdb'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 02:26:08 user nova-compute[72351]: DEBUG nova.objects.instance [None req-118075b0-4332-4c1e-a492-4c5c41df6585 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lazy-loading 'flavor' on Instance uuid 433e6729-fdd4-4467-971c-08ca962d887d {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:26:08 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-118075b0-4332-4c1e-a492-4c5c41df6585 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] attach device xml: <disk type="block" device="disk">
Aug 09 02:26:08 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:26:08 user nova-compute[72351]:   <source dev="/dev/sdb"/>
Aug 09 02:26:08 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:26:08 user nova-compute[72351]:   <serial>f19fe8f8-5857-44ff-bf31-b2f4a7671a6b</serial>
Aug 09 02:26:08 user nova-compute[72351]: </disk>
Aug 09 02:26:08 user nova-compute[72351]:  {{(pid=72351) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Aug 09 02:26:09 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-118075b0-4332-4c1e-a492-4c5c41df6585 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:26:09 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-118075b0-4332-4c1e-a492-4c5c41df6585 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:26:09 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-118075b0-4332-4c1e-a492-4c5c41df6585 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] No VIF found with MAC fa:16:3e:04:53:28, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:26:09 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-118075b0-4332-4c1e-a492-4c5c41df6585 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lock "433e6729-fdd4-4467-971c-08ca962d887d" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 10.828s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:26:12 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-2b86b241-76fc-40de-9421-4e176d6576d6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Acquiring lock "831c864a-75d9-42b3-b952-749f4201e40e" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:26:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-2b86b241-76fc-40de-9421-4e176d6576d6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "831c864a-75d9-42b3-b952-749f4201e40e" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:26:14 user nova-compute[72351]: DEBUG nova.objects.instance [None req-2b86b241-76fc-40de-9421-4e176d6576d6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lazy-loading 'flavor' on Instance uuid 831c864a-75d9-42b3-b952-749f4201e40e {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:26:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-2b86b241-76fc-40de-9421-4e176d6576d6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "831c864a-75d9-42b3-b952-749f4201e40e" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.083s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:26:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-2b86b241-76fc-40de-9421-4e176d6576d6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Acquiring lock "831c864a-75d9-42b3-b952-749f4201e40e" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:26:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-2b86b241-76fc-40de-9421-4e176d6576d6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "831c864a-75d9-42b3-b952-749f4201e40e" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:26:14 user nova-compute[72351]: INFO nova.compute.manager [None req-2b86b241-76fc-40de-9421-4e176d6576d6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Attaching volume 5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e to /dev/vdb
Aug 09 02:26:14 user nova-compute[72351]: DEBUG os_brick.utils [None req-2b86b241-76fc-40de-9421-4e176d6576d6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:26:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:14 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ffa79d3e-0366-4f8c-9a81-044cd3b9d9d2]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:a52269602ff4\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:14 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e7e0ce7f-88b5-45ca-91eb-dddb79ac5d3d]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.016s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:14 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[dae82969-eb55-4877-9563-64625e373fef]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:14 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[58eeecfe-1c87-4b54-a324-deda834bbf95]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-2b86b241-76fc-40de-9421-4e176d6576d6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Running cmd (subprocess): nvme version {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-2b86b241-76fc-40de-9421-4e176d6576d6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] 'nvme version' failed. Not Retrying. {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:26:14 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.nvmeof [None req-2b86b241-76fc-40de-9421-4e176d6576d6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] nvme not present on system {{(pid=72351) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Aug 09 02:26:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:26:14 user nova-compute[72351]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Aug 09 02:26:14 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[fcc0ac6d-6731-4dac-9c14-8428591cea13]: (4, '') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:14 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-2b86b241-76fc-40de-9421-4e176d6576d6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72351) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Aug 09 02:26:14 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-2b86b241-76fc-40de-9421-4e176d6576d6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Aug 09 02:26:14 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-2b86b241-76fc-40de-9421-4e176d6576d6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] LIGHTOS: no hostnqn found. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Aug 09 02:26:14 user nova-compute[72351]: DEBUG os_brick.utils [None req-2b86b241-76fc-40de-9421-4e176d6576d6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] <== get_connector_properties: return (119ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:a52269602ff4', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:26:14 user nova-compute[72351]: DEBUG nova.virt.block_device [None req-2b86b241-76fc-40de-9421-4e176d6576d6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Updating existing volume attachment record: dd28b6eb-ac25-4b95-9d79-e0108173f619 {{(pid=72351) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Aug 09 02:26:16 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-9fbbe74c-f9b4-49d1-b1c2-6ab280e45afb tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Acquiring lock "0825f13f-c923-417d-8fa0-f4e61616e4db" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:26:16 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-9fbbe74c-f9b4-49d1-b1c2-6ab280e45afb tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Lock "0825f13f-c923-417d-8fa0-f4e61616e4db" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:26:16 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-9fbbe74c-f9b4-49d1-b1c2-6ab280e45afb tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Acquiring lock "0825f13f-c923-417d-8fa0-f4e61616e4db-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:26:16 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-9fbbe74c-f9b4-49d1-b1c2-6ab280e45afb tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Lock "0825f13f-c923-417d-8fa0-f4e61616e4db-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:26:16 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-9fbbe74c-f9b4-49d1-b1c2-6ab280e45afb tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Lock "0825f13f-c923-417d-8fa0-f4e61616e4db-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:26:16 user nova-compute[72351]: INFO nova.compute.manager [None req-9fbbe74c-f9b4-49d1-b1c2-6ab280e45afb tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Terminating instance
Aug 09 02:26:16 user nova-compute[72351]: DEBUG nova.compute.manager [None req-9fbbe74c-f9b4-49d1-b1c2-6ab280e45afb tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Start destroying the instance on the hypervisor. {{(pid=72351) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3124}}
Aug 09 02:26:16 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:16 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:16 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG nova.compute.manager [req-e178966a-fc84-4cae-af08-85f07c2e86f8 req-32d8742a-7d14-4d80-b317-6dc1abece5e9 service nova] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Received event network-vif-unplugged-10079897-954d-49a0-9007-51efb10241bc {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-e178966a-fc84-4cae-af08-85f07c2e86f8 req-32d8742a-7d14-4d80-b317-6dc1abece5e9 service nova] Acquiring lock "0825f13f-c923-417d-8fa0-f4e61616e4db-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-e178966a-fc84-4cae-af08-85f07c2e86f8 req-32d8742a-7d14-4d80-b317-6dc1abece5e9 service nova] Lock "0825f13f-c923-417d-8fa0-f4e61616e4db-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-e178966a-fc84-4cae-af08-85f07c2e86f8 req-32d8742a-7d14-4d80-b317-6dc1abece5e9 service nova] Lock "0825f13f-c923-417d-8fa0-f4e61616e4db-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG nova.compute.manager [req-e178966a-fc84-4cae-af08-85f07c2e86f8 req-32d8742a-7d14-4d80-b317-6dc1abece5e9 service nova] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] No waiting events found dispatching network-vif-unplugged-10079897-954d-49a0-9007-51efb10241bc {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG nova.compute.manager [req-e178966a-fc84-4cae-af08-85f07c2e86f8 req-32d8742a-7d14-4d80-b317-6dc1abece5e9 service nova] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Received event network-vif-unplugged-10079897-954d-49a0-9007-51efb10241bc for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-c852f29a-fb98-41e3-a5e9-95b324b4c994 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c852f29a-fb98-41e3-a5e9-95b324b4c994 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'dEclGUPX', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:26:17 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-c852f29a-fb98-41e3-a5e9-95b324b4c994 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:26:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-c852f29a-fb98-41e3-a5e9-95b324b4c994 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-c852f29a-fb98-41e3-a5e9-95b324b4c994 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:26:17 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-c852f29a-fb98-41e3-a5e9-95b324b4c994 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:26:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-c852f29a-fb98-41e3-a5e9-95b324b4c994 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-c852f29a-fb98-41e3-a5e9-95b324b4c994 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:26:17 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-c852f29a-fb98-41e3-a5e9-95b324b4c994 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 02:26:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 -p 172.16.0.220:3260" returned: 21 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[3d3f042e-daa9-4652-81ef-31f5a8da55b9]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c852f29a-fb98-41e3-a5e9-95b324b4c994 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Aug 09 02:26:17 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 -p 172.16.0.220:3260 --interface default --op new {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e6029225-df9e-4a08-99b4-e128bf19f4ec]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80] added\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c852f29a-fb98-41e3-a5e9-95b324b4c994 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80] added
Aug 09 02:26:17 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.026s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[2f1105d1-25b3-40b9-bbf9-77224bc1a54a]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c852f29a-fb98-41e3-a5e9-95b324b4c994 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.032s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[81c419da-405d-4d41-a95c-0b02766480c5]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c852f29a-fb98-41e3-a5e9-95b324b4c994 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v dEclGUPX {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v dEclGUPX" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[5c4d4b35-ed00-4a2d-9606-b8efa5db8445]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c852f29a-fb98-41e3-a5e9-95b324b4c994 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'dEclGUPX'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[c58c8a1a-e174-44e4-83be-3745b6cd5903]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c852f29a-fb98-41e3-a5e9-95b324b4c994 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[5e7cc5a2-09a1-4a89-a8e6-7e40b4084476]: (4, ('tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)\ntcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)\ntcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c852f29a-fb98-41e3-a5e9-95b324b4c994 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)
Aug 09 02:26:17 user nova-compute[72351]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:26:17 user nova-compute[72351]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b (non-flash)
Aug 09 02:26:17 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c852f29a-fb98-41e3-a5e9-95b324b4c994 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsi session list stdout=tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)
Aug 09 02:26:17 user nova-compute[72351]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:26:17 user nova-compute[72351]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b (non-flash)
Aug 09 02:26:17 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 -p 172.16.0.220:3260 --login {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 -p 172.16.0.220:3260 --login" returned: 0 in 0.064s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[07bc49cc-7d55-4da8-a617-cca48d65b2c3]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c852f29a-fb98-41e3-a5e9-95b324b4c994 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80, portal: 172.16.0.220,3260]
Aug 09 02:26:17 user nova-compute[72351]: Login to [iface: default, target: iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80, portal: 172.16.0.220,3260] successful.
Aug 09 02:26:17 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:17 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Instance destroyed successfully.
Aug 09 02:26:17 user nova-compute[72351]: DEBUG nova.objects.instance [None req-9fbbe74c-f9b4-49d1-b1c2-6ab280e45afb tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Lazy-loading 'resources' on Instance uuid 0825f13f-c923-417d-8fa0-f4e61616e4db {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.026s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[77f9eee9-12cb-44d5-a036-ba4cf09f0cb1]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c852f29a-fb98-41e3-a5e9-95b324b4c994 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-9fbbe74c-f9b4-49d1-b1c2-6ab280e45afb tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:24:26Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-VolumesNegativeTest-instance-120898461',display_name='tempest-VolumesNegativeTest-instance-120898461',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-volumesnegativetest-instance-120898461',id=24,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:24:35Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='1af75c014509460eab5821b6a5c1e124',ramdisk_id='',reservation_id='r-05i3waz6',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-VolumesNegativeTest-1226059554',owner_user_name='tempest-VolumesNegativeTest-1226059554-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:24:36Z,user_data=None,user_id='693fd098694d41dd91edb717fd8170ba',uuid=0825f13f-c923-417d-8fa0-f4e61616e4db,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "10079897-954d-49a0-9007-51efb10241bc", "address": "fa:16:3e:0d:57:48", "network": {"id": "f9d6cd8a-f9ec-46e3-a28a-22530e7ddda2", "bridge": "br-int", "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.142", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.1"}}], "meta": {"injected": false, "tenant_id": "f9af963f7c574a7ab2deedfcf953a666", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap10079897-95", "ovs_interfaceid": "10079897-954d-49a0-9007-51efb10241bc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-9fbbe74c-f9b4-49d1-b1c2-6ab280e45afb tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Converting VIF {"id": "10079897-954d-49a0-9007-51efb10241bc", "address": "fa:16:3e:0d:57:48", "network": {"id": "f9d6cd8a-f9ec-46e3-a28a-22530e7ddda2", "bridge": "br-int", "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.142", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.1"}}], "meta": {"injected": false, "tenant_id": "f9af963f7c574a7ab2deedfcf953a666", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap10079897-95", "ovs_interfaceid": "10079897-954d-49a0-9007-51efb10241bc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-9fbbe74c-f9b4-49d1-b1c2-6ab280e45afb tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:0d:57:48,bridge_name='br-int',has_traffic_filtering=True,id=10079897-954d-49a0-9007-51efb10241bc,network=Network(f9d6cd8a-f9ec-46e3-a28a-22530e7ddda2),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap10079897-95') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG os_vif [None req-9fbbe74c-f9b4-49d1-b1c2-6ab280e45afb tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:0d:57:48,bridge_name='br-int',has_traffic_filtering=True,id=10079897-954d-49a0-9007-51efb10241bc,network=Network(f9d6cd8a-f9ec-46e3-a28a-22530e7ddda2),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap10079897-95') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap10079897-95, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:17 user nova-compute[72351]: INFO os_vif [None req-9fbbe74c-f9b4-49d1-b1c2-6ab280e45afb tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:0d:57:48,bridge_name='br-int',has_traffic_filtering=True,id=10079897-954d-49a0-9007-51efb10241bc,network=Network(f9d6cd8a-f9ec-46e3-a28a-22530e7ddda2),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap10079897-95')
Aug 09 02:26:17 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-9fbbe74c-f9b4-49d1-b1c2-6ab280e45afb tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Deleting instance files /opt/stack/data/nova/instances/0825f13f-c923-417d-8fa0-f4e61616e4db_del
Aug 09 02:26:17 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-9fbbe74c-f9b4-49d1-b1c2-6ab280e45afb tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Deletion of /opt/stack/data/nova/instances/0825f13f-c923-417d-8fa0-f4e61616e4db_del complete
Aug 09 02:26:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[5fdcf17b-1f77-44ab-855e-3e778f5b0f4c]: (4, ('tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)\ntcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)\ntcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b (non-flash)\ntcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c852f29a-fb98-41e3-a5e9-95b324b4c994 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)
Aug 09 02:26:17 user nova-compute[72351]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:26:17 user nova-compute[72351]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b (non-flash)
Aug 09 02:26:17 user nova-compute[72351]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)
Aug 09 02:26:17 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c852f29a-fb98-41e3-a5e9-95b324b4c994 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsi session list stdout=tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)
Aug 09 02:26:17 user nova-compute[72351]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:26:17 user nova-compute[72351]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b (non-flash)
Aug 09 02:26:17 user nova-compute[72351]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)
Aug 09 02:26:17 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-c852f29a-fb98-41e3-a5e9-95b324b4c994 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.314s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c852f29a-fb98-41e3-a5e9-95b324b4c994 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-c852f29a-fb98-41e3-a5e9-95b324b4c994 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] HCTL ('36', '-', '-', 0) found on session 23 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-c852f29a-fb98-41e3-a5e9-95b324b4c994 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Scanning host 36 c: -, t: -, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host36/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host36/scan" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[781930bd-35ea-44a6-92d3-92f75a0bfadd]: (4, ('- - 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-c852f29a-fb98-41e3-a5e9-95b324b4c994 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Searching for a device in session 23 and hctl ['36', '*', '*', 0] yield: sde {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c852f29a-fb98-41e3-a5e9-95b324b4c994 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Connected to sde using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'dEclGUPX', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 02:26:17 user nova-compute[72351]: INFO nova.compute.manager [None req-9fbbe74c-f9b4-49d1-b1c2-6ab280e45afb tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Took 0.88 seconds to destroy the instance on the hypervisor.
Aug 09 02:26:17 user nova-compute[72351]: DEBUG oslo.service.loopingcall [None req-9fbbe74c-f9b4-49d1-b1c2-6ab280e45afb tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72351) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG nova.compute.manager [-] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Deallocating network for instance {{(pid=72351) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2263}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] deallocate_for_instance() {{(pid=72351) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1795}}
Aug 09 02:26:17 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:18 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:26:18 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Took 0.47 seconds to deallocate network for instance.
Aug 09 02:26:18 user nova-compute[72351]: DEBUG nova.compute.manager [req-d08a03e4-cf0f-451a-8c8a-d47a178422b7 req-09355779-0b79-4d73-a732-b51afeeda10b service nova] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Received event network-vif-deleted-10079897-954d-49a0-9007-51efb10241bc {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:26:18 user nova-compute[72351]: INFO nova.compute.manager [req-d08a03e4-cf0f-451a-8c8a-d47a178422b7 req-09355779-0b79-4d73-a732-b51afeeda10b service nova] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Neutron deleted interface 10079897-954d-49a0-9007-51efb10241bc; detaching it from the instance and deleting it from the info cache
Aug 09 02:26:18 user nova-compute[72351]: DEBUG nova.network.neutron [req-d08a03e4-cf0f-451a-8c8a-d47a178422b7 req-09355779-0b79-4d73-a732-b51afeeda10b service nova] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:26:18 user nova-compute[72351]: DEBUG nova.compute.manager [req-d08a03e4-cf0f-451a-8c8a-d47a178422b7 req-09355779-0b79-4d73-a732-b51afeeda10b service nova] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Detach interface failed, port_id=10079897-954d-49a0-9007-51efb10241bc, reason: Instance 0825f13f-c923-417d-8fa0-f4e61616e4db could not be found. {{(pid=72351) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10866}}
Aug 09 02:26:18 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-9fbbe74c-f9b4-49d1-b1c2-6ab280e45afb tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:26:18 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-9fbbe74c-f9b4-49d1-b1c2-6ab280e45afb tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:26:18 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-9fbbe74c-f9b4-49d1-b1c2-6ab280e45afb tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:26:18 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-9fbbe74c-f9b4-49d1-b1c2-6ab280e45afb tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:26:18 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-9fbbe74c-f9b4-49d1-b1c2-6ab280e45afb tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.401s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:26:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-c852f29a-fb98-41e3-a5e9-95b324b4c994 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.351s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:26:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-c852f29a-fb98-41e3-a5e9-95b324b4c994 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] <== connect_volume: return (1352ms) {'type': 'block', 'scsi_wwn': '23330386161636566', 'path': '/dev/sde'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:26:18 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-c852f29a-fb98-41e3-a5e9-95b324b4c994 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23330386161636566', 'path': '/dev/sde'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 02:26:18 user nova-compute[72351]: DEBUG nova.objects.instance [None req-c852f29a-fb98-41e3-a5e9-95b324b4c994 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lazy-loading 'flavor' on Instance uuid 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:26:18 user nova-compute[72351]: INFO nova.scheduler.client.report [None req-9fbbe74c-f9b4-49d1-b1c2-6ab280e45afb tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Deleted allocations for instance 0825f13f-c923-417d-8fa0-f4e61616e4db
Aug 09 02:26:18 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-c852f29a-fb98-41e3-a5e9-95b324b4c994 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] attach device xml: <disk type="block" device="disk">
Aug 09 02:26:18 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:26:18 user nova-compute[72351]:   <source dev="/dev/sde"/>
Aug 09 02:26:18 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:26:18 user nova-compute[72351]:   <serial>40cdcbc5-17b6-490e-8989-3d6ab80bdd80</serial>
Aug 09 02:26:18 user nova-compute[72351]: </disk>
Aug 09 02:26:18 user nova-compute[72351]:  {{(pid=72351) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Aug 09 02:26:18 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-9fbbe74c-f9b4-49d1-b1c2-6ab280e45afb tempest-VolumesNegativeTest-1226059554 tempest-VolumesNegativeTest-1226059554-project-member] Lock "0825f13f-c923-417d-8fa0-f4e61616e4db" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 1.978s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:26:19 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c852f29a-fb98-41e3-a5e9-95b324b4c994 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:26:19 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c852f29a-fb98-41e3-a5e9-95b324b4c994 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:26:19 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c852f29a-fb98-41e3-a5e9-95b324b4c994 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] No VIF found with MAC fa:16:3e:e5:b0:10, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:26:19 user nova-compute[72351]: DEBUG nova.compute.manager [req-bd2093be-6357-4a3b-9fbb-df1cd31bcdd1 req-c19bcb00-e1e5-4e34-bb25-ff30cef21aa3 service nova] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Received event network-vif-plugged-10079897-954d-49a0-9007-51efb10241bc {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:26:19 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-bd2093be-6357-4a3b-9fbb-df1cd31bcdd1 req-c19bcb00-e1e5-4e34-bb25-ff30cef21aa3 service nova] Acquiring lock "0825f13f-c923-417d-8fa0-f4e61616e4db-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:26:19 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-bd2093be-6357-4a3b-9fbb-df1cd31bcdd1 req-c19bcb00-e1e5-4e34-bb25-ff30cef21aa3 service nova] Lock "0825f13f-c923-417d-8fa0-f4e61616e4db-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:26:19 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-bd2093be-6357-4a3b-9fbb-df1cd31bcdd1 req-c19bcb00-e1e5-4e34-bb25-ff30cef21aa3 service nova] Lock "0825f13f-c923-417d-8fa0-f4e61616e4db-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:26:19 user nova-compute[72351]: DEBUG nova.compute.manager [req-bd2093be-6357-4a3b-9fbb-df1cd31bcdd1 req-c19bcb00-e1e5-4e34-bb25-ff30cef21aa3 service nova] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] No waiting events found dispatching network-vif-plugged-10079897-954d-49a0-9007-51efb10241bc {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:26:19 user nova-compute[72351]: WARNING nova.compute.manager [req-bd2093be-6357-4a3b-9fbb-df1cd31bcdd1 req-c19bcb00-e1e5-4e34-bb25-ff30cef21aa3 service nova] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Received unexpected event network-vif-plugged-10079897-954d-49a0-9007-51efb10241bc for instance with vm_state deleted and task_state None.
Aug 09 02:26:19 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c852f29a-fb98-41e3-a5e9-95b324b4c994 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "585c81a5-ac24-4d12-a8a6-1e7b12a2d53a" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 10.888s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:26:20 user nova-compute[72351]: INFO nova.compute.manager [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Rebuilding instance
Aug 09 02:26:20 user nova-compute[72351]: DEBUG nova.objects.instance [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lazy-loading 'trusted_certs' on Instance uuid 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:26:20 user nova-compute[72351]: DEBUG nova.compute.manager [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:26:20 user nova-compute[72351]: DEBUG nova.objects.instance [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lazy-loading 'pci_requests' on Instance uuid 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:26:20 user nova-compute[72351]: DEBUG nova.objects.instance [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lazy-loading 'pci_devices' on Instance uuid 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:26:20 user nova-compute[72351]: DEBUG nova.objects.instance [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lazy-loading 'resources' on Instance uuid 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:26:20 user nova-compute[72351]: DEBUG nova.objects.instance [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lazy-loading 'migration_context' on Instance uuid 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:26:20 user nova-compute[72351]: DEBUG nova.objects.instance [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Trying to apply a migration context that does not seem to be set for this instance {{(pid=72351) apply_migration_context /opt/stack/nova/nova/objects/instance.py:1067}}
Aug 09 02:26:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:20 user nova-compute[72351]: DEBUG nova.compute.manager [req-d3845099-7273-4e96-a1d1-8656c888c380 req-e09bd1f2-2de8-4437-8181-bf7ec1e30807 service nova] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Received event network-vif-unplugged-e521fae4-093d-4c92-a32d-684c1a5137f5 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:26:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d3845099-7273-4e96-a1d1-8656c888c380 req-e09bd1f2-2de8-4437-8181-bf7ec1e30807 service nova] Acquiring lock "585c81a5-ac24-4d12-a8a6-1e7b12a2d53a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:26:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d3845099-7273-4e96-a1d1-8656c888c380 req-e09bd1f2-2de8-4437-8181-bf7ec1e30807 service nova] Lock "585c81a5-ac24-4d12-a8a6-1e7b12a2d53a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:26:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d3845099-7273-4e96-a1d1-8656c888c380 req-e09bd1f2-2de8-4437-8181-bf7ec1e30807 service nova] Lock "585c81a5-ac24-4d12-a8a6-1e7b12a2d53a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:26:20 user nova-compute[72351]: DEBUG nova.compute.manager [req-d3845099-7273-4e96-a1d1-8656c888c380 req-e09bd1f2-2de8-4437-8181-bf7ec1e30807 service nova] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] No waiting events found dispatching network-vif-unplugged-e521fae4-093d-4c92-a32d-684c1a5137f5 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:26:20 user nova-compute[72351]: WARNING nova.compute.manager [req-d3845099-7273-4e96-a1d1-8656c888c380 req-e09bd1f2-2de8-4437-8181-bf7ec1e30807 service nova] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Received unexpected event network-vif-unplugged-e521fae4-093d-4c92-a32d-684c1a5137f5 for instance with vm_state active and task_state rebuilding.
Aug 09 02:26:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:20 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Instance destroyed successfully.
Aug 09 02:26:21 user nova-compute[72351]: INFO nova.compute.manager [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Detaching volume 40cdcbc5-17b6-490e-8989-3d6ab80bdd80
Aug 09 02:26:21 user nova-compute[72351]: INFO nova.virt.block_device [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Attempting to driver detach volume 40cdcbc5-17b6-490e-8989-3d6ab80bdd80 from mountpoint /dev/vdb
Aug 09 02:26:21 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Attempting to detach device vdb from instance 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a from the persistent domain config. {{(pid=72351) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2477}}
Aug 09 02:26:21 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 02:26:21 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:26:21 user nova-compute[72351]:   <source dev="/dev/sde"/>
Aug 09 02:26:21 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:26:21 user nova-compute[72351]:   <serial>40cdcbc5-17b6-490e-8989-3d6ab80bdd80</serial>
Aug 09 02:26:21 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Aug 09 02:26:21 user nova-compute[72351]: </disk>
Aug 09 02:26:21 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 02:26:21 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Successfully detached device vdb from instance 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a from the persistent domain config.
Aug 09 02:26:21 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 02:26:21 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'dEclGUPX', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sde'}, None), 'kwargs': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:26:21 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release: NotImplementedError
Aug 09 02:26:21 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:26:21 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:26:21 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 02:26:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:21 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[8d964fa0-2d8a-4aa7-a2e4-815de6887ee2]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.030s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:21 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[2a41db83-663f-4f67-a8f1-df9a71f837f8]: (4, ('tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)\ntcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)\ntcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b (non-flash)\ntcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:21 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)
Aug 09 02:26:21 user nova-compute[72351]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:26:21 user nova-compute[72351]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b (non-flash)
Aug 09 02:26:21 user nova-compute[72351]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)
Aug 09 02:26:21 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:26:21 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsi session list stdout=tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)
Aug 09 02:26:21 user nova-compute[72351]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:26:21 user nova-compute[72351]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b (non-flash)
Aug 09 02:26:21 user nova-compute[72351]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)
Aug 09 02:26:21 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:26:21 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fcfbc744d30>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80'): ({'sde'}, set())}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 02:26:21 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Removing single pathed devices sde {{(pid=72351) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Aug 09 02:26:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:21 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[91b09943-02d0-4691-bd56-c3825afe6525]: (4, ('path checker states:\nup                  3\npending             1\n\npaths: 0\nbusy: False\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sde {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sde" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:21 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a5a1e792-935a-4bee-8b65-6d679319bd07]: (4, ('ok\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:21 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Flushing IO for device /dev/sde {{(pid=72351) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Aug 09 02:26:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sde {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sde" returned: 0 in 0.016s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:21 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d6a2dacc-0a28-49f3-a35b-6d6ac3a811e3]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:21 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Remove SCSI device /dev/sde with /sys/block/sde/device/delete {{(pid=72351) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Aug 09 02:26:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sde/device/delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sde/device/delete" returned: 0 in 0.047s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:21 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[7648bbd2-4d6c-4d19-a723-c996b1964728]: (4, ('1', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:21 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Checking to see if SCSI volumes sde have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Aug 09 02:26:21 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] SCSI volumes sde have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Aug 09 02:26:21 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80')] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 02:26:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:21 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ffc629b6-03f0-4a6a-8025-65e1dc4a2aed]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:21 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:26:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 -p 172.16.0.220:3260 --logout {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 -p 172.16.0.220:3260 --logout" returned: 0 in 0.062s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:21 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[666f9703-95a7-4a62-a79d-7c1d5f2267ef]: (4, ('Logging out of session [sid: 23, target: iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80, portal: 172.16.0.220,3260]\nLogout of [sid: 23, target: iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:21 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 23, target: iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80, portal: 172.16.0.220,3260]
Aug 09 02:26:21 user nova-compute[72351]: Logout of [sid: 23, target: iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80, portal: 172.16.0.220,3260] successful.
Aug 09 02:26:21 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:26:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 -p 172.16.0.220:3260 --op delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.025s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:21 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[c89942ec-6640-4ebf-8acb-bbc03811c90e]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:21 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:26:21 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.297s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:26:21 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] <== disconnect_volume: return (297ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:26:21 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 02:26:21 user nova-compute[72351]: INFO nova.compute.manager [None req-3e6e00e1-14e2-4865-a9c3-0d540a6cfe36 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Rescuing
Aug 09 02:26:21 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-3e6e00e1-14e2-4865-a9c3-0d540a6cfe36 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Acquiring lock "refresh_cache-e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:26:21 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-3e6e00e1-14e2-4865-a9c3-0d540a6cfe36 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Acquired lock "refresh_cache-e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:26:21 user nova-compute[72351]: DEBUG nova.network.neutron [None req-3e6e00e1-14e2-4865-a9c3-0d540a6cfe36 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:26:22 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Instance destroyed successfully.
Aug 09 02:26:22 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:22:20Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestOtherA-server-446242587',display_name='tempest-ServerActionsTestOtherA-server-1834248220',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serveractionstestothera-server-446242587',id=19,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBB4HNXglactRKXsG6uozX3XR7A39DBllYpgpCz+5+1RhfLOeXdmVkNfDkIG7vQrgoVNKWK/MwqtjPDLEVURcnWQfOuv4ySYdfEPj4hs6FpDqfJbAa5qX0FAbZQuEg69FVQ==',key_name='tempest-keypair-976839412',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:22:30Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={rebuild='server'},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='69d584c230b94ff8aae722eb2f364c5c',ramdisk_id='',reservation_id='r-x0v0gmsw',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerActionsTestOtherA-1303549305',owner_user_name='tempest-ServerActionsTestOtherA-1303549305-project-member'},tags=<?>,task_state='rebuilding',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:26:20Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='5dca34fdadbb4386a1ff1cb255265d47',uuid=585c81a5-ac24-4d12-a8a6-1e7b12a2d53a,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "e521fae4-093d-4c92-a32d-684c1a5137f5", "address": "fa:16:3e:e5:b0:10", "network": {"id": "b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-210689817-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.41", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "69d584c230b94ff8aae722eb2f364c5c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape521fae4-09", "ovs_interfaceid": "e521fae4-093d-4c92-a32d-684c1a5137f5", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:26:22 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Converting VIF {"id": "e521fae4-093d-4c92-a32d-684c1a5137f5", "address": "fa:16:3e:e5:b0:10", "network": {"id": "b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-210689817-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.41", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "69d584c230b94ff8aae722eb2f364c5c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape521fae4-09", "ovs_interfaceid": "e521fae4-093d-4c92-a32d-684c1a5137f5", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:26:22 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:e5:b0:10,bridge_name='br-int',has_traffic_filtering=True,id=e521fae4-093d-4c92-a32d-684c1a5137f5,network=Network(b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape521fae4-09') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:26:22 user nova-compute[72351]: DEBUG os_vif [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:e5:b0:10,bridge_name='br-int',has_traffic_filtering=True,id=e521fae4-093d-4c92-a32d-684c1a5137f5,network=Network(b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape521fae4-09') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:26:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tape521fae4-09, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:26:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:22 user nova-compute[72351]: INFO os_vif [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:e5:b0:10,bridge_name='br-int',has_traffic_filtering=True,id=e521fae4-093d-4c92-a32d-684c1a5137f5,network=Network(b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape521fae4-09')
Aug 09 02:26:22 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 02:26:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'dEclGUPX', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sde'}, None), 'kwargs': True}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:26:22 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release: NotImplementedError
Aug 09 02:26:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:26:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:26:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 02:26:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:22 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[00fa01cf-21d7-4828-8e41-4478b6924318]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.029s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:22 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[65d27e1d-92ce-4cab-8a5d-0ef93444a460]: (4, ('tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)\ntcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)\ntcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)
Aug 09 02:26:22 user nova-compute[72351]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:26:22 user nova-compute[72351]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b (non-flash)
Aug 09 02:26:22 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:26:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsi session list stdout=tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)
Aug 09 02:26:22 user nova-compute[72351]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:26:22 user nova-compute[72351]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b (non-flash)
Aug 09 02:26:22 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:26:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fcffc1989d0>, {}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 02:26:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Disconnecting from: [] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 02:26:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.062s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:26:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] <== disconnect_volume: return (63ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:26:22 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 02:26:22 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Deleting instance files /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a_del
Aug 09 02:26:22 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Deletion of /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a_del complete
Aug 09 02:26:22 user nova-compute[72351]: DEBUG nova.network.neutron [None req-3e6e00e1-14e2-4865-a9c3-0d540a6cfe36 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Updating instance_info_cache with network_info: [{"id": "9481e457-9c23-4c45-b0bd-1faf63f8b7b8", "address": "fa:16:3e:69:03:30", "network": {"id": "49e8751c-e119-430f-9110-3dc6069b1589", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-899275135-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3aea1e66805146f7a8cd1d97e64a24af", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9481e457-9c", "ovs_interfaceid": "9481e457-9c23-4c45-b0bd-1faf63f8b7b8", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:26:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-3e6e00e1-14e2-4865-a9c3-0d540a6cfe36 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Releasing lock "refresh_cache-e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:26:22 user nova-compute[72351]: INFO nova.virt.block_device [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Booting with volume 40cdcbc5-17b6-490e-8989-3d6ab80bdd80 at /dev/vdb
Aug 09 02:26:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:22 user nova-compute[72351]: DEBUG os_brick.utils [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:26:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:22 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[688aa2b8-9ae3-4c36-9ef2-403d0e94a4ad]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:a52269602ff4\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:22 user nova-compute[72351]: DEBUG nova.compute.manager [req-28c8fb0e-5f35-4231-a5ed-9cf4776c8b7d req-8fc86c1b-5d4f-4bdd-8fe4-d09d0ad8e9e8 service nova] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Received event network-vif-plugged-e521fae4-093d-4c92-a32d-684c1a5137f5 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:26:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-28c8fb0e-5f35-4231-a5ed-9cf4776c8b7d req-8fc86c1b-5d4f-4bdd-8fe4-d09d0ad8e9e8 service nova] Acquiring lock "585c81a5-ac24-4d12-a8a6-1e7b12a2d53a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:26:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-28c8fb0e-5f35-4231-a5ed-9cf4776c8b7d req-8fc86c1b-5d4f-4bdd-8fe4-d09d0ad8e9e8 service nova] Lock "585c81a5-ac24-4d12-a8a6-1e7b12a2d53a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:26:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-28c8fb0e-5f35-4231-a5ed-9cf4776c8b7d req-8fc86c1b-5d4f-4bdd-8fe4-d09d0ad8e9e8 service nova] Lock "585c81a5-ac24-4d12-a8a6-1e7b12a2d53a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:26:22 user nova-compute[72351]: DEBUG nova.compute.manager [req-28c8fb0e-5f35-4231-a5ed-9cf4776c8b7d req-8fc86c1b-5d4f-4bdd-8fe4-d09d0ad8e9e8 service nova] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] No waiting events found dispatching network-vif-plugged-e521fae4-093d-4c92-a32d-684c1a5137f5 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:26:22 user nova-compute[72351]: WARNING nova.compute.manager [req-28c8fb0e-5f35-4231-a5ed-9cf4776c8b7d req-8fc86c1b-5d4f-4bdd-8fe4-d09d0ad8e9e8 service nova] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Received unexpected event network-vif-plugged-e521fae4-093d-4c92-a32d-684c1a5137f5 for instance with vm_state active and task_state rebuild_block_device_mapping.
Aug 09 02:26:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.016s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:22 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[5a22fcfb-b125-4b7a-9f19-3a691b8cb640]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:22 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[4a1af95d-5382-4d40-a0cb-8d4c14188340]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:22 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[3a674bee-11bb-4136-a404-d6e47faedd94]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Running cmd (subprocess): nvme version {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] 'nvme version' failed. Not Retrying. {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:26:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.nvmeof [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] nvme not present on system {{(pid=72351) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Aug 09 02:26:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:26:22 user nova-compute[72351]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Aug 09 02:26:22 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[7d07a077-641e-4aa3-8d83-a16350346823]: (4, '') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72351) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Aug 09 02:26:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Aug 09 02:26:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] LIGHTOS: no hostnqn found. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Aug 09 02:26:22 user nova-compute[72351]: DEBUG os_brick.utils [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] <== get_connector_properties: return (133ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:a52269602ff4', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:26:22 user nova-compute[72351]: DEBUG nova.virt.block_device [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Updating existing volume attachment record: a2c0b5d5-afe8-4219-be0d-eb0752f48e0b {{(pid=72351) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Aug 09 02:26:22 user nova-compute[72351]: DEBUG nova.compute.manager [req-50d289ca-ec98-4be4-9d41-0f03125e10f4 req-73458eb6-7ec6-4c98-bf44-a193a7953f36 service nova] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Received event network-vif-unplugged-9481e457-9c23-4c45-b0bd-1faf63f8b7b8 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:26:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-50d289ca-ec98-4be4-9d41-0f03125e10f4 req-73458eb6-7ec6-4c98-bf44-a193a7953f36 service nova] Acquiring lock "e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:26:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-50d289ca-ec98-4be4-9d41-0f03125e10f4 req-73458eb6-7ec6-4c98-bf44-a193a7953f36 service nova] Lock "e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:26:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-50d289ca-ec98-4be4-9d41-0f03125e10f4 req-73458eb6-7ec6-4c98-bf44-a193a7953f36 service nova] Lock "e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:26:22 user nova-compute[72351]: DEBUG nova.compute.manager [req-50d289ca-ec98-4be4-9d41-0f03125e10f4 req-73458eb6-7ec6-4c98-bf44-a193a7953f36 service nova] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] No waiting events found dispatching network-vif-unplugged-9481e457-9c23-4c45-b0bd-1faf63f8b7b8 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:26:22 user nova-compute[72351]: WARNING nova.compute.manager [req-50d289ca-ec98-4be4-9d41-0f03125e10f4 req-73458eb6-7ec6-4c98-bf44-a193a7953f36 service nova] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Received unexpected event network-vif-unplugged-9481e457-9c23-4c45-b0bd-1faf63f8b7b8 for instance with vm_state active and task_state rescuing.
Aug 09 02:26:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:23 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Instance destroyed successfully.
Aug 09 02:26:23 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-3e6e00e1-14e2-4865-a9c3-0d540a6cfe36 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Attempting rescue
Aug 09 02:26:23 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-3e6e00e1-14e2-4865-a9c3-0d540a6cfe36 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] rescue generated disk_info: {'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'disk.rescue': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} {{(pid=72351) rescue /opt/stack/nova/nova/virt/libvirt/driver.py:4308}}
Aug 09 02:26:23 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-3e6e00e1-14e2-4865-a9c3-0d540a6cfe36 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Instance directory exists: not creating {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4713}}
Aug 09 02:26:23 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-3e6e00e1-14e2-4865-a9c3-0d540a6cfe36 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Creating image(s)
Aug 09 02:26:23 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-3e6e00e1-14e2-4865-a9c3-0d540a6cfe36 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Acquiring lock "/opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:26:23 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-3e6e00e1-14e2-4865-a9c3-0d540a6cfe36 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "/opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:26:23 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-3e6e00e1-14e2-4865-a9c3-0d540a6cfe36 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "/opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:26:23 user nova-compute[72351]: DEBUG nova.objects.instance [None req-3e6e00e1-14e2-4865-a9c3-0d540a6cfe36 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lazy-loading 'trusted_certs' on Instance uuid e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:26:23 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-3e6e00e1-14e2-4865-a9c3-0d540a6cfe36 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Acquiring lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:26:23 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-3e6e00e1-14e2-4865-a9c3-0d540a6cfe36 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:26:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-3e6e00e1-14e2-4865-a9c3-0d540a6cfe36 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-3e6e00e1-14e2-4865-a9c3-0d540a6cfe36 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.129s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-3e6e00e1-14e2-4865-a9c3-0d540a6cfe36 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk.rescue {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-3e6e00e1-14e2-4865-a9c3-0d540a6cfe36 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk.rescue" returned: 0 in 0.049s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:23 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-3e6e00e1-14e2-4865-a9c3-0d540a6cfe36 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.186s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:26:23 user nova-compute[72351]: DEBUG nova.objects.instance [None req-3e6e00e1-14e2-4865-a9c3-0d540a6cfe36 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lazy-loading 'migration_context' on Instance uuid e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:26:23 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-3e6e00e1-14e2-4865-a9c3-0d540a6cfe36 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Created local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4851}}
Aug 09 02:26:23 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-3e6e00e1-14e2-4865-a9c3-0d540a6cfe36 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Start _get_guest_xml network_info=[{"id": "9481e457-9c23-4c45-b0bd-1faf63f8b7b8", "address": "fa:16:3e:69:03:30", "network": {"id": "49e8751c-e119-430f-9110-3dc6069b1589", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-899275135-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [], "label": "tempest-ServerRescueNegativeTestJSON-899275135-network", "vif_mac": "fa:16:3e:69:03:30"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3aea1e66805146f7a8cd1d97e64a24af", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9481e457-9c", "ovs_interfaceid": "9481e457-9c23-4c45-b0bd-1faf63f8b7b8", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'disk.rescue': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>) rescue={'image_id': '18ccda7f-31d3-4978-ab9d-4ed2f74eff94', 'kernel_id': '', 'ramdisk_id': ''} block_device_info=None {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:26:23 user nova-compute[72351]: DEBUG nova.objects.instance [None req-3e6e00e1-14e2-4865-a9c3-0d540a6cfe36 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lazy-loading 'resources' on Instance uuid e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:26:23 user nova-compute[72351]: DEBUG nova.objects.instance [None req-3e6e00e1-14e2-4865-a9c3-0d540a6cfe36 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lazy-loading 'numa_topology' on Instance uuid e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:26:23 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-3e6e00e1-14e2-4865-a9c3-0d540a6cfe36 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:26:23 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-3e6e00e1-14e2-4865-a9c3-0d540a6cfe36 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:26:23 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-3e6e00e1-14e2-4865-a9c3-0d540a6cfe36 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:26:23 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-3e6e00e1-14e2-4865-a9c3-0d540a6cfe36 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:26:23 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-3e6e00e1-14e2-4865-a9c3-0d540a6cfe36 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:26:23 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-3e6e00e1-14e2-4865-a9c3-0d540a6cfe36 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:26:23 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-3e6e00e1-14e2-4865-a9c3-0d540a6cfe36 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:26:23 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-3e6e00e1-14e2-4865-a9c3-0d540a6cfe36 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:26:23 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-3e6e00e1-14e2-4865-a9c3-0d540a6cfe36 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:26:23 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-3e6e00e1-14e2-4865-a9c3-0d540a6cfe36 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:26:23 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-3e6e00e1-14e2-4865-a9c3-0d540a6cfe36 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:26:23 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-3e6e00e1-14e2-4865-a9c3-0d540a6cfe36 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:26:23 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-3e6e00e1-14e2-4865-a9c3-0d540a6cfe36 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:26:23 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-3e6e00e1-14e2-4865-a9c3-0d540a6cfe36 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:26:23 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-3e6e00e1-14e2-4865-a9c3-0d540a6cfe36 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:26:23 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-3e6e00e1-14e2-4865-a9c3-0d540a6cfe36 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:26:23 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-3e6e00e1-14e2-4865-a9c3-0d540a6cfe36 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:26:23 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-3e6e00e1-14e2-4865-a9c3-0d540a6cfe36 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:26:23 user nova-compute[72351]: DEBUG nova.objects.instance [None req-3e6e00e1-14e2-4865-a9c3-0d540a6cfe36 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lazy-loading 'vcpu_model' on Instance uuid e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:26:23 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-3e6e00e1-14e2-4865-a9c3-0d540a6cfe36 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:21:34Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerRescueNegativeTestJSON-server-938229174',display_name='tempest-ServerRescueNegativeTestJSON-server-938229174',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serverrescuenegativetestjson-server-938229174',id=16,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:21:44Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='3aea1e66805146f7a8cd1d97e64a24af',ramdisk_id='',reservation_id='r-14yazlt3',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerRescueNegativeTestJSON-289364438',owner_user_name='tempest-ServerRescueNegativeTestJSON-289364438-project-member'},tags=<?>,task_state='rescuing',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:21:45Z,user_data=None,user_id='419b28d5a46c481db435715a30ab2614',uuid=e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "9481e457-9c23-4c45-b0bd-1faf63f8b7b8", "address": "fa:16:3e:69:03:30", "network": {"id": "49e8751c-e119-430f-9110-3dc6069b1589", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-899275135-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [], "label": "tempest-ServerRescueNegativeTestJSON-899275135-network", "vif_mac": "fa:16:3e:69:03:30"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3aea1e66805146f7a8cd1d97e64a24af", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9481e457-9c", "ovs_interfaceid": "9481e457-9c23-4c45-b0bd-1faf63f8b7b8", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:26:23 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-3e6e00e1-14e2-4865-a9c3-0d540a6cfe36 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Converting VIF {"id": "9481e457-9c23-4c45-b0bd-1faf63f8b7b8", "address": "fa:16:3e:69:03:30", "network": {"id": "49e8751c-e119-430f-9110-3dc6069b1589", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-899275135-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [], "label": "tempest-ServerRescueNegativeTestJSON-899275135-network", "vif_mac": "fa:16:3e:69:03:30"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3aea1e66805146f7a8cd1d97e64a24af", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9481e457-9c", "ovs_interfaceid": "9481e457-9c23-4c45-b0bd-1faf63f8b7b8", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:26:23 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-3e6e00e1-14e2-4865-a9c3-0d540a6cfe36 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:69:03:30,bridge_name='br-int',has_traffic_filtering=True,id=9481e457-9c23-4c45-b0bd-1faf63f8b7b8,network=Network(49e8751c-e119-430f-9110-3dc6069b1589),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap9481e457-9c') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:26:23 user nova-compute[72351]: DEBUG nova.objects.instance [None req-3e6e00e1-14e2-4865-a9c3-0d540a6cfe36 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lazy-loading 'pci_devices' on Instance uuid e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:26:23 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-3e6e00e1-14e2-4865-a9c3-0d540a6cfe36 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:26:23 user nova-compute[72351]:   <uuid>e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335</uuid>
Aug 09 02:26:23 user nova-compute[72351]:   <name>instance-00000010</name>
Aug 09 02:26:23 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:26:23 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:26:23 user nova-compute[72351]:   <metadata>
Aug 09 02:26:23 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:26:23 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:26:23 user nova-compute[72351]:       <nova:name>tempest-ServerRescueNegativeTestJSON-server-938229174</nova:name>
Aug 09 02:26:23 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:26:23</nova:creationTime>
Aug 09 02:26:23 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:26:23 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:26:23 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:26:23 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:26:23 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:26:23 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:26:23 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:26:23 user nova-compute[72351]:       <nova:owner>
Aug 09 02:26:23 user nova-compute[72351]:         <nova:user uuid="419b28d5a46c481db435715a30ab2614">tempest-ServerRescueNegativeTestJSON-289364438-project-member</nova:user>
Aug 09 02:26:23 user nova-compute[72351]:         <nova:project uuid="3aea1e66805146f7a8cd1d97e64a24af">tempest-ServerRescueNegativeTestJSON-289364438</nova:project>
Aug 09 02:26:23 user nova-compute[72351]:       </nova:owner>
Aug 09 02:26:23 user nova-compute[72351]:       <nova:root type="image" uuid="18ccda7f-31d3-4978-ab9d-4ed2f74eff94"/>
Aug 09 02:26:23 user nova-compute[72351]:       <nova:ports>
Aug 09 02:26:23 user nova-compute[72351]:         <nova:port uuid="9481e457-9c23-4c45-b0bd-1faf63f8b7b8">
Aug 09 02:26:23 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.12" ipVersion="4"/>
Aug 09 02:26:23 user nova-compute[72351]:         </nova:port>
Aug 09 02:26:23 user nova-compute[72351]:       </nova:ports>
Aug 09 02:26:23 user nova-compute[72351]:     </nova:instance>
Aug 09 02:26:23 user nova-compute[72351]:   </metadata>
Aug 09 02:26:23 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:26:23 user nova-compute[72351]:     <system>
Aug 09 02:26:23 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:26:23 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:26:23 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:26:23 user nova-compute[72351]:       <entry name="serial">e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335</entry>
Aug 09 02:26:23 user nova-compute[72351]:       <entry name="uuid">e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335</entry>
Aug 09 02:26:23 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:26:23 user nova-compute[72351]:     </system>
Aug 09 02:26:23 user nova-compute[72351]:   </sysinfo>
Aug 09 02:26:23 user nova-compute[72351]:   <os>
Aug 09 02:26:23 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:26:23 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:26:23 user nova-compute[72351]:   </os>
Aug 09 02:26:23 user nova-compute[72351]:   <features>
Aug 09 02:26:23 user nova-compute[72351]:     <acpi/>
Aug 09 02:26:23 user nova-compute[72351]:     <apic/>
Aug 09 02:26:23 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:26:23 user nova-compute[72351]:   </features>
Aug 09 02:26:23 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:26:23 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:26:23 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:26:23 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:26:23 user nova-compute[72351]:   </clock>
Aug 09 02:26:23 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:26:23 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:26:23 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:26:23 user nova-compute[72351]:   </cpu>
Aug 09 02:26:23 user nova-compute[72351]:   <devices>
Aug 09 02:26:23 user nova-compute[72351]:     <disk type="file" device="disk">
Aug 09 02:26:23 user nova-compute[72351]:       <driver name="qemu" type="qcow2" cache="none"/>
Aug 09 02:26:23 user nova-compute[72351]:       <source file="/opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk.rescue"/>
Aug 09 02:26:23 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:26:23 user nova-compute[72351]:     </disk>
Aug 09 02:26:23 user nova-compute[72351]:     <disk type="file" device="disk">
Aug 09 02:26:23 user nova-compute[72351]:       <driver name="qemu" type="qcow2" cache="none"/>
Aug 09 02:26:23 user nova-compute[72351]:       <source file="/opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk"/>
Aug 09 02:26:23 user nova-compute[72351]:       <target dev="vdb" bus="virtio"/>
Aug 09 02:26:23 user nova-compute[72351]:     </disk>
Aug 09 02:26:23 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:26:23 user nova-compute[72351]:       <mac address="fa:16:3e:69:03:30"/>
Aug 09 02:26:23 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:26:23 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:26:23 user nova-compute[72351]:       <target dev="tap9481e457-9c"/>
Aug 09 02:26:23 user nova-compute[72351]:     </interface>
Aug 09 02:26:23 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:26:23 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/console.log" append="off"/>
Aug 09 02:26:23 user nova-compute[72351]:     </serial>
Aug 09 02:26:23 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:26:23 user nova-compute[72351]:     <video>
Aug 09 02:26:23 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:26:23 user nova-compute[72351]:     </video>
Aug 09 02:26:23 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:26:23 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:26:23 user nova-compute[72351]:     </rng>
Aug 09 02:26:23 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:26:23 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:26:23 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:26:23 user nova-compute[72351]:     </memballoon>
Aug 09 02:26:23 user nova-compute[72351]:   </devices>
Aug 09 02:26:23 user nova-compute[72351]: </domain>
Aug 09 02:26:23 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:26:23 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Instance destroyed successfully.
Aug 09 02:26:23 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-3e6e00e1-14e2-4865-a9c3-0d540a6cfe36 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:26:23 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-3e6e00e1-14e2-4865-a9c3-0d540a6cfe36 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:26:23 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-3e6e00e1-14e2-4865-a9c3-0d540a6cfe36 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] No VIF found with MAC fa:16:3e:69:03:30, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:26:24 user nova-compute[72351]: DEBUG nova.compute.manager [req-447bbe8e-5231-4d37-8536-beef3c0c02ab req-f934a9b1-cb2b-4d33-9740-04544a82c9e7 service nova] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Received event network-vif-plugged-9481e457-9c23-4c45-b0bd-1faf63f8b7b8 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:26:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-447bbe8e-5231-4d37-8536-beef3c0c02ab req-f934a9b1-cb2b-4d33-9740-04544a82c9e7 service nova] Acquiring lock "e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:26:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-447bbe8e-5231-4d37-8536-beef3c0c02ab req-f934a9b1-cb2b-4d33-9740-04544a82c9e7 service nova] Lock "e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:26:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-447bbe8e-5231-4d37-8536-beef3c0c02ab req-f934a9b1-cb2b-4d33-9740-04544a82c9e7 service nova] Lock "e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:26:24 user nova-compute[72351]: DEBUG nova.compute.manager [req-447bbe8e-5231-4d37-8536-beef3c0c02ab req-f934a9b1-cb2b-4d33-9740-04544a82c9e7 service nova] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] No waiting events found dispatching network-vif-plugged-9481e457-9c23-4c45-b0bd-1faf63f8b7b8 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:26:24 user nova-compute[72351]: WARNING nova.compute.manager [req-447bbe8e-5231-4d37-8536-beef3c0c02ab req-f934a9b1-cb2b-4d33-9740-04544a82c9e7 service nova] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Received unexpected event network-vif-plugged-9481e457-9c23-4c45-b0bd-1faf63f8b7b8 for instance with vm_state active and task_state rescuing.
Aug 09 02:26:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Acquiring lock "957c1510-1320-4492-9052-a58ff540319d" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:26:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "957c1510-1320-4492-9052-a58ff540319d" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:26:25 user nova-compute[72351]: DEBUG nova.compute.manager [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 957c1510-1320-4492-9052-a58ff540319d] Starting instance... {{(pid=72351) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2406}}
Aug 09 02:26:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:26:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.003s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:26:25 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72351) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Aug 09 02:26:25 user nova-compute[72351]: INFO nova.compute.claims [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 957c1510-1320-4492-9052-a58ff540319d] Claim successful on node user
Aug 09 02:26:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:25 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:26:25 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:26:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.502s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:26:25 user nova-compute[72351]: DEBUG nova.compute.manager [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 957c1510-1320-4492-9052-a58ff540319d] Start building networks asynchronously for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2803}}
Aug 09 02:26:26 user nova-compute[72351]: DEBUG nova.compute.manager [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 957c1510-1320-4492-9052-a58ff540319d] Allocating IP information in the background. {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1956}}
Aug 09 02:26:26 user nova-compute[72351]: DEBUG nova.network.neutron [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 957c1510-1320-4492-9052-a58ff540319d] allocate_for_instance() {{(pid=72351) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Aug 09 02:26:26 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 957c1510-1320-4492-9052-a58ff540319d] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Aug 09 02:26:26 user nova-compute[72351]: DEBUG nova.compute.manager [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 957c1510-1320-4492-9052-a58ff540319d] Start building block device mappings for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2838}}
Aug 09 02:26:26 user nova-compute[72351]: INFO nova.virt.block_device [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 957c1510-1320-4492-9052-a58ff540319d] Booting with blank volume at /dev/vda
Aug 09 02:26:26 user nova-compute[72351]: DEBUG nova.policy [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '73d05c48be5640c28455e8ba8b05ef31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '321111f935dd4e8aab4a083206ea9a7e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72351) authorize /opt/stack/nova/nova/policy.py:203}}
Aug 09 02:26:26 user nova-compute[72351]: DEBUG nova.compute.manager [req-44ff0d2b-0527-4125-ae9f-961927d6a4b6 req-1db7d14f-35ad-4838-95bf-b59672c37de8 service nova] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Received event network-vif-plugged-9481e457-9c23-4c45-b0bd-1faf63f8b7b8 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:26:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-44ff0d2b-0527-4125-ae9f-961927d6a4b6 req-1db7d14f-35ad-4838-95bf-b59672c37de8 service nova] Acquiring lock "e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:26:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-44ff0d2b-0527-4125-ae9f-961927d6a4b6 req-1db7d14f-35ad-4838-95bf-b59672c37de8 service nova] Lock "e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:26:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-44ff0d2b-0527-4125-ae9f-961927d6a4b6 req-1db7d14f-35ad-4838-95bf-b59672c37de8 service nova] Lock "e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:26:26 user nova-compute[72351]: DEBUG nova.compute.manager [req-44ff0d2b-0527-4125-ae9f-961927d6a4b6 req-1db7d14f-35ad-4838-95bf-b59672c37de8 service nova] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] No waiting events found dispatching network-vif-plugged-9481e457-9c23-4c45-b0bd-1faf63f8b7b8 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:26:26 user nova-compute[72351]: WARNING nova.compute.manager [req-44ff0d2b-0527-4125-ae9f-961927d6a4b6 req-1db7d14f-35ad-4838-95bf-b59672c37de8 service nova] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Received unexpected event network-vif-plugged-9481e457-9c23-4c45-b0bd-1faf63f8b7b8 for instance with vm_state active and task_state rescuing.
Aug 09 02:26:26 user nova-compute[72351]: DEBUG nova.compute.manager [req-44ff0d2b-0527-4125-ae9f-961927d6a4b6 req-1db7d14f-35ad-4838-95bf-b59672c37de8 service nova] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Received event network-vif-plugged-9481e457-9c23-4c45-b0bd-1faf63f8b7b8 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:26:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-44ff0d2b-0527-4125-ae9f-961927d6a4b6 req-1db7d14f-35ad-4838-95bf-b59672c37de8 service nova] Acquiring lock "e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:26:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-44ff0d2b-0527-4125-ae9f-961927d6a4b6 req-1db7d14f-35ad-4838-95bf-b59672c37de8 service nova] Lock "e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:26:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-44ff0d2b-0527-4125-ae9f-961927d6a4b6 req-1db7d14f-35ad-4838-95bf-b59672c37de8 service nova] Lock "e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:26:26 user nova-compute[72351]: DEBUG nova.compute.manager [req-44ff0d2b-0527-4125-ae9f-961927d6a4b6 req-1db7d14f-35ad-4838-95bf-b59672c37de8 service nova] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] No waiting events found dispatching network-vif-plugged-9481e457-9c23-4c45-b0bd-1faf63f8b7b8 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:26:26 user nova-compute[72351]: WARNING nova.compute.manager [req-44ff0d2b-0527-4125-ae9f-961927d6a4b6 req-1db7d14f-35ad-4838-95bf-b59672c37de8 service nova] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Received unexpected event network-vif-plugged-9481e457-9c23-4c45-b0bd-1faf63f8b7b8 for instance with vm_state active and task_state rescuing.
Aug 09 02:26:27 user nova-compute[72351]: DEBUG nova.network.neutron [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 957c1510-1320-4492-9052-a58ff540319d] Successfully created port: e679cfd9-123f-4b88-8fd2-397e4c8616cc {{(pid=72351) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Aug 09 02:26:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:27 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Removed pending event for e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335 due to event {{(pid=72351) _event_emit_delayed /opt/stack/nova/nova/virt/libvirt/host.py:438}}
Aug 09 02:26:27 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547987.3503838, e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335 => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:26:27 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] VM Resumed (Lifecycle Event)
Aug 09 02:26:27 user nova-compute[72351]: DEBUG nova.compute.manager [None req-3e6e00e1-14e2-4865-a9c3-0d540a6cfe36 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:26:27 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:26:27 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: active, current task_state: rescuing, current DB power_state: 1, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:26:27 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] During sync_power_state the instance has a pending task (rescuing). Skip.
Aug 09 02:26:27 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547987.3525147, e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335 => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:26:27 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] VM Started (Lifecycle Event)
Aug 09 02:26:27 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:26:27 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Synchronizing instance power state after lifecycle event "Started"; current vm_state: active, current task_state: rescuing, current DB power_state: 1, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:26:27 user nova-compute[72351]: DEBUG nova.network.neutron [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 957c1510-1320-4492-9052-a58ff540319d] Successfully updated port: e679cfd9-123f-4b88-8fd2-397e4c8616cc {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:26:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:27 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Acquiring lock "refresh_cache-957c1510-1320-4492-9052-a58ff540319d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:26:27 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Acquired lock "refresh_cache-957c1510-1320-4492-9052-a58ff540319d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:26:27 user nova-compute[72351]: DEBUG nova.network.neutron [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 957c1510-1320-4492-9052-a58ff540319d] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:26:27 user nova-compute[72351]: DEBUG nova.compute.manager [req-ec7006f2-a393-4c8b-866f-155bfdeda7ca req-1fc5717a-93c0-4d1f-b3d6-e67e5c4e72d0 service nova] [instance: 957c1510-1320-4492-9052-a58ff540319d] Received event network-changed-e679cfd9-123f-4b88-8fd2-397e4c8616cc {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:26:27 user nova-compute[72351]: DEBUG nova.compute.manager [req-ec7006f2-a393-4c8b-866f-155bfdeda7ca req-1fc5717a-93c0-4d1f-b3d6-e67e5c4e72d0 service nova] [instance: 957c1510-1320-4492-9052-a58ff540319d] Refreshing instance network info cache due to event network-changed-e679cfd9-123f-4b88-8fd2-397e4c8616cc. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:26:27 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ec7006f2-a393-4c8b-866f-155bfdeda7ca req-1fc5717a-93c0-4d1f-b3d6-e67e5c4e72d0 service nova] Acquiring lock "refresh_cache-957c1510-1320-4492-9052-a58ff540319d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:26:28 user nova-compute[72351]: DEBUG nova.network.neutron [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 957c1510-1320-4492-9052-a58ff540319d] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:26:28 user nova-compute[72351]: DEBUG nova.network.neutron [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 957c1510-1320-4492-9052-a58ff540319d] Updating instance_info_cache with network_info: [{"id": "e679cfd9-123f-4b88-8fd2-397e4c8616cc", "address": "fa:16:3e:7f:d6:1b", "network": {"id": "9a452414-d61f-4045-a6cc-9be93c892c4c", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "321111f935dd4e8aab4a083206ea9a7e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape679cfd9-12", "ovs_interfaceid": "e679cfd9-123f-4b88-8fd2-397e4c8616cc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:26:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Releasing lock "refresh_cache-957c1510-1320-4492-9052-a58ff540319d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:26:28 user nova-compute[72351]: DEBUG nova.compute.manager [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 957c1510-1320-4492-9052-a58ff540319d] Instance network_info: |[{"id": "e679cfd9-123f-4b88-8fd2-397e4c8616cc", "address": "fa:16:3e:7f:d6:1b", "network": {"id": "9a452414-d61f-4045-a6cc-9be93c892c4c", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "321111f935dd4e8aab4a083206ea9a7e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape679cfd9-12", "ovs_interfaceid": "e679cfd9-123f-4b88-8fd2-397e4c8616cc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1971}}
Aug 09 02:26:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ec7006f2-a393-4c8b-866f-155bfdeda7ca req-1fc5717a-93c0-4d1f-b3d6-e67e5c4e72d0 service nova] Acquired lock "refresh_cache-957c1510-1320-4492-9052-a58ff540319d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:26:28 user nova-compute[72351]: DEBUG nova.network.neutron [req-ec7006f2-a393-4c8b-866f-155bfdeda7ca req-1fc5717a-93c0-4d1f-b3d6-e67e5c4e72d0 service nova] [instance: 957c1510-1320-4492-9052-a58ff540319d] Refreshing network info cache for port e679cfd9-123f-4b88-8fd2-397e4c8616cc {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:26:28 user nova-compute[72351]: DEBUG nova.network.neutron [req-ec7006f2-a393-4c8b-866f-155bfdeda7ca req-1fc5717a-93c0-4d1f-b3d6-e67e5c4e72d0 service nova] [instance: 957c1510-1320-4492-9052-a58ff540319d] Updated VIF entry in instance network info cache for port e679cfd9-123f-4b88-8fd2-397e4c8616cc. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:26:28 user nova-compute[72351]: DEBUG nova.network.neutron [req-ec7006f2-a393-4c8b-866f-155bfdeda7ca req-1fc5717a-93c0-4d1f-b3d6-e67e5c4e72d0 service nova] [instance: 957c1510-1320-4492-9052-a58ff540319d] Updating instance_info_cache with network_info: [{"id": "e679cfd9-123f-4b88-8fd2-397e4c8616cc", "address": "fa:16:3e:7f:d6:1b", "network": {"id": "9a452414-d61f-4045-a6cc-9be93c892c4c", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "321111f935dd4e8aab4a083206ea9a7e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape679cfd9-12", "ovs_interfaceid": "e679cfd9-123f-4b88-8fd2-397e4c8616cc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:26:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ec7006f2-a393-4c8b-866f-155bfdeda7ca req-1fc5717a-93c0-4d1f-b3d6-e67e5c4e72d0 service nova] Releasing lock "refresh_cache-957c1510-1320-4492-9052-a58ff540319d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-2b86b241-76fc-40de-9421-4e176d6576d6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-2b86b241-76fc-40de-9421-4e176d6576d6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'HmkUMfVE', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:26:31 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-2b86b241-76fc-40de-9421-4e176d6576d6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:26:31 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-2b86b241-76fc-40de-9421-4e176d6576d6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-2b86b241-76fc-40de-9421-4e176d6576d6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:26:31 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-2b86b241-76fc-40de-9421-4e176d6576d6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:26:31 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-2b86b241-76fc-40de-9421-4e176d6576d6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-2b86b241-76fc-40de-9421-4e176d6576d6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:26:31 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-2b86b241-76fc-40de-9421-4e176d6576d6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 02:26:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e -p 172.16.0.220:3260" returned: 21 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[0e303371-3abf-4222-bee2-e18ea46c0a7a]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-2b86b241-76fc-40de-9421-4e176d6576d6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Aug 09 02:26:31 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e -p 172.16.0.220:3260 --interface default --op new {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.024s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ea2f2422-7655-4c60-8466-0e75181be451]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e] added\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-2b86b241-76fc-40de-9421-4e176d6576d6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e] added
Aug 09 02:26:31 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[8ee5edcc-5014-4ae5-adc2-681810a86978]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-2b86b241-76fc-40de-9421-4e176d6576d6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.026s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[920f7d2f-ffe7-4c98-b86c-5cfcf1556356]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-2b86b241-76fc-40de-9421-4e176d6576d6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e -p 172.16.0.220:3260 --op update -n node.session.auth.username -v HmkUMfVE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e -p 172.16.0.220:3260 --op update -n node.session.auth.username -v HmkUMfVE" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[afc881e2-872d-413a-9a48-01a049327766]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-2b86b241-76fc-40de-9421-4e176d6576d6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'HmkUMfVE'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.025s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[5f7a0da8-9a4b-4866-8c1b-e65cd462d557]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-2b86b241-76fc-40de-9421-4e176d6576d6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[faa2d268-5cad-4968-94da-00cd3e43f9d2]: (4, ('tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)\ntcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)\ntcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-2b86b241-76fc-40de-9421-4e176d6576d6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)
Aug 09 02:26:31 user nova-compute[72351]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:26:31 user nova-compute[72351]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b (non-flash)
Aug 09 02:26:31 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-2b86b241-76fc-40de-9421-4e176d6576d6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] iscsi session list stdout=tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)
Aug 09 02:26:31 user nova-compute[72351]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:26:31 user nova-compute[72351]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b (non-flash)
Aug 09 02:26:31 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e -p 172.16.0.220:3260 --login {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e -p 172.16.0.220:3260 --login" returned: 0 in 0.081s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e199965a-b3da-45dd-a01a-7e705437bbb1]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-2b86b241-76fc-40de-9421-4e176d6576d6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e, portal: 172.16.0.220,3260]
Aug 09 02:26:31 user nova-compute[72351]: Login to [iface: default, target: iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e, portal: 172.16.0.220,3260] successful.
Aug 09 02:26:31 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[6219eefc-efff-421e-9103-2fcbd3fb2aba]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-2b86b241-76fc-40de-9421-4e176d6576d6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.031s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ec2e1fdf-4cef-4f0c-972c-d8065593923e]: (4, ('tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)\ntcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)\ntcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b (non-flash)\ntcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-2b86b241-76fc-40de-9421-4e176d6576d6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)
Aug 09 02:26:31 user nova-compute[72351]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:26:31 user nova-compute[72351]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b (non-flash)
Aug 09 02:26:31 user nova-compute[72351]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)
Aug 09 02:26:31 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-2b86b241-76fc-40de-9421-4e176d6576d6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] iscsi session list stdout=tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)
Aug 09 02:26:31 user nova-compute[72351]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:26:31 user nova-compute[72351]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b (non-flash)
Aug 09 02:26:31 user nova-compute[72351]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)
Aug 09 02:26:31 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-2b86b241-76fc-40de-9421-4e176d6576d6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.351s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-2b86b241-76fc-40de-9421-4e176d6576d6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-2b86b241-76fc-40de-9421-4e176d6576d6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] HCTL ('36', '-', '-', 0) found on session 24 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-2b86b241-76fc-40de-9421-4e176d6576d6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Scanning host 36 c: -, t: -, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host36/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host36/scan" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[22cec07f-2a67-4c71-98fd-05f21b8bee9d]: (4, ('- - 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-2b86b241-76fc-40de-9421-4e176d6576d6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Searching for a device in session 24 and hctl ['36', '*', '*', 0] yield: sde {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-2b86b241-76fc-40de-9421-4e176d6576d6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Connected to sde using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'HmkUMfVE', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Creating instance directory {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4717}}
Aug 09 02:26:31 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Creating image(s)
Aug 09 02:26:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Acquiring lock "/opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "/opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "/opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG os_brick.utils [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.013s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d934e06d-bef7-45b6-a47d-67dc192fc02a]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:a52269602ff4\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.161s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Acquiring lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:26:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.016s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e35e1962-560c-435c-b84b-d81725fa806c]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[83194061-25e9-484c-ba59-275469c33b54]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[49abde32-a484-4c15-8684-b601cfefb67b]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Running cmd (subprocess): nvme version {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] 'nvme version' failed. Not Retrying. {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.nvmeof [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] nvme not present on system {{(pid=72351) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:26:32 user nova-compute[72351]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Aug 09 02:26:32 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[027ad8d6-99a3-46d3-ac52-6696856f211e]: (4, '') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72351) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] LIGHTOS: no hostnqn found. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG os_brick.utils [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] <== get_connector_properties: return (186ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:a52269602ff4', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG nova.virt.block_device [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 957c1510-1320-4492-9052-a58ff540319d] Updating existing volume attachment record: d39213a9-ff03-4a4b-a1f7-78783e8e17c7 {{(pid=72351) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.170s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk 1073741824 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk 1073741824" returned: 0 in 0.062s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.236s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.166s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Checking if we can resize image /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk. size=1073741824 {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-2b86b241-76fc-40de-9421-4e176d6576d6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.399s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-2b86b241-76fc-40de-9421-4e176d6576d6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] <== connect_volume: return (1403ms) {'type': 'block', 'scsi_wwn': '26232323332383862', 'path': '/dev/sde'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-2b86b241-76fc-40de-9421-4e176d6576d6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26232323332383862', 'path': '/dev/sde'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG nova.objects.instance [None req-2b86b241-76fc-40de-9421-4e176d6576d6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lazy-loading 'flavor' on Instance uuid 831c864a-75d9-42b3-b952-749f4201e40e {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json" returned: 0 in 0.152s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Cannot resize image /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk to a smaller size. {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Created local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4851}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Ensure instance console log exists: /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/console.log {{(pid=72351) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4603}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Start _get_guest_xml network_info=[{"id": "e521fae4-093d-4c92-a32d-684c1a5137f5", "address": "fa:16:3e:e5:b0:10", "network": {"id": "b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-210689817-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.41", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "69d584c230b94ff8aae722eb2f364c5c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape521fae4-09", "ovs_interfaceid": "e521fae4-093d-4c92-a32d-684c1a5137f5", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vdb': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'size': 0, 'device_type': 'disk', 'encryption_secret_uuid': None, 'guest_format': None, 'encryption_options': None, 'disk_bus': 'virtio', 'encrypted': False, 'encryption_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'image_id': '18ccda7f-31d3-4978-ab9d-4ed2f74eff94'}], 'ephemerals': [], 'block_device_mapping': [{'device_type': 'disk', 'guest_format': None, 'disk_bus': 'virtio', 'mount_device': '/dev/vdb', 'boot_index': None, 'attachment_id': 'a2c0b5d5-afe8-4219-be0d-eb0752f48e0b', 'delete_on_termination': False, 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqns': ['iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'dEclGUPX', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '585c81a5-ac24-4d12-a8a6-1e7b12a2d53a', 'attached_at': '', 'detached_at': '', 'volume_id': '40cdcbc5-17b6-490e-8989-3d6ab80bdd80', 'serial': '40cdcbc5-17b6-490e-8989-3d6ab80bdd80'}, 'volume_type': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-2b86b241-76fc-40de-9421-4e176d6576d6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] attach device xml: <disk type="block" device="disk">
Aug 09 02:26:32 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:26:32 user nova-compute[72351]:   <source dev="/dev/sde"/>
Aug 09 02:26:32 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:26:32 user nova-compute[72351]:   <serial>5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e</serial>
Aug 09 02:26:32 user nova-compute[72351]: </disk>
Aug 09 02:26:32 user nova-compute[72351]:  {{(pid=72351) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Aug 09 02:26:32 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.: NotImplementedError
Aug 09 02:26:32 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.: NotImplementedError
Aug 09 02:26:32 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG nova.objects.instance [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lazy-loading 'vcpu_model' on Instance uuid 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691547977.603854, 0825f13f-c923-417d-8fa0-f4e61616e4db => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:26:32 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] VM Stopped (Lifecycle Event)
Aug 09 02:26:32 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'dEclGUPX', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:26:32 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release: NotImplementedError
Aug 09 02:26:32 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:26:32 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release: NotImplementedError
Aug 09 02:26:32 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:26:32 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 02:26:32 user nova-compute[72351]: DEBUG nova.compute.manager [None req-3d6c760a-58b1-45d7-aca6-dd1a6d5b6d3a None None] [instance: 0825f13f-c923-417d-8fa0-f4e61616e4db] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 -p 172.16.0.220:3260" returned: 21 in 0.024s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[524501e3-fe9e-4734-8f49-febb8201f254]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Aug 09 02:26:32 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 -p 172.16.0.220:3260 --interface default --op new {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[9627245e-1266-40f9-b83b-2c61a3df7234]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80] added\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80] added
Aug 09 02:26:32 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[8a68e77a-2d89-48d7-800b-eafaec1a752b]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.025s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[49201d49-a1dc-4b38-a19c-3ff3dc4d6ca3]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v dEclGUPX {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v dEclGUPX" returned: 0 in 0.027s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[7bb94ba2-d9f3-46f3-a4c2-9f68b725c0ae]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'dEclGUPX'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e0958d80-9b3a-4fef-8daf-7b5d64e49e66]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[4604f672-89a4-401e-918a-9dbe1376705b]: (4, ('tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)\ntcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)\ntcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b (non-flash)\ntcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)
Aug 09 02:26:32 user nova-compute[72351]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:26:32 user nova-compute[72351]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b (non-flash)
Aug 09 02:26:32 user nova-compute[72351]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)
Aug 09 02:26:32 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsi session list stdout=tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)
Aug 09 02:26:32 user nova-compute[72351]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:26:32 user nova-compute[72351]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b (non-flash)
Aug 09 02:26:32 user nova-compute[72351]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)
Aug 09 02:26:32 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 -p 172.16.0.220:3260 --login {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 -p 172.16.0.220:3260 --login" returned: 0 in 0.076s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[53655fde-bc06-4112-bd6a-1cb41de33a12]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80, portal: 172.16.0.220,3260]
Aug 09 02:26:32 user nova-compute[72351]: Login to [iface: default, target: iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80, portal: 172.16.0.220,3260] successful.
Aug 09 02:26:32 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[00579647-8217-419b-88d9-6d155c12c7cf]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:32 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:26:33 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:33 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-2b86b241-76fc-40de-9421-4e176d6576d6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:26:33 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-2b86b241-76fc-40de-9421-4e176d6576d6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:26:33 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-2b86b241-76fc-40de-9421-4e176d6576d6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] No VIF found with MAC fa:16:3e:32:5d:c2, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:26:33 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.029s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:33 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[918dbd5a-da97-452a-b14d-801895a08720]: (4, ('tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)\ntcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)\ntcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b (non-flash)\ntcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)\ntcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:33 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)
Aug 09 02:26:33 user nova-compute[72351]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:26:33 user nova-compute[72351]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b (non-flash)
Aug 09 02:26:33 user nova-compute[72351]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)
Aug 09 02:26:33 user nova-compute[72351]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)
Aug 09 02:26:33 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:26:33 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsi session list stdout=tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)
Aug 09 02:26:33 user nova-compute[72351]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:26:33 user nova-compute[72351]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b (non-flash)
Aug 09 02:26:33 user nova-compute[72351]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)
Aug 09 02:26:33 user nova-compute[72351]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)
Aug 09 02:26:33 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:26:33 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.358s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:26:33 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 02:26:33 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] HCTL ('37', '-', '-', 0) found on session 25 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 02:26:33 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Scanning host 37 c: -, t: -, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 02:26:33 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host37/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:33 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host37/scan" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:33 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[f6502fa8-df4a-41dc-a395-aebfe2f7ebb4]: (4, ('- - 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:33 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Searching for a device in session 25 and hctl ['37', '*', '*', 0] yield: sdf {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:26:33 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Connected to sdf using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'dEclGUPX', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 02:26:33 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-2b86b241-76fc-40de-9421-4e176d6576d6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "831c864a-75d9-42b3-b952-749f4201e40e" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 18.724s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:26:34 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.410s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:26:34 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] <== connect_volume: return (1411ms) {'type': 'block', 'scsi_wwn': '23330386161636566', 'path': '/dev/sdf'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:26:34 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23330386161636566', 'path': '/dev/sdf'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 02:26:34 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,compute_id=1,config_drive='',created_at=2023-08-09T02:22:20Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestOtherA-server-446242587',display_name='tempest-ServerActionsTestOtherA-server-1834248220',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serveractionstestothera-server-446242587',id=19,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBB4HNXglactRKXsG6uozX3XR7A39DBllYpgpCz+5+1RhfLOeXdmVkNfDkIG7vQrgoVNKWK/MwqtjPDLEVURcnWQfOuv4ySYdfEPj4hs6FpDqfJbAa5qX0FAbZQuEg69FVQ==',key_name='tempest-keypair-976839412',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:22:30Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={rebuild='server'},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='69d584c230b94ff8aae722eb2f364c5c',ramdisk_id='',reservation_id='r-x0v0gmsw',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerActionsTestOtherA-1303549305',owner_user_name='tempest-ServerActionsTestOtherA-1303549305-project-member'},tags=<?>,task_state='rebuild_spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:26:22Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='5dca34fdadbb4386a1ff1cb255265d47',uuid=585c81a5-ac24-4d12-a8a6-1e7b12a2d53a,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "e521fae4-093d-4c92-a32d-684c1a5137f5", "address": "fa:16:3e:e5:b0:10", "network": {"id": "b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-210689817-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.41", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "69d584c230b94ff8aae722eb2f364c5c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape521fae4-09", "ovs_interfaceid": "e521fae4-093d-4c92-a32d-684c1a5137f5", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:26:34 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Converting VIF {"id": "e521fae4-093d-4c92-a32d-684c1a5137f5", "address": "fa:16:3e:e5:b0:10", "network": {"id": "b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-210689817-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.41", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "69d584c230b94ff8aae722eb2f364c5c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape521fae4-09", "ovs_interfaceid": "e521fae4-093d-4c92-a32d-684c1a5137f5", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:26:34 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:e5:b0:10,bridge_name='br-int',has_traffic_filtering=True,id=e521fae4-093d-4c92-a32d-684c1a5137f5,network=Network(b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape521fae4-09') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:26:34 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:26:34 user nova-compute[72351]:   <uuid>585c81a5-ac24-4d12-a8a6-1e7b12a2d53a</uuid>
Aug 09 02:26:34 user nova-compute[72351]:   <name>instance-00000013</name>
Aug 09 02:26:34 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:26:34 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:26:34 user nova-compute[72351]:   <metadata>
Aug 09 02:26:34 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:26:34 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:26:34 user nova-compute[72351]:       <nova:name>tempest-ServerActionsTestOtherA-server-1834248220</nova:name>
Aug 09 02:26:34 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:26:32</nova:creationTime>
Aug 09 02:26:34 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:26:34 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:26:34 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:26:34 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:26:34 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:26:34 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:26:34 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:26:34 user nova-compute[72351]:       <nova:owner>
Aug 09 02:26:34 user nova-compute[72351]:         <nova:user uuid="5dca34fdadbb4386a1ff1cb255265d47">tempest-ServerActionsTestOtherA-1303549305-project-member</nova:user>
Aug 09 02:26:34 user nova-compute[72351]:         <nova:project uuid="69d584c230b94ff8aae722eb2f364c5c">tempest-ServerActionsTestOtherA-1303549305</nova:project>
Aug 09 02:26:34 user nova-compute[72351]:       </nova:owner>
Aug 09 02:26:34 user nova-compute[72351]:       <nova:root type="image" uuid="18ccda7f-31d3-4978-ab9d-4ed2f74eff94"/>
Aug 09 02:26:34 user nova-compute[72351]:       <nova:ports>
Aug 09 02:26:34 user nova-compute[72351]:         <nova:port uuid="e521fae4-093d-4c92-a32d-684c1a5137f5">
Aug 09 02:26:34 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.4" ipVersion="4"/>
Aug 09 02:26:34 user nova-compute[72351]:         </nova:port>
Aug 09 02:26:34 user nova-compute[72351]:       </nova:ports>
Aug 09 02:26:34 user nova-compute[72351]:     </nova:instance>
Aug 09 02:26:34 user nova-compute[72351]:   </metadata>
Aug 09 02:26:34 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:26:34 user nova-compute[72351]:     <system>
Aug 09 02:26:34 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:26:34 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:26:34 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:26:34 user nova-compute[72351]:       <entry name="serial">585c81a5-ac24-4d12-a8a6-1e7b12a2d53a</entry>
Aug 09 02:26:34 user nova-compute[72351]:       <entry name="uuid">585c81a5-ac24-4d12-a8a6-1e7b12a2d53a</entry>
Aug 09 02:26:34 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:26:34 user nova-compute[72351]:     </system>
Aug 09 02:26:34 user nova-compute[72351]:   </sysinfo>
Aug 09 02:26:34 user nova-compute[72351]:   <os>
Aug 09 02:26:34 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:26:34 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:26:34 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:26:34 user nova-compute[72351]:   </os>
Aug 09 02:26:34 user nova-compute[72351]:   <features>
Aug 09 02:26:34 user nova-compute[72351]:     <acpi/>
Aug 09 02:26:34 user nova-compute[72351]:     <apic/>
Aug 09 02:26:34 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:26:34 user nova-compute[72351]:   </features>
Aug 09 02:26:34 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:26:34 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:26:34 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:26:34 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:26:34 user nova-compute[72351]:   </clock>
Aug 09 02:26:34 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:26:34 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:26:34 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:26:34 user nova-compute[72351]:   </cpu>
Aug 09 02:26:34 user nova-compute[72351]:   <devices>
Aug 09 02:26:34 user nova-compute[72351]:     <disk type="file" device="disk">
Aug 09 02:26:34 user nova-compute[72351]:       <driver name="qemu" type="qcow2" cache="none"/>
Aug 09 02:26:34 user nova-compute[72351]:       <source file="/opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk"/>
Aug 09 02:26:34 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:26:34 user nova-compute[72351]:     </disk>
Aug 09 02:26:34 user nova-compute[72351]:     <disk type="block" device="disk">
Aug 09 02:26:34 user nova-compute[72351]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:26:34 user nova-compute[72351]:       <source dev="/dev/sdf"/>
Aug 09 02:26:34 user nova-compute[72351]:       <target dev="vdb" bus="virtio"/>
Aug 09 02:26:34 user nova-compute[72351]:       <serial>40cdcbc5-17b6-490e-8989-3d6ab80bdd80</serial>
Aug 09 02:26:34 user nova-compute[72351]:     </disk>
Aug 09 02:26:34 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:26:34 user nova-compute[72351]:       <mac address="fa:16:3e:e5:b0:10"/>
Aug 09 02:26:34 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:26:34 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:26:34 user nova-compute[72351]:       <target dev="tape521fae4-09"/>
Aug 09 02:26:34 user nova-compute[72351]:     </interface>
Aug 09 02:26:34 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:26:34 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/console.log" append="off"/>
Aug 09 02:26:34 user nova-compute[72351]:     </serial>
Aug 09 02:26:34 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:26:34 user nova-compute[72351]:     <video>
Aug 09 02:26:34 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:26:34 user nova-compute[72351]:     </video>
Aug 09 02:26:34 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:26:34 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:26:34 user nova-compute[72351]:     </rng>
Aug 09 02:26:34 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:26:34 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:26:34 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:26:34 user nova-compute[72351]:     </memballoon>
Aug 09 02:26:34 user nova-compute[72351]:   </devices>
Aug 09 02:26:34 user nova-compute[72351]: </domain>
Aug 09 02:26:34 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:26:34 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,compute_id=1,config_drive='',created_at=2023-08-09T02:22:20Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestOtherA-server-446242587',display_name='tempest-ServerActionsTestOtherA-server-1834248220',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serveractionstestothera-server-446242587',id=19,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBB4HNXglactRKXsG6uozX3XR7A39DBllYpgpCz+5+1RhfLOeXdmVkNfDkIG7vQrgoVNKWK/MwqtjPDLEVURcnWQfOuv4ySYdfEPj4hs6FpDqfJbAa5qX0FAbZQuEg69FVQ==',key_name='tempest-keypair-976839412',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:22:30Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={rebuild='server'},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='69d584c230b94ff8aae722eb2f364c5c',ramdisk_id='',reservation_id='r-x0v0gmsw',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerActionsTestOtherA-1303549305',owner_user_name='tempest-ServerActionsTestOtherA-1303549305-project-member'},tags=<?>,task_state='rebuild_spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:26:22Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='5dca34fdadbb4386a1ff1cb255265d47',uuid=585c81a5-ac24-4d12-a8a6-1e7b12a2d53a,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "e521fae4-093d-4c92-a32d-684c1a5137f5", "address": "fa:16:3e:e5:b0:10", "network": {"id": "b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-210689817-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.41", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "69d584c230b94ff8aae722eb2f364c5c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape521fae4-09", "ovs_interfaceid": "e521fae4-093d-4c92-a32d-684c1a5137f5", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:26:34 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Converting VIF {"id": "e521fae4-093d-4c92-a32d-684c1a5137f5", "address": "fa:16:3e:e5:b0:10", "network": {"id": "b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-210689817-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.41", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "69d584c230b94ff8aae722eb2f364c5c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape521fae4-09", "ovs_interfaceid": "e521fae4-093d-4c92-a32d-684c1a5137f5", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:26:34 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:e5:b0:10,bridge_name='br-int',has_traffic_filtering=True,id=e521fae4-093d-4c92-a32d-684c1a5137f5,network=Network(b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape521fae4-09') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:26:34 user nova-compute[72351]: DEBUG os_vif [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Plugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:e5:b0:10,bridge_name='br-int',has_traffic_filtering=True,id=e521fae4-093d-4c92-a32d-684c1a5137f5,network=Network(b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape521fae4-09') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:26:34 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:34 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:26:34 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:26:34 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:34 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tape521fae4-09, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:26:34 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tape521fae4-09, col_values=(('external_ids', {'iface-id': 'e521fae4-093d-4c92-a32d-684c1a5137f5', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:e5:b0:10', 'vm-uuid': '585c81a5-ac24-4d12-a8a6-1e7b12a2d53a'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:26:34 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:34 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:26:34 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:34 user nova-compute[72351]: INFO os_vif [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Successfully plugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:e5:b0:10,bridge_name='br-int',has_traffic_filtering=True,id=e521fae4-093d-4c92-a32d-684c1a5137f5,network=Network(b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape521fae4-09')
Aug 09 02:26:34 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:26:34 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:26:34 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] No VIF found with MAC fa:16:3e:e5:b0:10, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:26:35 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:35 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:35 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:35 user nova-compute[72351]: DEBUG nova.compute.manager [req-7140a66b-f3fa-45d8-8059-3752f75f433c req-9e2d9921-2234-48c4-938c-361d71b4648d service nova] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Received event network-vif-plugged-e521fae4-093d-4c92-a32d-684c1a5137f5 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:26:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-7140a66b-f3fa-45d8-8059-3752f75f433c req-9e2d9921-2234-48c4-938c-361d71b4648d service nova] Acquiring lock "585c81a5-ac24-4d12-a8a6-1e7b12a2d53a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:26:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-7140a66b-f3fa-45d8-8059-3752f75f433c req-9e2d9921-2234-48c4-938c-361d71b4648d service nova] Lock "585c81a5-ac24-4d12-a8a6-1e7b12a2d53a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:26:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-7140a66b-f3fa-45d8-8059-3752f75f433c req-9e2d9921-2234-48c4-938c-361d71b4648d service nova] Lock "585c81a5-ac24-4d12-a8a6-1e7b12a2d53a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:26:35 user nova-compute[72351]: DEBUG nova.compute.manager [req-7140a66b-f3fa-45d8-8059-3752f75f433c req-9e2d9921-2234-48c4-938c-361d71b4648d service nova] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] No waiting events found dispatching network-vif-plugged-e521fae4-093d-4c92-a32d-684c1a5137f5 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:26:35 user nova-compute[72351]: WARNING nova.compute.manager [req-7140a66b-f3fa-45d8-8059-3752f75f433c req-9e2d9921-2234-48c4-938c-361d71b4648d service nova] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Received unexpected event network-vif-plugged-e521fae4-093d-4c92-a32d-684c1a5137f5 for instance with vm_state active and task_state rebuild_spawning.
Aug 09 02:26:35 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691547980.9902515, 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:26:35 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] VM Stopped (Lifecycle Event)
Aug 09 02:26:36 user nova-compute[72351]: DEBUG nova.compute.manager [None req-d0771aa9-125f-406c-bb52-003f0419be62 None None] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:26:36 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:36 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:36 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:36 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:36 user nova-compute[72351]: DEBUG nova.compute.manager [req-a833079f-6581-4c9c-8fc2-76099f3d6243 req-4394c660-f315-4c11-b2f8-3f6ca53837ea service nova] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Received event network-changed-02c9ff8a-e779-443c-9b3b-567d2e69cea0 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:26:36 user nova-compute[72351]: DEBUG nova.compute.manager [req-a833079f-6581-4c9c-8fc2-76099f3d6243 req-4394c660-f315-4c11-b2f8-3f6ca53837ea service nova] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Refreshing instance network info cache due to event network-changed-02c9ff8a-e779-443c-9b3b-567d2e69cea0. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:26:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a833079f-6581-4c9c-8fc2-76099f3d6243 req-4394c660-f315-4c11-b2f8-3f6ca53837ea service nova] Acquiring lock "refresh_cache-831c864a-75d9-42b3-b952-749f4201e40e" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:26:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a833079f-6581-4c9c-8fc2-76099f3d6243 req-4394c660-f315-4c11-b2f8-3f6ca53837ea service nova] Acquired lock "refresh_cache-831c864a-75d9-42b3-b952-749f4201e40e" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:26:36 user nova-compute[72351]: DEBUG nova.network.neutron [req-a833079f-6581-4c9c-8fc2-76099f3d6243 req-4394c660-f315-4c11-b2f8-3f6ca53837ea service nova] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Refreshing network info cache for port 02c9ff8a-e779-443c-9b3b-567d2e69cea0 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:26:37 user nova-compute[72351]: DEBUG nova.network.neutron [req-a833079f-6581-4c9c-8fc2-76099f3d6243 req-4394c660-f315-4c11-b2f8-3f6ca53837ea service nova] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Updated VIF entry in instance network info cache for port 02c9ff8a-e779-443c-9b3b-567d2e69cea0. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:26:37 user nova-compute[72351]: DEBUG nova.network.neutron [req-a833079f-6581-4c9c-8fc2-76099f3d6243 req-4394c660-f315-4c11-b2f8-3f6ca53837ea service nova] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Updating instance_info_cache with network_info: [{"id": "02c9ff8a-e779-443c-9b3b-567d2e69cea0", "address": "fa:16:3e:32:5d:c2", "network": {"id": "7024d1f6-a400-4482-9364-78ae3b6adc80", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1603258958-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.94", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c2f67da658d94fabbab82ee6a6171c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap02c9ff8a-e7", "ovs_interfaceid": "02c9ff8a-e779-443c-9b3b-567d2e69cea0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:26:37 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a833079f-6581-4c9c-8fc2-76099f3d6243 req-4394c660-f315-4c11-b2f8-3f6ca53837ea service nova] Releasing lock "refresh_cache-831c864a-75d9-42b3-b952-749f4201e40e" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:26:37 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:38 user nova-compute[72351]: DEBUG nova.compute.manager [req-d5dd8a2a-b69b-4777-8bf9-d9af7b53038b req-47d4a87a-e242-4b37-b6d2-2cfdc85fa4e8 service nova] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Received event network-vif-plugged-e521fae4-093d-4c92-a32d-684c1a5137f5 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:26:38 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d5dd8a2a-b69b-4777-8bf9-d9af7b53038b req-47d4a87a-e242-4b37-b6d2-2cfdc85fa4e8 service nova] Acquiring lock "585c81a5-ac24-4d12-a8a6-1e7b12a2d53a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:26:38 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d5dd8a2a-b69b-4777-8bf9-d9af7b53038b req-47d4a87a-e242-4b37-b6d2-2cfdc85fa4e8 service nova] Lock "585c81a5-ac24-4d12-a8a6-1e7b12a2d53a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:26:38 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d5dd8a2a-b69b-4777-8bf9-d9af7b53038b req-47d4a87a-e242-4b37-b6d2-2cfdc85fa4e8 service nova] Lock "585c81a5-ac24-4d12-a8a6-1e7b12a2d53a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:26:38 user nova-compute[72351]: DEBUG nova.compute.manager [req-d5dd8a2a-b69b-4777-8bf9-d9af7b53038b req-47d4a87a-e242-4b37-b6d2-2cfdc85fa4e8 service nova] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] No waiting events found dispatching network-vif-plugged-e521fae4-093d-4c92-a32d-684c1a5137f5 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:26:38 user nova-compute[72351]: WARNING nova.compute.manager [req-d5dd8a2a-b69b-4777-8bf9-d9af7b53038b req-47d4a87a-e242-4b37-b6d2-2cfdc85fa4e8 service nova] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Received unexpected event network-vif-plugged-e521fae4-093d-4c92-a32d-684c1a5137f5 for instance with vm_state active and task_state rebuild_spawning.
Aug 09 02:26:38 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547998.2477007, 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:26:38 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] VM Resumed (Lifecycle Event)
Aug 09 02:26:38 user nova-compute[72351]: DEBUG nova.compute.manager [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:26:38 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Guest created on hypervisor {{(pid=72351) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4411}}
Aug 09 02:26:38 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Instance spawned successfully.
Aug 09 02:26:38 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:894}}
Aug 09 02:26:38 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:26:38 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: active, current task_state: rebuild_spawning, current DB power_state: 1, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:26:38 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Found default for hw_cdrom_bus of ide {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:26:38 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Found default for hw_disk_bus of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:26:38 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Found default for hw_input_bus of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:26:38 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Found default for hw_pointer_model of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:26:38 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Found default for hw_video_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:26:38 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Found default for hw_vif_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:26:38 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] During sync_power_state the instance has a pending task (rebuild_spawning). Skip.
Aug 09 02:26:38 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691547998.2506497, 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:26:38 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] VM Started (Lifecycle Event)
Aug 09 02:26:38 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:26:38 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Synchronizing instance power state after lifecycle event "Started"; current vm_state: active, current task_state: rebuild_spawning, current DB power_state: 1, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:26:38 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] During sync_power_state the instance has a pending task (rebuild_spawning). Skip.
Aug 09 02:26:38 user nova-compute[72351]: DEBUG nova.compute.manager [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:26:38 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.finish_evacuation" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:26:38 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.finish_evacuation" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:26:38 user nova-compute[72351]: DEBUG nova.objects.instance [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Trying to apply a migration context that does not seem to be set for this instance {{(pid=72351) apply_migration_context /opt/stack/nova/nova/objects/instance.py:1067}}
Aug 09 02:26:38 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5501d414-f921-4f40-87e0-d64256cc4d17 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.finish_evacuation" :: held 0.111s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:26:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:40 user nova-compute[72351]: DEBUG nova.compute.manager [req-8fb77267-8a15-4857-9150-366e286254f6 req-034514b6-3273-48f2-9a83-19e93763c5fc service nova] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Received event network-changed-02c9ff8a-e779-443c-9b3b-567d2e69cea0 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:26:40 user nova-compute[72351]: DEBUG nova.compute.manager [req-8fb77267-8a15-4857-9150-366e286254f6 req-034514b6-3273-48f2-9a83-19e93763c5fc service nova] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Refreshing instance network info cache due to event network-changed-02c9ff8a-e779-443c-9b3b-567d2e69cea0. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:26:40 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-8fb77267-8a15-4857-9150-366e286254f6 req-034514b6-3273-48f2-9a83-19e93763c5fc service nova] Acquiring lock "refresh_cache-831c864a-75d9-42b3-b952-749f4201e40e" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:26:40 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-8fb77267-8a15-4857-9150-366e286254f6 req-034514b6-3273-48f2-9a83-19e93763c5fc service nova] Acquired lock "refresh_cache-831c864a-75d9-42b3-b952-749f4201e40e" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:26:40 user nova-compute[72351]: DEBUG nova.network.neutron [req-8fb77267-8a15-4857-9150-366e286254f6 req-034514b6-3273-48f2-9a83-19e93763c5fc service nova] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Refreshing network info cache for port 02c9ff8a-e779-443c-9b3b-567d2e69cea0 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:26:40 user nova-compute[72351]: DEBUG nova.network.neutron [req-8fb77267-8a15-4857-9150-366e286254f6 req-034514b6-3273-48f2-9a83-19e93763c5fc service nova] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Updated VIF entry in instance network info cache for port 02c9ff8a-e779-443c-9b3b-567d2e69cea0. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:26:40 user nova-compute[72351]: DEBUG nova.network.neutron [req-8fb77267-8a15-4857-9150-366e286254f6 req-034514b6-3273-48f2-9a83-19e93763c5fc service nova] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Updating instance_info_cache with network_info: [{"id": "02c9ff8a-e779-443c-9b3b-567d2e69cea0", "address": "fa:16:3e:32:5d:c2", "network": {"id": "7024d1f6-a400-4482-9364-78ae3b6adc80", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1603258958-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.94", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c2f67da658d94fabbab82ee6a6171c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap02c9ff8a-e7", "ovs_interfaceid": "02c9ff8a-e779-443c-9b3b-567d2e69cea0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:26:40 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-8fb77267-8a15-4857-9150-366e286254f6 req-034514b6-3273-48f2-9a83-19e93763c5fc service nova] Releasing lock "refresh_cache-831c864a-75d9-42b3-b952-749f4201e40e" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:26:40 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-217a655b-d5d4-46c4-aeb6-3ed8bb68a5a1 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Acquiring lock "831c864a-75d9-42b3-b952-749f4201e40e" by "nova.compute.manager.ComputeManager.reboot_instance.<locals>.do_reboot_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:26:40 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-217a655b-d5d4-46c4-aeb6-3ed8bb68a5a1 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "831c864a-75d9-42b3-b952-749f4201e40e" acquired by "nova.compute.manager.ComputeManager.reboot_instance.<locals>.do_reboot_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:26:40 user nova-compute[72351]: INFO nova.compute.manager [None req-217a655b-d5d4-46c4-aeb6-3ed8bb68a5a1 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Rebooting instance
Aug 09 02:26:41 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-217a655b-d5d4-46c4-aeb6-3ed8bb68a5a1 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Acquiring lock "refresh_cache-831c864a-75d9-42b3-b952-749f4201e40e" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:26:41 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-217a655b-d5d4-46c4-aeb6-3ed8bb68a5a1 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Acquired lock "refresh_cache-831c864a-75d9-42b3-b952-749f4201e40e" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:26:41 user nova-compute[72351]: DEBUG nova.network.neutron [None req-217a655b-d5d4-46c4-aeb6-3ed8bb68a5a1 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:26:41 user nova-compute[72351]: DEBUG nova.network.neutron [None req-217a655b-d5d4-46c4-aeb6-3ed8bb68a5a1 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Updating instance_info_cache with network_info: [{"id": "02c9ff8a-e779-443c-9b3b-567d2e69cea0", "address": "fa:16:3e:32:5d:c2", "network": {"id": "7024d1f6-a400-4482-9364-78ae3b6adc80", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1603258958-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.94", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c2f67da658d94fabbab82ee6a6171c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap02c9ff8a-e7", "ovs_interfaceid": "02c9ff8a-e779-443c-9b3b-567d2e69cea0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:26:41 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-217a655b-d5d4-46c4-aeb6-3ed8bb68a5a1 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Releasing lock "refresh_cache-831c864a-75d9-42b3-b952-749f4201e40e" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:26:41 user nova-compute[72351]: DEBUG nova.compute.manager [None req-217a655b-d5d4-46c4-aeb6-3ed8bb68a5a1 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:26:42 user nova-compute[72351]: DEBUG nova.compute.manager [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 957c1510-1320-4492-9052-a58ff540319d] Start spawning the instance on the hypervisor. {{(pid=72351) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2612}}
Aug 09 02:26:42 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 957c1510-1320-4492-9052-a58ff540319d] Creating instance directory {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4717}}
Aug 09 02:26:42 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 957c1510-1320-4492-9052-a58ff540319d] Creating image(s)
Aug 09 02:26:42 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 957c1510-1320-4492-9052-a58ff540319d] Did not create local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4853}}
Aug 09 02:26:42 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 957c1510-1320-4492-9052-a58ff540319d] Ensure instance console log exists: /opt/stack/data/nova/instances/957c1510-1320-4492-9052-a58ff540319d/console.log {{(pid=72351) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4603}}
Aug 09 02:26:42 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:26:42 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:26:42 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:26:42 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 957c1510-1320-4492-9052-a58ff540319d] Start _get_guest_xml network_info=[{"id": "e679cfd9-123f-4b88-8fd2-397e4c8616cc", "address": "fa:16:3e:7f:d6:1b", "network": {"id": "9a452414-d61f-4045-a6cc-9be93c892c4c", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "321111f935dd4e8aab4a083206ea9a7e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape679cfd9-12", "ovs_interfaceid": "e679cfd9-123f-4b88-8fd2-397e4c8616cc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'device_type': 'disk', 'guest_format': None, 'disk_bus': 'virtio', 'mount_device': '/dev/vda', 'boot_index': 0, 'attachment_id': 'd39213a9-ff03-4a4b-a1f7-78783e8e17c7', 'delete_on_termination': False, 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqns': ['iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'MbqJXivK', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '957c1510-1320-4492-9052-a58ff540319d', 'attached_at': '', 'detached_at': '', 'volume_id': '492c8c70-5cca-4ba1-954d-e1a7ebea4235', 'serial': '492c8c70-5cca-4ba1-954d-e1a7ebea4235'}, 'volume_type': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:26:42 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:26:42 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:26:42 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:26:42 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:26:42 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:26:42 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:26:42 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:26:42 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:26:42 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:26:42 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:26:42 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:26:42 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:26:42 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:26:42 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:26:42 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:26:42 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:26:42 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:26:42 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:26:42 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 957c1510-1320-4492-9052-a58ff540319d] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 02:26:42 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'MbqJXivK', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:26:42 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:26:42 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:26:42 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:26:42 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:26:42 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:26:42 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:26:42 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 02:26:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 -p 172.16.0.220:3260" returned: 21 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:42 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[cb5f0983-09e7-4408-80d4-4d466335a4a3]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:42 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Aug 09 02:26:42 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:26:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 -p 172.16.0.220:3260 --interface default --op new {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:42 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[7db472f6-204d-4e07-aeb8-755f2f47ea7a]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235] added\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:42 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235] added
Aug 09 02:26:42 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:26:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:42 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d06a9284-ae53-4c74-bcea-3a218ccf63e8]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:42 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:26:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:42 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d4e267d1-b728-4bca-ac92-1b5b136c5bfd]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:42 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:26:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v MbqJXivK {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v MbqJXivK" returned: 0 in 0.029s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:42 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[df313c2e-0372-4228-92c4-c54ecbd4ead4]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:42 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'MbqJXivK'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:26:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:42 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[345fd7a8-f100-4ffa-a296-06aea32715ab]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:42 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:26:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.029s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:42 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[9b9a23d5-8017-4648-b659-e5a68fe06b70]: (4, ('tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)\ntcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)\ntcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b (non-flash)\ntcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)\ntcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:42 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)
Aug 09 02:26:42 user nova-compute[72351]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:26:42 user nova-compute[72351]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b (non-flash)
Aug 09 02:26:42 user nova-compute[72351]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)
Aug 09 02:26:42 user nova-compute[72351]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)
Aug 09 02:26:42 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:26:42 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] iscsi session list stdout=tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)
Aug 09 02:26:42 user nova-compute[72351]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:26:42 user nova-compute[72351]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b (non-flash)
Aug 09 02:26:42 user nova-compute[72351]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)
Aug 09 02:26:42 user nova-compute[72351]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)
Aug 09 02:26:42 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:26:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 -p 172.16.0.220:3260 --login {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:42 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 -p 172.16.0.220:3260 --login" returned: 0 in 0.083s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:42 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[37f0f7eb-a119-45e2-9834-b25670375d49]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:43 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235, portal: 172.16.0.220,3260]
Aug 09 02:26:43 user nova-compute[72351]: Login to [iface: default, target: iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235, portal: 172.16.0.220,3260] successful.
Aug 09 02:26:43 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:26:43 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:43 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:43 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[6cb27c46-0b86-4ca5-906c-c012930d3ed8]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:43 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:26:43 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:43 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.028s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:43 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[17bf2969-22b4-43e2-b78d-09b9fd079c1f]: (4, ('tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)\ntcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)\ntcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b (non-flash)\ntcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)\ntcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)\ntcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:43 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)
Aug 09 02:26:43 user nova-compute[72351]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:26:43 user nova-compute[72351]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b (non-flash)
Aug 09 02:26:43 user nova-compute[72351]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)
Aug 09 02:26:43 user nova-compute[72351]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)
Aug 09 02:26:43 user nova-compute[72351]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:26:43 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:26:43 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] iscsi session list stdout=tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)
Aug 09 02:26:43 user nova-compute[72351]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:26:43 user nova-compute[72351]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b (non-flash)
Aug 09 02:26:43 user nova-compute[72351]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)
Aug 09 02:26:43 user nova-compute[72351]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)
Aug 09 02:26:43 user nova-compute[72351]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:26:43 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:26:43 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.327s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:26:43 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 02:26:43 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] HCTL ('38', '-', '-', 0) found on session 26 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 02:26:43 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Scanning host 38 c: -, t: -, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 02:26:43 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host38/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:43 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host38/scan" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:43 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[b4a21a25-a5e5-4319-98e6-9399268640c4]: (4, ('- - 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:43 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Searching for a device in session 26 and hctl ['38', '*', '*', 0] yield: None {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:26:44 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Searching for a device in session 26 and hctl ['38', '*', '*', 0] yield: sdg {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:26:44 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Connected to sdg using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'MbqJXivK', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 02:26:44 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.358s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:26:44 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] <== connect_volume: return (1359ms) {'type': 'block', 'scsi_wwn': '26461343430643335', 'path': '/dev/sdg'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:26:44 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 957c1510-1320-4492-9052-a58ff540319d] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26461343430643335', 'path': '/dev/sdg'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 02:26:44 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:26:24Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-821495844',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-821495844',id=28,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='321111f935dd4e8aab4a083206ea9a7e',ramdisk_id='',reservation_id='r-081hk0tj',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-2108329302',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:26:26Z,user_data=None,user_id='73d05c48be5640c28455e8ba8b05ef31',uuid=957c1510-1320-4492-9052-a58ff540319d,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "e679cfd9-123f-4b88-8fd2-397e4c8616cc", "address": "fa:16:3e:7f:d6:1b", "network": {"id": "9a452414-d61f-4045-a6cc-9be93c892c4c", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "321111f935dd4e8aab4a083206ea9a7e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape679cfd9-12", "ovs_interfaceid": "e679cfd9-123f-4b88-8fd2-397e4c8616cc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:26:44 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Converting VIF {"id": "e679cfd9-123f-4b88-8fd2-397e4c8616cc", "address": "fa:16:3e:7f:d6:1b", "network": {"id": "9a452414-d61f-4045-a6cc-9be93c892c4c", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "321111f935dd4e8aab4a083206ea9a7e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape679cfd9-12", "ovs_interfaceid": "e679cfd9-123f-4b88-8fd2-397e4c8616cc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:26:44 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:7f:d6:1b,bridge_name='br-int',has_traffic_filtering=True,id=e679cfd9-123f-4b88-8fd2-397e4c8616cc,network=Network(9a452414-d61f-4045-a6cc-9be93c892c4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape679cfd9-12') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:26:44 user nova-compute[72351]: DEBUG nova.objects.instance [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lazy-loading 'pci_devices' on Instance uuid 957c1510-1320-4492-9052-a58ff540319d {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:26:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:44 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 957c1510-1320-4492-9052-a58ff540319d] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:26:44 user nova-compute[72351]:   <uuid>957c1510-1320-4492-9052-a58ff540319d</uuid>
Aug 09 02:26:44 user nova-compute[72351]:   <name>instance-0000001c</name>
Aug 09 02:26:44 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:26:44 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:26:44 user nova-compute[72351]:   <metadata>
Aug 09 02:26:44 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:26:44 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:26:44 user nova-compute[72351]:       <nova:name>tempest-ServerBootFromVolumeStableRescueTest-server-821495844</nova:name>
Aug 09 02:26:44 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:26:42</nova:creationTime>
Aug 09 02:26:44 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:26:44 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:26:44 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:26:44 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:26:44 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:26:44 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:26:44 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:26:44 user nova-compute[72351]:       <nova:owner>
Aug 09 02:26:44 user nova-compute[72351]:         <nova:user uuid="73d05c48be5640c28455e8ba8b05ef31">tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member</nova:user>
Aug 09 02:26:44 user nova-compute[72351]:         <nova:project uuid="321111f935dd4e8aab4a083206ea9a7e">tempest-ServerBootFromVolumeStableRescueTest-2108329302</nova:project>
Aug 09 02:26:44 user nova-compute[72351]:       </nova:owner>
Aug 09 02:26:44 user nova-compute[72351]:       <nova:root type="image" uuid="18ccda7f-31d3-4978-ab9d-4ed2f74eff94"/>
Aug 09 02:26:44 user nova-compute[72351]:       <nova:ports>
Aug 09 02:26:44 user nova-compute[72351]:         <nova:port uuid="e679cfd9-123f-4b88-8fd2-397e4c8616cc">
Aug 09 02:26:44 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.7" ipVersion="4"/>
Aug 09 02:26:44 user nova-compute[72351]:         </nova:port>
Aug 09 02:26:44 user nova-compute[72351]:       </nova:ports>
Aug 09 02:26:44 user nova-compute[72351]:     </nova:instance>
Aug 09 02:26:44 user nova-compute[72351]:   </metadata>
Aug 09 02:26:44 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:26:44 user nova-compute[72351]:     <system>
Aug 09 02:26:44 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:26:44 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:26:44 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:26:44 user nova-compute[72351]:       <entry name="serial">957c1510-1320-4492-9052-a58ff540319d</entry>
Aug 09 02:26:44 user nova-compute[72351]:       <entry name="uuid">957c1510-1320-4492-9052-a58ff540319d</entry>
Aug 09 02:26:44 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:26:44 user nova-compute[72351]:     </system>
Aug 09 02:26:44 user nova-compute[72351]:   </sysinfo>
Aug 09 02:26:44 user nova-compute[72351]:   <os>
Aug 09 02:26:44 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:26:44 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:26:44 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:26:44 user nova-compute[72351]:   </os>
Aug 09 02:26:44 user nova-compute[72351]:   <features>
Aug 09 02:26:44 user nova-compute[72351]:     <acpi/>
Aug 09 02:26:44 user nova-compute[72351]:     <apic/>
Aug 09 02:26:44 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:26:44 user nova-compute[72351]:   </features>
Aug 09 02:26:44 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:26:44 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:26:44 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:26:44 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:26:44 user nova-compute[72351]:   </clock>
Aug 09 02:26:44 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:26:44 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:26:44 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:26:44 user nova-compute[72351]:   </cpu>
Aug 09 02:26:44 user nova-compute[72351]:   <devices>
Aug 09 02:26:44 user nova-compute[72351]:     <disk type="block" device="disk">
Aug 09 02:26:44 user nova-compute[72351]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:26:44 user nova-compute[72351]:       <source dev="/dev/sdg"/>
Aug 09 02:26:44 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:26:44 user nova-compute[72351]:       <serial>492c8c70-5cca-4ba1-954d-e1a7ebea4235</serial>
Aug 09 02:26:44 user nova-compute[72351]:     </disk>
Aug 09 02:26:44 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:26:44 user nova-compute[72351]:       <mac address="fa:16:3e:7f:d6:1b"/>
Aug 09 02:26:44 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:26:44 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:26:44 user nova-compute[72351]:       <target dev="tape679cfd9-12"/>
Aug 09 02:26:44 user nova-compute[72351]:     </interface>
Aug 09 02:26:44 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:26:44 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/957c1510-1320-4492-9052-a58ff540319d/console.log" append="off"/>
Aug 09 02:26:44 user nova-compute[72351]:     </serial>
Aug 09 02:26:44 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:26:44 user nova-compute[72351]:     <video>
Aug 09 02:26:44 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:26:44 user nova-compute[72351]:     </video>
Aug 09 02:26:44 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:26:44 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:26:44 user nova-compute[72351]:     </rng>
Aug 09 02:26:44 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:26:44 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:26:44 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:26:44 user nova-compute[72351]:     </memballoon>
Aug 09 02:26:44 user nova-compute[72351]:   </devices>
Aug 09 02:26:44 user nova-compute[72351]: </domain>
Aug 09 02:26:44 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:26:44 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:26:24Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-821495844',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-821495844',id=28,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='321111f935dd4e8aab4a083206ea9a7e',ramdisk_id='',reservation_id='r-081hk0tj',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-2108329302',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:26:26Z,user_data=None,user_id='73d05c48be5640c28455e8ba8b05ef31',uuid=957c1510-1320-4492-9052-a58ff540319d,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "e679cfd9-123f-4b88-8fd2-397e4c8616cc", "address": "fa:16:3e:7f:d6:1b", "network": {"id": "9a452414-d61f-4045-a6cc-9be93c892c4c", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "321111f935dd4e8aab4a083206ea9a7e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape679cfd9-12", "ovs_interfaceid": "e679cfd9-123f-4b88-8fd2-397e4c8616cc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:26:44 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Converting VIF {"id": "e679cfd9-123f-4b88-8fd2-397e4c8616cc", "address": "fa:16:3e:7f:d6:1b", "network": {"id": "9a452414-d61f-4045-a6cc-9be93c892c4c", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "321111f935dd4e8aab4a083206ea9a7e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape679cfd9-12", "ovs_interfaceid": "e679cfd9-123f-4b88-8fd2-397e4c8616cc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:26:44 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:7f:d6:1b,bridge_name='br-int',has_traffic_filtering=True,id=e679cfd9-123f-4b88-8fd2-397e4c8616cc,network=Network(9a452414-d61f-4045-a6cc-9be93c892c4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape679cfd9-12') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:26:44 user nova-compute[72351]: DEBUG os_vif [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:7f:d6:1b,bridge_name='br-int',has_traffic_filtering=True,id=e679cfd9-123f-4b88-8fd2-397e4c8616cc,network=Network(9a452414-d61f-4045-a6cc-9be93c892c4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape679cfd9-12') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:26:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:26:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:26:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tape679cfd9-12, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:26:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tape679cfd9-12, col_values=(('external_ids', {'iface-id': 'e679cfd9-123f-4b88-8fd2-397e4c8616cc', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:7f:d6:1b', 'vm-uuid': '957c1510-1320-4492-9052-a58ff540319d'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:26:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:26:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:44 user nova-compute[72351]: INFO os_vif [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:7f:d6:1b,bridge_name='br-int',has_traffic_filtering=True,id=e679cfd9-123f-4b88-8fd2-397e4c8616cc,network=Network(9a452414-d61f-4045-a6cc-9be93c892c4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape679cfd9-12')
Aug 09 02:26:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:44 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:26:44 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] No VIF found with MAC fa:16:3e:7f:d6:1b, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:26:44 user nova-compute[72351]: DEBUG nova.compute.manager [req-fe9bd2b3-4b10-480b-8918-a0636b17fbb4 req-a2eae533-f8aa-4f5d-bfaf-65d543f5fb21 service nova] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Received event network-vif-unplugged-02c9ff8a-e779-443c-9b3b-567d2e69cea0 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:26:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-fe9bd2b3-4b10-480b-8918-a0636b17fbb4 req-a2eae533-f8aa-4f5d-bfaf-65d543f5fb21 service nova] Acquiring lock "831c864a-75d9-42b3-b952-749f4201e40e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:26:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-fe9bd2b3-4b10-480b-8918-a0636b17fbb4 req-a2eae533-f8aa-4f5d-bfaf-65d543f5fb21 service nova] Lock "831c864a-75d9-42b3-b952-749f4201e40e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:26:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-fe9bd2b3-4b10-480b-8918-a0636b17fbb4 req-a2eae533-f8aa-4f5d-bfaf-65d543f5fb21 service nova] Lock "831c864a-75d9-42b3-b952-749f4201e40e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:26:44 user nova-compute[72351]: DEBUG nova.compute.manager [req-fe9bd2b3-4b10-480b-8918-a0636b17fbb4 req-a2eae533-f8aa-4f5d-bfaf-65d543f5fb21 service nova] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] No waiting events found dispatching network-vif-unplugged-02c9ff8a-e779-443c-9b3b-567d2e69cea0 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:26:44 user nova-compute[72351]: WARNING nova.compute.manager [req-fe9bd2b3-4b10-480b-8918-a0636b17fbb4 req-a2eae533-f8aa-4f5d-bfaf-65d543f5fb21 service nova] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Received unexpected event network-vif-unplugged-02c9ff8a-e779-443c-9b3b-567d2e69cea0 for instance with vm_state active and task_state reboot_started.
Aug 09 02:26:45 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-217a655b-d5d4-46c4-aeb6-3ed8bb68a5a1 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Instance shutdown successfully.
Aug 09 02:26:46 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:46 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:46 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:46 user nova-compute[72351]: DEBUG nova.compute.manager [req-b4906da9-5e27-4d90-ac73-eea6fb923c25 req-ff97cc91-62ff-48a9-83a5-b078d9adc31e service nova] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Received event network-vif-plugged-02c9ff8a-e779-443c-9b3b-567d2e69cea0 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:26:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-b4906da9-5e27-4d90-ac73-eea6fb923c25 req-ff97cc91-62ff-48a9-83a5-b078d9adc31e service nova] Acquiring lock "831c864a-75d9-42b3-b952-749f4201e40e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:26:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-b4906da9-5e27-4d90-ac73-eea6fb923c25 req-ff97cc91-62ff-48a9-83a5-b078d9adc31e service nova] Lock "831c864a-75d9-42b3-b952-749f4201e40e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:26:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-b4906da9-5e27-4d90-ac73-eea6fb923c25 req-ff97cc91-62ff-48a9-83a5-b078d9adc31e service nova] Lock "831c864a-75d9-42b3-b952-749f4201e40e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:26:46 user nova-compute[72351]: DEBUG nova.compute.manager [req-b4906da9-5e27-4d90-ac73-eea6fb923c25 req-ff97cc91-62ff-48a9-83a5-b078d9adc31e service nova] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] No waiting events found dispatching network-vif-plugged-02c9ff8a-e779-443c-9b3b-567d2e69cea0 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:26:46 user nova-compute[72351]: WARNING nova.compute.manager [req-b4906da9-5e27-4d90-ac73-eea6fb923c25 req-ff97cc91-62ff-48a9-83a5-b078d9adc31e service nova] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Received unexpected event network-vif-plugged-02c9ff8a-e779-443c-9b3b-567d2e69cea0 for instance with vm_state active and task_state reboot_started.
Aug 09 02:26:46 user nova-compute[72351]: DEBUG nova.compute.manager [req-a5d9aa67-7031-4c75-9263-c7f3e728c8c0 req-dfa63c65-37be-4c05-b4f7-3632b0fcdee1 service nova] [instance: 957c1510-1320-4492-9052-a58ff540319d] Received event network-vif-plugged-e679cfd9-123f-4b88-8fd2-397e4c8616cc {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:26:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a5d9aa67-7031-4c75-9263-c7f3e728c8c0 req-dfa63c65-37be-4c05-b4f7-3632b0fcdee1 service nova] Acquiring lock "957c1510-1320-4492-9052-a58ff540319d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:26:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a5d9aa67-7031-4c75-9263-c7f3e728c8c0 req-dfa63c65-37be-4c05-b4f7-3632b0fcdee1 service nova] Lock "957c1510-1320-4492-9052-a58ff540319d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:26:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a5d9aa67-7031-4c75-9263-c7f3e728c8c0 req-dfa63c65-37be-4c05-b4f7-3632b0fcdee1 service nova] Lock "957c1510-1320-4492-9052-a58ff540319d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:26:46 user nova-compute[72351]: DEBUG nova.compute.manager [req-a5d9aa67-7031-4c75-9263-c7f3e728c8c0 req-dfa63c65-37be-4c05-b4f7-3632b0fcdee1 service nova] [instance: 957c1510-1320-4492-9052-a58ff540319d] No waiting events found dispatching network-vif-plugged-e679cfd9-123f-4b88-8fd2-397e4c8616cc {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:26:46 user nova-compute[72351]: WARNING nova.compute.manager [req-a5d9aa67-7031-4c75-9263-c7f3e728c8c0 req-dfa63c65-37be-4c05-b4f7-3632b0fcdee1 service nova] [instance: 957c1510-1320-4492-9052-a58ff540319d] Received unexpected event network-vif-plugged-e679cfd9-123f-4b88-8fd2-397e4c8616cc for instance with vm_state building and task_state spawning.
Aug 09 02:26:46 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:46 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:47 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:47 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:47 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:47 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:47 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:48 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:48 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:48 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:48 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f0fba6ea-3795-40eb-acec-2897ecefecaf tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquiring lock "1d772c10-1c8a-4101-9a61-f8a60bb19c66" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:26:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f0fba6ea-3795-40eb-acec-2897ecefecaf tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "1d772c10-1c8a-4101-9a61-f8a60bb19c66" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:26:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f0fba6ea-3795-40eb-acec-2897ecefecaf tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquiring lock "1d772c10-1c8a-4101-9a61-f8a60bb19c66-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:26:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f0fba6ea-3795-40eb-acec-2897ecefecaf tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "1d772c10-1c8a-4101-9a61-f8a60bb19c66-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:26:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f0fba6ea-3795-40eb-acec-2897ecefecaf tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "1d772c10-1c8a-4101-9a61-f8a60bb19c66-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:26:48 user nova-compute[72351]: INFO nova.compute.manager [None req-f0fba6ea-3795-40eb-acec-2897ecefecaf tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Terminating instance
Aug 09 02:26:48 user nova-compute[72351]: DEBUG nova.compute.manager [None req-f0fba6ea-3795-40eb-acec-2897ecefecaf tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Start destroying the instance on the hypervisor. {{(pid=72351) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3124}}
Aug 09 02:26:48 user nova-compute[72351]: DEBUG nova.compute.manager [req-dacdff8f-d791-4149-97c9-fe722b4b55b8 req-bb79380f-f37c-405a-b104-d69678c46e80 service nova] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Received event network-vif-plugged-02c9ff8a-e779-443c-9b3b-567d2e69cea0 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:26:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-dacdff8f-d791-4149-97c9-fe722b4b55b8 req-bb79380f-f37c-405a-b104-d69678c46e80 service nova] Acquiring lock "831c864a-75d9-42b3-b952-749f4201e40e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:26:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-dacdff8f-d791-4149-97c9-fe722b4b55b8 req-bb79380f-f37c-405a-b104-d69678c46e80 service nova] Lock "831c864a-75d9-42b3-b952-749f4201e40e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:26:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-dacdff8f-d791-4149-97c9-fe722b4b55b8 req-bb79380f-f37c-405a-b104-d69678c46e80 service nova] Lock "831c864a-75d9-42b3-b952-749f4201e40e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:26:48 user nova-compute[72351]: DEBUG nova.compute.manager [req-dacdff8f-d791-4149-97c9-fe722b4b55b8 req-bb79380f-f37c-405a-b104-d69678c46e80 service nova] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] No waiting events found dispatching network-vif-plugged-02c9ff8a-e779-443c-9b3b-567d2e69cea0 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:26:48 user nova-compute[72351]: WARNING nova.compute.manager [req-dacdff8f-d791-4149-97c9-fe722b4b55b8 req-bb79380f-f37c-405a-b104-d69678c46e80 service nova] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Received unexpected event network-vif-plugged-02c9ff8a-e779-443c-9b3b-567d2e69cea0 for instance with vm_state active and task_state reboot_started.
Aug 09 02:26:48 user nova-compute[72351]: DEBUG nova.compute.manager [req-dacdff8f-d791-4149-97c9-fe722b4b55b8 req-bb79380f-f37c-405a-b104-d69678c46e80 service nova] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Received event network-vif-plugged-02c9ff8a-e779-443c-9b3b-567d2e69cea0 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:26:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-dacdff8f-d791-4149-97c9-fe722b4b55b8 req-bb79380f-f37c-405a-b104-d69678c46e80 service nova] Acquiring lock "831c864a-75d9-42b3-b952-749f4201e40e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:26:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-dacdff8f-d791-4149-97c9-fe722b4b55b8 req-bb79380f-f37c-405a-b104-d69678c46e80 service nova] Lock "831c864a-75d9-42b3-b952-749f4201e40e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:26:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-dacdff8f-d791-4149-97c9-fe722b4b55b8 req-bb79380f-f37c-405a-b104-d69678c46e80 service nova] Lock "831c864a-75d9-42b3-b952-749f4201e40e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:26:48 user nova-compute[72351]: DEBUG nova.compute.manager [req-dacdff8f-d791-4149-97c9-fe722b4b55b8 req-bb79380f-f37c-405a-b104-d69678c46e80 service nova] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] No waiting events found dispatching network-vif-plugged-02c9ff8a-e779-443c-9b3b-567d2e69cea0 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:26:48 user nova-compute[72351]: WARNING nova.compute.manager [req-dacdff8f-d791-4149-97c9-fe722b4b55b8 req-bb79380f-f37c-405a-b104-d69678c46e80 service nova] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Received unexpected event network-vif-plugged-02c9ff8a-e779-443c-9b3b-567d2e69cea0 for instance with vm_state active and task_state reboot_started.
Aug 09 02:26:48 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:48 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:48 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:48 user nova-compute[72351]: DEBUG nova.compute.manager [req-ba86a635-3d09-4641-8e0b-03b22995f166 req-43f6e4fc-b7bb-4878-a62c-0f547843b036 service nova] [instance: 957c1510-1320-4492-9052-a58ff540319d] Received event network-vif-plugged-e679cfd9-123f-4b88-8fd2-397e4c8616cc {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:26:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ba86a635-3d09-4641-8e0b-03b22995f166 req-43f6e4fc-b7bb-4878-a62c-0f547843b036 service nova] Acquiring lock "957c1510-1320-4492-9052-a58ff540319d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:26:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ba86a635-3d09-4641-8e0b-03b22995f166 req-43f6e4fc-b7bb-4878-a62c-0f547843b036 service nova] Lock "957c1510-1320-4492-9052-a58ff540319d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:26:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ba86a635-3d09-4641-8e0b-03b22995f166 req-43f6e4fc-b7bb-4878-a62c-0f547843b036 service nova] Lock "957c1510-1320-4492-9052-a58ff540319d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:26:48 user nova-compute[72351]: DEBUG nova.compute.manager [req-ba86a635-3d09-4641-8e0b-03b22995f166 req-43f6e4fc-b7bb-4878-a62c-0f547843b036 service nova] [instance: 957c1510-1320-4492-9052-a58ff540319d] No waiting events found dispatching network-vif-plugged-e679cfd9-123f-4b88-8fd2-397e4c8616cc {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:26:48 user nova-compute[72351]: WARNING nova.compute.manager [req-ba86a635-3d09-4641-8e0b-03b22995f166 req-43f6e4fc-b7bb-4878-a62c-0f547843b036 service nova] [instance: 957c1510-1320-4492-9052-a58ff540319d] Received unexpected event network-vif-plugged-e679cfd9-123f-4b88-8fd2-397e4c8616cc for instance with vm_state building and task_state spawning.
Aug 09 02:26:48 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:48 user nova-compute[72351]: DEBUG nova.compute.manager [req-76372366-a1a2-43a6-9816-051089624e73 req-c8f833df-0109-40d6-a909-ffc4c6cd84e5 service nova] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Received event network-vif-unplugged-55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:26:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-76372366-a1a2-43a6-9816-051089624e73 req-c8f833df-0109-40d6-a909-ffc4c6cd84e5 service nova] Acquiring lock "1d772c10-1c8a-4101-9a61-f8a60bb19c66-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:26:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-76372366-a1a2-43a6-9816-051089624e73 req-c8f833df-0109-40d6-a909-ffc4c6cd84e5 service nova] Lock "1d772c10-1c8a-4101-9a61-f8a60bb19c66-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:26:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-76372366-a1a2-43a6-9816-051089624e73 req-c8f833df-0109-40d6-a909-ffc4c6cd84e5 service nova] Lock "1d772c10-1c8a-4101-9a61-f8a60bb19c66-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:26:48 user nova-compute[72351]: DEBUG nova.compute.manager [req-76372366-a1a2-43a6-9816-051089624e73 req-c8f833df-0109-40d6-a909-ffc4c6cd84e5 service nova] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] No waiting events found dispatching network-vif-unplugged-55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:26:48 user nova-compute[72351]: DEBUG nova.compute.manager [req-76372366-a1a2-43a6-9816-051089624e73 req-c8f833df-0109-40d6-a909-ffc4c6cd84e5 service nova] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Received event network-vif-unplugged-55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7 for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:26:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:49 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Instance destroyed successfully.
Aug 09 02:26:49 user nova-compute[72351]: DEBUG nova.objects.instance [None req-f0fba6ea-3795-40eb-acec-2897ecefecaf tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lazy-loading 'resources' on Instance uuid 1d772c10-1c8a-4101-9a61-f8a60bb19c66 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:26:49 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-f0fba6ea-3795-40eb-acec-2897ecefecaf tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:21:56Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-volume-backed-server-543725645',display_name='tempest-TestVolumeBootPattern-volume-backed-server-543725645',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testvolumebootpattern-volume-backed-server-543725645',id=18,image_ref='',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBG11cGkjeY18Po8k08DMyUzQtsrtCaWAq6iJyHMr9Whc2GLjfVhsEnL4e/8YHs5tlZeSHiwOT/FcEFipB83gNXXJkM/cq3cq4F+DuxaxT/aGoMoUGxsL8LTol/71KOxcMQ==',key_name='tempest-keypair-1791716351',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:22:18Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='a4c6bbfd8d904ba1ac69ba5da4564513',ramdisk_id='',reservation_id='r-2d0wwhw3',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TestVolumeBootPattern-350743460',owner_user_name='tempest-TestVolumeBootPattern-350743460-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:22:19Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='07d8d5a634274df19c2db769d2d27b29',uuid=1d772c10-1c8a-4101-9a61-f8a60bb19c66,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7", "address": "fa:16:3e:75:70:ff", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.234", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap55e3cfd3-c6", "ovs_interfaceid": "55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:26:49 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-f0fba6ea-3795-40eb-acec-2897ecefecaf tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Converting VIF {"id": "55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7", "address": "fa:16:3e:75:70:ff", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.234", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap55e3cfd3-c6", "ovs_interfaceid": "55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:26:49 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-f0fba6ea-3795-40eb-acec-2897ecefecaf tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:75:70:ff,bridge_name='br-int',has_traffic_filtering=True,id=55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7,network=Network(5ef428cf-e408-490c-8f60-70934f760f4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap55e3cfd3-c6') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:26:49 user nova-compute[72351]: DEBUG os_vif [None req-f0fba6ea-3795-40eb-acec-2897ecefecaf tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:75:70:ff,bridge_name='br-int',has_traffic_filtering=True,id=55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7,network=Network(5ef428cf-e408-490c-8f60-70934f760f4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap55e3cfd3-c6') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:26:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap55e3cfd3-c6, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:26:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:49 user nova-compute[72351]: INFO os_vif [None req-f0fba6ea-3795-40eb-acec-2897ecefecaf tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:75:70:ff,bridge_name='br-int',has_traffic_filtering=True,id=55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7,network=Network(5ef428cf-e408-490c-8f60-70934f760f4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap55e3cfd3-c6')
Aug 09 02:26:49 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-f0fba6ea-3795-40eb-acec-2897ecefecaf tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 02:26:49 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-f0fba6ea-3795-40eb-acec-2897ecefecaf tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'xbIDCRQN', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdd'}, None), 'kwargs': True}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:26:49 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-f0fba6ea-3795-40eb-acec-2897ecefecaf tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:26:49 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-f0fba6ea-3795-40eb-acec-2897ecefecaf tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:26:49 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-f0fba6ea-3795-40eb-acec-2897ecefecaf tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:26:49 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-f0fba6ea-3795-40eb-acec-2897ecefecaf tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 02:26:49 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:49 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.026s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:49 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[67f9e985-85c5-4cf5-89d1-45791068b54f]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:49 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:49 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.036s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:49 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[474720e9-0a97-4b7e-b8d4-f195bbfcc963]: (4, ('tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)\ntcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)\ntcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b (non-flash)\ntcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)\ntcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)\ntcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:49 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-f0fba6ea-3795-40eb-acec-2897ecefecaf tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)
Aug 09 02:26:49 user nova-compute[72351]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:26:49 user nova-compute[72351]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b (non-flash)
Aug 09 02:26:49 user nova-compute[72351]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)
Aug 09 02:26:49 user nova-compute[72351]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)
Aug 09 02:26:49 user nova-compute[72351]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:26:49 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:26:49 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-f0fba6ea-3795-40eb-acec-2897ecefecaf tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsi session list stdout=tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 (non-flash)
Aug 09 02:26:49 user nova-compute[72351]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:26:49 user nova-compute[72351]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b (non-flash)
Aug 09 02:26:49 user nova-compute[72351]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)
Aug 09 02:26:49 user nova-compute[72351]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)
Aug 09 02:26:49 user nova-compute[72351]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:26:49 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:26:49 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-f0fba6ea-3795-40eb-acec-2897ecefecaf tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fcfdc1ddd80>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274'): ({'sdd'}, set())}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 02:26:49 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-f0fba6ea-3795-40eb-acec-2897ecefecaf tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Removing single pathed devices sdd {{(pid=72351) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Aug 09 02:26:49 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:49 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.025s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:49 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[045d9082-9b1c-4cae-b08d-e612d1917252]: (4, ('path checker states:\nup                  5\npending             1\n\npaths: 0\nbusy: False\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:49 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdd {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:49 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdd" returned: 0 in 0.035s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:49 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[df061d9c-6282-4bf8-8d5e-4ee89bd9820b]: (4, ('ok\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:49 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-f0fba6ea-3795-40eb-acec-2897ecefecaf tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Flushing IO for device /dev/sdd {{(pid=72351) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Aug 09 02:26:49 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdd {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:49 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdd" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:49 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[cc4bf63d-7a71-491a-adc0-b1d8d97876a3]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:49 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-f0fba6ea-3795-40eb-acec-2897ecefecaf tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Remove SCSI device /dev/sdd with /sys/block/sdd/device/delete {{(pid=72351) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Aug 09 02:26:49 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdd/device/delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:49 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdd/device/delete" returned: 0 in 0.059s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:49 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[fbea9d22-034d-4164-b962-9c06755a4d08]: (4, ('1', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:49 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-f0fba6ea-3795-40eb-acec-2897ecefecaf tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Checking to see if SCSI volumes sdd have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Aug 09 02:26:49 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-f0fba6ea-3795-40eb-acec-2897ecefecaf tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] SCSI volumes sdd have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Aug 09 02:26:49 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-f0fba6ea-3795-40eb-acec-2897ecefecaf tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274')] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 02:26:49 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:49 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.030s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:49 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[c4d1ce1b-c6c9-4ecf-8390-70b5b746a088]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:49 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-f0fba6ea-3795-40eb-acec-2897ecefecaf tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:26:49 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 -p 172.16.0.220:3260 --logout {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:49 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 -p 172.16.0.220:3260 --logout" returned: 0 in 0.091s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:49 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[6b1883a6-f8ca-4f76-8130-06827f048a63]: (4, ('Logging out of session [sid: 17, target: iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274, portal: 172.16.0.220,3260]\nLogout of [sid: 17, target: iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:49 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-f0fba6ea-3795-40eb-acec-2897ecefecaf tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 17, target: iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274, portal: 172.16.0.220,3260]
Aug 09 02:26:49 user nova-compute[72351]: Logout of [sid: 17, target: iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274, portal: 172.16.0.220,3260] successful.
Aug 09 02:26:49 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:26:49 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 -p 172.16.0.220:3260 --op delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:49 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ba022fd7-964b-4325-9d01-74fb60f45274 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.045s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:49 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e4ae7b90-eefb-4384-aca6-1b2fcb675aaf]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:49 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-f0fba6ea-3795-40eb-acec-2897ecefecaf tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:26:49 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-f0fba6ea-3795-40eb-acec-2897ecefecaf tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.450s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:26:49 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-f0fba6ea-3795-40eb-acec-2897ecefecaf tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] <== disconnect_volume: return (451ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:26:49 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-f0fba6ea-3795-40eb-acec-2897ecefecaf tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 02:26:49 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-f0fba6ea-3795-40eb-acec-2897ecefecaf tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Deleting instance files /opt/stack/data/nova/instances/1d772c10-1c8a-4101-9a61-f8a60bb19c66_del
Aug 09 02:26:49 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-f0fba6ea-3795-40eb-acec-2897ecefecaf tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Deletion of /opt/stack/data/nova/instances/1d772c10-1c8a-4101-9a61-f8a60bb19c66_del complete
Aug 09 02:26:49 user nova-compute[72351]: INFO nova.compute.manager [None req-f0fba6ea-3795-40eb-acec-2897ecefecaf tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Took 1.44 seconds to destroy the instance on the hypervisor.
Aug 09 02:26:49 user nova-compute[72351]: DEBUG oslo.service.loopingcall [None req-f0fba6ea-3795-40eb-acec-2897ecefecaf tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72351) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Aug 09 02:26:49 user nova-compute[72351]: DEBUG nova.compute.manager [-] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Deallocating network for instance {{(pid=72351) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2263}}
Aug 09 02:26:49 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] deallocate_for_instance() {{(pid=72351) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1795}}
Aug 09 02:26:50 user nova-compute[72351]: DEBUG nova.compute.manager [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 957c1510-1320-4492-9052-a58ff540319d] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:26:50 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 957c1510-1320-4492-9052-a58ff540319d] Guest created on hypervisor {{(pid=72351) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4411}}
Aug 09 02:26:50 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548010.014357, 957c1510-1320-4492-9052-a58ff540319d => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:26:50 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 957c1510-1320-4492-9052-a58ff540319d] VM Resumed (Lifecycle Event)
Aug 09 02:26:50 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:50 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 957c1510-1320-4492-9052-a58ff540319d] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:26:50 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 957c1510-1320-4492-9052-a58ff540319d] Instance spawned successfully.
Aug 09 02:26:50 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 957c1510-1320-4492-9052-a58ff540319d] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:894}}
Aug 09 02:26:50 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:50 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 957c1510-1320-4492-9052-a58ff540319d] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:26:50 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 957c1510-1320-4492-9052-a58ff540319d] Found default for hw_cdrom_bus of ide {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:26:50 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 957c1510-1320-4492-9052-a58ff540319d] Found default for hw_disk_bus of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:26:50 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 957c1510-1320-4492-9052-a58ff540319d] Found default for hw_input_bus of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:26:50 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 957c1510-1320-4492-9052-a58ff540319d] Found default for hw_pointer_model of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:26:50 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 957c1510-1320-4492-9052-a58ff540319d] Found default for hw_video_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:26:50 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 957c1510-1320-4492-9052-a58ff540319d] Found default for hw_vif_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:26:50 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 957c1510-1320-4492-9052-a58ff540319d] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:26:50 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548010.0147307, 957c1510-1320-4492-9052-a58ff540319d => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:26:50 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 957c1510-1320-4492-9052-a58ff540319d] VM Started (Lifecycle Event)
Aug 09 02:26:50 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 957c1510-1320-4492-9052-a58ff540319d] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:26:50 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 957c1510-1320-4492-9052-a58ff540319d] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:26:50 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Instance running successfully.
Aug 09 02:26:50 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-217a655b-d5d4-46c4-aeb6-3ed8bb68a5a1 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Instance soft rebooted successfully.
Aug 09 02:26:50 user nova-compute[72351]: DEBUG nova.compute.manager [None req-217a655b-d5d4-46c4-aeb6-3ed8bb68a5a1 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:26:50 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 957c1510-1320-4492-9052-a58ff540319d] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:26:50 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Removed pending event for 831c864a-75d9-42b3-b952-749f4201e40e due to event {{(pid=72351) _event_emit_delayed /opt/stack/nova/nova/virt/libvirt/host.py:438}}
Aug 09 02:26:50 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548010.1679683, 831c864a-75d9-42b3-b952-749f4201e40e => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:26:50 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] VM Resumed (Lifecycle Event)
Aug 09 02:26:50 user nova-compute[72351]: INFO nova.compute.manager [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 957c1510-1320-4492-9052-a58ff540319d] Took 7.52 seconds to spawn the instance on the hypervisor.
Aug 09 02:26:50 user nova-compute[72351]: DEBUG nova.compute.manager [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 957c1510-1320-4492-9052-a58ff540319d] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:26:50 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:26:50 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:50 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:50 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:50 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: active, current task_state: reboot_started, current DB power_state: 1, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:26:50 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-217a655b-d5d4-46c4-aeb6-3ed8bb68a5a1 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "831c864a-75d9-42b3-b952-749f4201e40e" "released" by "nova.compute.manager.ComputeManager.reboot_instance.<locals>.do_reboot_instance" :: held 9.320s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:26:50 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548010.169837, 831c864a-75d9-42b3-b952-749f4201e40e => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:26:50 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] VM Started (Lifecycle Event)
Aug 09 02:26:50 user nova-compute[72351]: INFO nova.compute.manager [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 957c1510-1320-4492-9052-a58ff540319d] Took 24.96 seconds to build instance.
Aug 09 02:26:50 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-612e4433-6cc6-45ab-97a0-4b55f50f0a1c tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "957c1510-1320-4492-9052-a58ff540319d" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 25.059s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:26:50 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:26:50 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Synchronizing instance power state after lifecycle event "Started"; current vm_state: active, current task_state: None, current DB power_state: 1, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:26:50 user nova-compute[72351]: DEBUG nova.compute.manager [req-556acc23-3d5e-44cf-84d2-2589d589f5a1 req-529d10f6-f54a-42fd-89c9-efc852d07d00 service nova] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Received event network-changed-dc244cf9-b0f6-4906-b9f3-53634b1847e3 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:26:50 user nova-compute[72351]: DEBUG nova.compute.manager [req-556acc23-3d5e-44cf-84d2-2589d589f5a1 req-529d10f6-f54a-42fd-89c9-efc852d07d00 service nova] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Refreshing instance network info cache due to event network-changed-dc244cf9-b0f6-4906-b9f3-53634b1847e3. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:26:50 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-556acc23-3d5e-44cf-84d2-2589d589f5a1 req-529d10f6-f54a-42fd-89c9-efc852d07d00 service nova] Acquiring lock "refresh_cache-f457e6ea-8d06-46c6-be2e-f650e28a3b69" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:26:50 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-556acc23-3d5e-44cf-84d2-2589d589f5a1 req-529d10f6-f54a-42fd-89c9-efc852d07d00 service nova] Acquired lock "refresh_cache-f457e6ea-8d06-46c6-be2e-f650e28a3b69" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:26:50 user nova-compute[72351]: DEBUG nova.network.neutron [req-556acc23-3d5e-44cf-84d2-2589d589f5a1 req-529d10f6-f54a-42fd-89c9-efc852d07d00 service nova] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Refreshing network info cache for port dc244cf9-b0f6-4906-b9f3-53634b1847e3 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:26:50 user nova-compute[72351]: DEBUG nova.compute.manager [req-ed3b7b69-01d0-414e-a1dd-69d48639c7df req-ed9796f4-afae-4edb-b8ba-0a1b12d1e535 service nova] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Received event network-vif-plugged-55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:26:50 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ed3b7b69-01d0-414e-a1dd-69d48639c7df req-ed9796f4-afae-4edb-b8ba-0a1b12d1e535 service nova] Acquiring lock "1d772c10-1c8a-4101-9a61-f8a60bb19c66-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:26:50 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ed3b7b69-01d0-414e-a1dd-69d48639c7df req-ed9796f4-afae-4edb-b8ba-0a1b12d1e535 service nova] Lock "1d772c10-1c8a-4101-9a61-f8a60bb19c66-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:26:50 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ed3b7b69-01d0-414e-a1dd-69d48639c7df req-ed9796f4-afae-4edb-b8ba-0a1b12d1e535 service nova] Lock "1d772c10-1c8a-4101-9a61-f8a60bb19c66-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:26:50 user nova-compute[72351]: DEBUG nova.compute.manager [req-ed3b7b69-01d0-414e-a1dd-69d48639c7df req-ed9796f4-afae-4edb-b8ba-0a1b12d1e535 service nova] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] No waiting events found dispatching network-vif-plugged-55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:26:50 user nova-compute[72351]: WARNING nova.compute.manager [req-ed3b7b69-01d0-414e-a1dd-69d48639c7df req-ed9796f4-afae-4edb-b8ba-0a1b12d1e535 service nova] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Received unexpected event network-vif-plugged-55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7 for instance with vm_state active and task_state deleting.
Aug 09 02:26:50 user nova-compute[72351]: DEBUG nova.compute.manager [req-ed3b7b69-01d0-414e-a1dd-69d48639c7df req-ed9796f4-afae-4edb-b8ba-0a1b12d1e535 service nova] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Received event network-vif-plugged-55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:26:50 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ed3b7b69-01d0-414e-a1dd-69d48639c7df req-ed9796f4-afae-4edb-b8ba-0a1b12d1e535 service nova] Acquiring lock "1d772c10-1c8a-4101-9a61-f8a60bb19c66-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:26:50 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ed3b7b69-01d0-414e-a1dd-69d48639c7df req-ed9796f4-afae-4edb-b8ba-0a1b12d1e535 service nova] Lock "1d772c10-1c8a-4101-9a61-f8a60bb19c66-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:26:50 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ed3b7b69-01d0-414e-a1dd-69d48639c7df req-ed9796f4-afae-4edb-b8ba-0a1b12d1e535 service nova] Lock "1d772c10-1c8a-4101-9a61-f8a60bb19c66-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:26:50 user nova-compute[72351]: DEBUG nova.compute.manager [req-ed3b7b69-01d0-414e-a1dd-69d48639c7df req-ed9796f4-afae-4edb-b8ba-0a1b12d1e535 service nova] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] No waiting events found dispatching network-vif-plugged-55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:26:50 user nova-compute[72351]: WARNING nova.compute.manager [req-ed3b7b69-01d0-414e-a1dd-69d48639c7df req-ed9796f4-afae-4edb-b8ba-0a1b12d1e535 service nova] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Received unexpected event network-vif-plugged-55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7 for instance with vm_state active and task_state deleting.
Aug 09 02:26:50 user nova-compute[72351]: DEBUG nova.compute.manager [req-ed3b7b69-01d0-414e-a1dd-69d48639c7df req-ed9796f4-afae-4edb-b8ba-0a1b12d1e535 service nova] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Received event network-vif-plugged-55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:26:50 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ed3b7b69-01d0-414e-a1dd-69d48639c7df req-ed9796f4-afae-4edb-b8ba-0a1b12d1e535 service nova] Acquiring lock "1d772c10-1c8a-4101-9a61-f8a60bb19c66-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:26:50 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ed3b7b69-01d0-414e-a1dd-69d48639c7df req-ed9796f4-afae-4edb-b8ba-0a1b12d1e535 service nova] Lock "1d772c10-1c8a-4101-9a61-f8a60bb19c66-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:26:50 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ed3b7b69-01d0-414e-a1dd-69d48639c7df req-ed9796f4-afae-4edb-b8ba-0a1b12d1e535 service nova] Lock "1d772c10-1c8a-4101-9a61-f8a60bb19c66-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:26:50 user nova-compute[72351]: DEBUG nova.compute.manager [req-ed3b7b69-01d0-414e-a1dd-69d48639c7df req-ed9796f4-afae-4edb-b8ba-0a1b12d1e535 service nova] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] No waiting events found dispatching network-vif-plugged-55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:26:50 user nova-compute[72351]: WARNING nova.compute.manager [req-ed3b7b69-01d0-414e-a1dd-69d48639c7df req-ed9796f4-afae-4edb-b8ba-0a1b12d1e535 service nova] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Received unexpected event network-vif-plugged-55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7 for instance with vm_state active and task_state deleting.
Aug 09 02:26:50 user nova-compute[72351]: DEBUG nova.compute.manager [req-ed3b7b69-01d0-414e-a1dd-69d48639c7df req-ed9796f4-afae-4edb-b8ba-0a1b12d1e535 service nova] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Received event network-vif-unplugged-55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:26:50 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ed3b7b69-01d0-414e-a1dd-69d48639c7df req-ed9796f4-afae-4edb-b8ba-0a1b12d1e535 service nova] Acquiring lock "1d772c10-1c8a-4101-9a61-f8a60bb19c66-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:26:50 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ed3b7b69-01d0-414e-a1dd-69d48639c7df req-ed9796f4-afae-4edb-b8ba-0a1b12d1e535 service nova] Lock "1d772c10-1c8a-4101-9a61-f8a60bb19c66-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:26:50 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ed3b7b69-01d0-414e-a1dd-69d48639c7df req-ed9796f4-afae-4edb-b8ba-0a1b12d1e535 service nova] Lock "1d772c10-1c8a-4101-9a61-f8a60bb19c66-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:26:50 user nova-compute[72351]: DEBUG nova.compute.manager [req-ed3b7b69-01d0-414e-a1dd-69d48639c7df req-ed9796f4-afae-4edb-b8ba-0a1b12d1e535 service nova] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] No waiting events found dispatching network-vif-unplugged-55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:26:50 user nova-compute[72351]: DEBUG nova.compute.manager [req-ed3b7b69-01d0-414e-a1dd-69d48639c7df req-ed9796f4-afae-4edb-b8ba-0a1b12d1e535 service nova] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Received event network-vif-unplugged-55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7 for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:26:50 user nova-compute[72351]: DEBUG nova.compute.manager [req-ed3b7b69-01d0-414e-a1dd-69d48639c7df req-ed9796f4-afae-4edb-b8ba-0a1b12d1e535 service nova] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Received event network-vif-plugged-55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:26:50 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ed3b7b69-01d0-414e-a1dd-69d48639c7df req-ed9796f4-afae-4edb-b8ba-0a1b12d1e535 service nova] Acquiring lock "1d772c10-1c8a-4101-9a61-f8a60bb19c66-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:26:50 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ed3b7b69-01d0-414e-a1dd-69d48639c7df req-ed9796f4-afae-4edb-b8ba-0a1b12d1e535 service nova] Lock "1d772c10-1c8a-4101-9a61-f8a60bb19c66-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:26:50 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ed3b7b69-01d0-414e-a1dd-69d48639c7df req-ed9796f4-afae-4edb-b8ba-0a1b12d1e535 service nova] Lock "1d772c10-1c8a-4101-9a61-f8a60bb19c66-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:26:50 user nova-compute[72351]: DEBUG nova.compute.manager [req-ed3b7b69-01d0-414e-a1dd-69d48639c7df req-ed9796f4-afae-4edb-b8ba-0a1b12d1e535 service nova] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] No waiting events found dispatching network-vif-plugged-55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:26:50 user nova-compute[72351]: WARNING nova.compute.manager [req-ed3b7b69-01d0-414e-a1dd-69d48639c7df req-ed9796f4-afae-4edb-b8ba-0a1b12d1e535 service nova] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Received unexpected event network-vif-plugged-55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7 for instance with vm_state active and task_state deleting.
Aug 09 02:26:51 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:26:51 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Took 1.44 seconds to deallocate network for instance.
Aug 09 02:26:51 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-d11d4da6-17f0-4aa0-a53d-200dedac438d tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Acquiring lock "interface-f457e6ea-8d06-46c6-be2e-f650e28a3b69-None" by "nova.compute.manager.ComputeManager.attach_interface.<locals>.do_attach_interface" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:26:51 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-d11d4da6-17f0-4aa0-a53d-200dedac438d tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Lock "interface-f457e6ea-8d06-46c6-be2e-f650e28a3b69-None" acquired by "nova.compute.manager.ComputeManager.attach_interface.<locals>.do_attach_interface" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:26:51 user nova-compute[72351]: DEBUG nova.objects.instance [None req-d11d4da6-17f0-4aa0-a53d-200dedac438d tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Lazy-loading 'flavor' on Instance uuid f457e6ea-8d06-46c6-be2e-f650e28a3b69 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:26:51 user nova-compute[72351]: DEBUG nova.objects.instance [None req-d11d4da6-17f0-4aa0-a53d-200dedac438d tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Lazy-loading 'pci_requests' on Instance uuid f457e6ea-8d06-46c6-be2e-f650e28a3b69 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:26:51 user nova-compute[72351]: DEBUG nova.network.neutron [None req-d11d4da6-17f0-4aa0-a53d-200dedac438d tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] allocate_for_instance() {{(pid=72351) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Aug 09 02:26:51 user nova-compute[72351]: DEBUG nova.network.neutron [req-556acc23-3d5e-44cf-84d2-2589d589f5a1 req-529d10f6-f54a-42fd-89c9-efc852d07d00 service nova] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Updated VIF entry in instance network info cache for port dc244cf9-b0f6-4906-b9f3-53634b1847e3. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:26:51 user nova-compute[72351]: DEBUG nova.network.neutron [req-556acc23-3d5e-44cf-84d2-2589d589f5a1 req-529d10f6-f54a-42fd-89c9-efc852d07d00 service nova] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Updating instance_info_cache with network_info: [{"id": "dc244cf9-b0f6-4906-b9f3-53634b1847e3", "address": "fa:16:3e:0e:77:88", "network": {"id": "3fdd9499-7552-4ecc-a897-ed4c58be9519", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-969934765-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.20", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bde0634670914ae8bfbbc6b31c09d6dd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdc244cf9-b0", "ovs_interfaceid": "dc244cf9-b0f6-4906-b9f3-53634b1847e3", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:26:51 user nova-compute[72351]: DEBUG nova.policy [None req-d11d4da6-17f0-4aa0-a53d-200dedac438d tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '8159c9e78af04e01a1faf9b3cfc3591e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bde0634670914ae8bfbbc6b31c09d6dd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72351) authorize /opt/stack/nova/nova/policy.py:203}}
Aug 09 02:26:51 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-556acc23-3d5e-44cf-84d2-2589d589f5a1 req-529d10f6-f54a-42fd-89c9-efc852d07d00 service nova] Releasing lock "refresh_cache-f457e6ea-8d06-46c6-be2e-f650e28a3b69" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:26:52 user nova-compute[72351]: DEBUG nova.network.neutron [None req-d11d4da6-17f0-4aa0-a53d-200dedac438d tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Successfully created port: 9bf76200-4b93-40dc-bd6f-0200cec7933a {{(pid=72351) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Aug 09 02:26:52 user nova-compute[72351]: DEBUG nova.compute.manager [req-3944f747-4dba-4a99-a07b-a50d1673d5ab req-cc051803-d418-41a2-b1cb-87921e58daed service nova] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Received event network-vif-deleted-55e3cfd3-c6b5-4a85-9b8c-ba68a11380f7 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:26:52 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:53 user nova-compute[72351]: DEBUG nova.network.neutron [None req-d11d4da6-17f0-4aa0-a53d-200dedac438d tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Successfully updated port: 9bf76200-4b93-40dc-bd6f-0200cec7933a {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:26:53 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-d11d4da6-17f0-4aa0-a53d-200dedac438d tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Acquiring lock "refresh_cache-f457e6ea-8d06-46c6-be2e-f650e28a3b69" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:26:53 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-d11d4da6-17f0-4aa0-a53d-200dedac438d tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Acquired lock "refresh_cache-f457e6ea-8d06-46c6-be2e-f650e28a3b69" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:26:53 user nova-compute[72351]: DEBUG nova.network.neutron [None req-d11d4da6-17f0-4aa0-a53d-200dedac438d tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:26:53 user nova-compute[72351]: DEBUG nova.network.neutron [None req-d11d4da6-17f0-4aa0-a53d-200dedac438d tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Updating instance_info_cache with network_info: [{"id": "dc244cf9-b0f6-4906-b9f3-53634b1847e3", "address": "fa:16:3e:0e:77:88", "network": {"id": "3fdd9499-7552-4ecc-a897-ed4c58be9519", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-969934765-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.20", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bde0634670914ae8bfbbc6b31c09d6dd", "mtu": null, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdc244cf9-b0", "ovs_interfaceid": "dc244cf9-b0f6-4906-b9f3-53634b1847e3", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "9bf76200-4b93-40dc-bd6f-0200cec7933a", "address": "fa:16:3e:30:6e:79", "network": {"id": "ea588e91-dd01-4f9c-ad88-30f0848e6839", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-596948871", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.51", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "bde0634670914ae8bfbbc6b31c09d6dd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9bf76200-4b", "ovs_interfaceid": "9bf76200-4b93-40dc-bd6f-0200cec7933a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:26:53 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-d11d4da6-17f0-4aa0-a53d-200dedac438d tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Releasing lock "refresh_cache-f457e6ea-8d06-46c6-be2e-f650e28a3b69" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:26:53 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-d11d4da6-17f0-4aa0-a53d-200dedac438d tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-08-09T02:24:58Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-443615500',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-443615500',id=25,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBK8nrHk7zghGHoCUEI3vExNw2oXJc2YwDe7QrMxgC2DNUP/PkVzpLwy9DWnyTLCCR4ddGxe1t7xIbXts5zbiafpc3PmU6by4kyzLopbWZ1BYcJyQcmguVSz4dz8HMbCR9g==',key_name='tempest-keypair-1705746290',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:25:05Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=<?>,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='bde0634670914ae8bfbbc6b31c09d6dd',ramdisk_id='',reservation_id='r-5xxup5ze',resources=<?>,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedAttachmentsTest-480515778',owner_user_name='tempest-TaggedAttachmentsTest-480515778-project-member'},tags=<?>,task_state=None,terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:25:06Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='8159c9e78af04e01a1faf9b3cfc3591e',uuid=f457e6ea-8d06-46c6-be2e-f650e28a3b69,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "9bf76200-4b93-40dc-bd6f-0200cec7933a", "address": "fa:16:3e:30:6e:79", "network": {"id": "ea588e91-dd01-4f9c-ad88-30f0848e6839", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-596948871", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.51", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "bde0634670914ae8bfbbc6b31c09d6dd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9bf76200-4b", "ovs_interfaceid": "9bf76200-4b93-40dc-bd6f-0200cec7933a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:26:53 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-d11d4da6-17f0-4aa0-a53d-200dedac438d tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Converting VIF {"id": "9bf76200-4b93-40dc-bd6f-0200cec7933a", "address": "fa:16:3e:30:6e:79", "network": {"id": "ea588e91-dd01-4f9c-ad88-30f0848e6839", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-596948871", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.51", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "bde0634670914ae8bfbbc6b31c09d6dd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9bf76200-4b", "ovs_interfaceid": "9bf76200-4b93-40dc-bd6f-0200cec7933a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:26:53 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-d11d4da6-17f0-4aa0-a53d-200dedac438d tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:30:6e:79,bridge_name='br-int',has_traffic_filtering=True,id=9bf76200-4b93-40dc-bd6f-0200cec7933a,network=Network(ea588e91-dd01-4f9c-ad88-30f0848e6839),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap9bf76200-4b') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:26:53 user nova-compute[72351]: DEBUG os_vif [None req-d11d4da6-17f0-4aa0-a53d-200dedac438d tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:30:6e:79,bridge_name='br-int',has_traffic_filtering=True,id=9bf76200-4b93-40dc-bd6f-0200cec7933a,network=Network(ea588e91-dd01-4f9c-ad88-30f0848e6839),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap9bf76200-4b') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:26:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:26:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:26:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap9bf76200-4b, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:26:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap9bf76200-4b, col_values=(('external_ids', {'iface-id': '9bf76200-4b93-40dc-bd6f-0200cec7933a', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:30:6e:79', 'vm-uuid': 'f457e6ea-8d06-46c6-be2e-f650e28a3b69'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:26:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:26:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:53 user nova-compute[72351]: INFO os_vif [None req-d11d4da6-17f0-4aa0-a53d-200dedac438d tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:30:6e:79,bridge_name='br-int',has_traffic_filtering=True,id=9bf76200-4b93-40dc-bd6f-0200cec7933a,network=Network(ea588e91-dd01-4f9c-ad88-30f0848e6839),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap9bf76200-4b')
Aug 09 02:26:53 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-d11d4da6-17f0-4aa0-a53d-200dedac438d tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-08-09T02:24:58Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-443615500',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-443615500',id=25,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBK8nrHk7zghGHoCUEI3vExNw2oXJc2YwDe7QrMxgC2DNUP/PkVzpLwy9DWnyTLCCR4ddGxe1t7xIbXts5zbiafpc3PmU6by4kyzLopbWZ1BYcJyQcmguVSz4dz8HMbCR9g==',key_name='tempest-keypair-1705746290',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:25:05Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=<?>,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='bde0634670914ae8bfbbc6b31c09d6dd',ramdisk_id='',reservation_id='r-5xxup5ze',resources=<?>,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedAttachmentsTest-480515778',owner_user_name='tempest-TaggedAttachmentsTest-480515778-project-member'},tags=<?>,task_state=None,terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:25:06Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='8159c9e78af04e01a1faf9b3cfc3591e',uuid=f457e6ea-8d06-46c6-be2e-f650e28a3b69,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "9bf76200-4b93-40dc-bd6f-0200cec7933a", "address": "fa:16:3e:30:6e:79", "network": {"id": "ea588e91-dd01-4f9c-ad88-30f0848e6839", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-596948871", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.51", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "bde0634670914ae8bfbbc6b31c09d6dd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9bf76200-4b", "ovs_interfaceid": "9bf76200-4b93-40dc-bd6f-0200cec7933a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:26:53 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-d11d4da6-17f0-4aa0-a53d-200dedac438d tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Converting VIF {"id": "9bf76200-4b93-40dc-bd6f-0200cec7933a", "address": "fa:16:3e:30:6e:79", "network": {"id": "ea588e91-dd01-4f9c-ad88-30f0848e6839", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-596948871", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.51", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "bde0634670914ae8bfbbc6b31c09d6dd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9bf76200-4b", "ovs_interfaceid": "9bf76200-4b93-40dc-bd6f-0200cec7933a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:26:53 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-d11d4da6-17f0-4aa0-a53d-200dedac438d tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:30:6e:79,bridge_name='br-int',has_traffic_filtering=True,id=9bf76200-4b93-40dc-bd6f-0200cec7933a,network=Network(ea588e91-dd01-4f9c-ad88-30f0848e6839),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap9bf76200-4b') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:26:53 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-d11d4da6-17f0-4aa0-a53d-200dedac438d tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] attach device xml: <interface type="ethernet">
Aug 09 02:26:53 user nova-compute[72351]:   <mac address="fa:16:3e:30:6e:79"/>
Aug 09 02:26:53 user nova-compute[72351]:   <model type="virtio"/>
Aug 09 02:26:53 user nova-compute[72351]:   <mtu size="1442"/>
Aug 09 02:26:53 user nova-compute[72351]:   <target dev="tap9bf76200-4b"/>
Aug 09 02:26:53 user nova-compute[72351]: </interface>
Aug 09 02:26:53 user nova-compute[72351]:  {{(pid=72351) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Aug 09 02:26:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:54 user nova-compute[72351]: INFO nova.compute.manager [None req-f0fba6ea-3795-40eb-acec-2897ecefecaf tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Took 2.89 seconds to detach 1 volumes for instance.
Aug 09 02:26:54 user nova-compute[72351]: DEBUG nova.compute.manager [None req-f0fba6ea-3795-40eb-acec-2897ecefecaf tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Deleting volume: ba022fd7-964b-4325-9d01-74fb60f45274 {{(pid=72351) _cleanup_volumes /opt/stack/nova/nova/compute/manager.py:3221}}
Aug 09 02:26:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-d11d4da6-17f0-4aa0-a53d-200dedac438d tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:26:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-d11d4da6-17f0-4aa0-a53d-200dedac438d tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] No BDM found with device name hda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:26:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-d11d4da6-17f0-4aa0-a53d-200dedac438d tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] No VIF found with MAC fa:16:3e:0e:77:88, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:26:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-d11d4da6-17f0-4aa0-a53d-200dedac438d tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] set metadata xml: <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:26:54 user nova-compute[72351]:   <nova:package version="0.0.0"/>
Aug 09 02:26:54 user nova-compute[72351]:   <nova:name>tempest-device-tagging-server-443615500</nova:name>
Aug 09 02:26:54 user nova-compute[72351]:   <nova:creationTime>2023-08-09 02:26:54</nova:creationTime>
Aug 09 02:26:54 user nova-compute[72351]:   <nova:flavor name="m1.tiny">
Aug 09 02:26:54 user nova-compute[72351]:     <nova:memory>512</nova:memory>
Aug 09 02:26:54 user nova-compute[72351]:     <nova:disk>1</nova:disk>
Aug 09 02:26:54 user nova-compute[72351]:     <nova:swap>0</nova:swap>
Aug 09 02:26:54 user nova-compute[72351]:     <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:26:54 user nova-compute[72351]:     <nova:vcpus>1</nova:vcpus>
Aug 09 02:26:54 user nova-compute[72351]:   </nova:flavor>
Aug 09 02:26:54 user nova-compute[72351]:   <nova:owner>
Aug 09 02:26:54 user nova-compute[72351]:     <nova:user uuid="8159c9e78af04e01a1faf9b3cfc3591e">tempest-TaggedAttachmentsTest-480515778-project-member</nova:user>
Aug 09 02:26:54 user nova-compute[72351]:     <nova:project uuid="bde0634670914ae8bfbbc6b31c09d6dd">tempest-TaggedAttachmentsTest-480515778</nova:project>
Aug 09 02:26:54 user nova-compute[72351]:   </nova:owner>
Aug 09 02:26:54 user nova-compute[72351]:   <nova:root type="image" uuid="18ccda7f-31d3-4978-ab9d-4ed2f74eff94"/>
Aug 09 02:26:54 user nova-compute[72351]:   <nova:ports>
Aug 09 02:26:54 user nova-compute[72351]:     <nova:port uuid="dc244cf9-b0f6-4906-b9f3-53634b1847e3">
Aug 09 02:26:54 user nova-compute[72351]:       <nova:ip type="fixed" address="10.0.0.9" ipVersion="4"/>
Aug 09 02:26:54 user nova-compute[72351]:     </nova:port>
Aug 09 02:26:54 user nova-compute[72351]:     <nova:port uuid="9bf76200-4b93-40dc-bd6f-0200cec7933a">
Aug 09 02:26:54 user nova-compute[72351]:       <nova:ip type="fixed" address="10.10.10.51" ipVersion="4"/>
Aug 09 02:26:54 user nova-compute[72351]:     </nova:port>
Aug 09 02:26:54 user nova-compute[72351]:   </nova:ports>
Aug 09 02:26:54 user nova-compute[72351]: </nova:instance>
Aug 09 02:26:54 user nova-compute[72351]:  {{(pid=72351) set_metadata /opt/stack/nova/nova/virt/libvirt/guest.py:359}}
Aug 09 02:26:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-d11d4da6-17f0-4aa0-a53d-200dedac438d tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Lock "interface-f457e6ea-8d06-46c6-be2e-f650e28a3b69-None" "released" by "nova.compute.manager.ComputeManager.attach_interface.<locals>.do_attach_interface" :: held 3.036s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:26:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f0fba6ea-3795-40eb-acec-2897ecefecaf tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:26:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f0fba6ea-3795-40eb-acec-2897ecefecaf tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:26:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-884142d8-9560-4d7c-93f1-388d5b9fede8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Acquiring lock "433e6729-fdd4-4467-971c-08ca962d887d" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:26:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-884142d8-9560-4d7c-93f1-388d5b9fede8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lock "433e6729-fdd4-4467-971c-08ca962d887d" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:26:54 user nova-compute[72351]: INFO nova.compute.manager [None req-884142d8-9560-4d7c-93f1-388d5b9fede8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Detaching volume f19fe8f8-5857-44ff-bf31-b2f4a7671a6b
Aug 09 02:26:54 user nova-compute[72351]: INFO nova.virt.block_device [None req-884142d8-9560-4d7c-93f1-388d5b9fede8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Attempting to driver detach volume f19fe8f8-5857-44ff-bf31-b2f4a7671a6b from mountpoint /dev/vdb
Aug 09 02:26:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-884142d8-9560-4d7c-93f1-388d5b9fede8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Attempting to detach device vdb from instance 433e6729-fdd4-4467-971c-08ca962d887d from the persistent domain config. {{(pid=72351) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2477}}
Aug 09 02:26:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-884142d8-9560-4d7c-93f1-388d5b9fede8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 02:26:54 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:26:54 user nova-compute[72351]:   <source dev="/dev/sdb"/>
Aug 09 02:26:54 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:26:54 user nova-compute[72351]:   <serial>f19fe8f8-5857-44ff-bf31-b2f4a7671a6b</serial>
Aug 09 02:26:54 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Aug 09 02:26:54 user nova-compute[72351]: </disk>
Aug 09 02:26:54 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 02:26:54 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-884142d8-9560-4d7c-93f1-388d5b9fede8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Successfully detached device vdb from instance 433e6729-fdd4-4467-971c-08ca962d887d from the persistent domain config.
Aug 09 02:26:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-884142d8-9560-4d7c-93f1-388d5b9fede8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 433e6729-fdd4-4467-971c-08ca962d887d from the live domain config. {{(pid=72351) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2513}}
Aug 09 02:26:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-884142d8-9560-4d7c-93f1-388d5b9fede8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 02:26:54 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:26:54 user nova-compute[72351]:   <source dev="/dev/sdb"/>
Aug 09 02:26:54 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:26:54 user nova-compute[72351]:   <serial>f19fe8f8-5857-44ff-bf31-b2f4a7671a6b</serial>
Aug 09 02:26:54 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Aug 09 02:26:54 user nova-compute[72351]: </disk>
Aug 09 02:26:54 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 02:26:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-afe1e9d0-2188-414b-9f8f-f1de9d143ef8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Acquiring lock "433e6729-fdd4-4467-971c-08ca962d887d" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:26:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1fa2042e-4a2d-4778-bb20-240b20e19cbb tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Acquiring lock "f457e6ea-8d06-46c6-be2e-f650e28a3b69" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:26:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1fa2042e-4a2d-4778-bb20-240b20e19cbb tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Lock "f457e6ea-8d06-46c6-be2e-f650e28a3b69" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:26:54 user nova-compute[72351]: DEBUG nova.objects.instance [None req-1fa2042e-4a2d-4778-bb20-240b20e19cbb tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Lazy-loading 'flavor' on Instance uuid f457e6ea-8d06-46c6-be2e-f650e28a3b69 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:26:54 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-f0fba6ea-3795-40eb-acec-2897ecefecaf tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:26:54 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-f0fba6ea-3795-40eb-acec-2897ecefecaf tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:26:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1fa2042e-4a2d-4778-bb20-240b20e19cbb tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Lock "f457e6ea-8d06-46c6-be2e-f650e28a3b69" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.056s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:26:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f0fba6ea-3795-40eb-acec-2897ecefecaf tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.482s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:26:54 user nova-compute[72351]: DEBUG nova.compute.manager [req-a28c3864-e496-4d9a-9124-58af69bc4c2e req-9daa823e-ecc2-459a-8b67-facfecec892c service nova] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Received event network-changed-9bf76200-4b93-40dc-bd6f-0200cec7933a {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:26:54 user nova-compute[72351]: DEBUG nova.compute.manager [req-a28c3864-e496-4d9a-9124-58af69bc4c2e req-9daa823e-ecc2-459a-8b67-facfecec892c service nova] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Refreshing instance network info cache due to event network-changed-9bf76200-4b93-40dc-bd6f-0200cec7933a. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:26:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a28c3864-e496-4d9a-9124-58af69bc4c2e req-9daa823e-ecc2-459a-8b67-facfecec892c service nova] Acquiring lock "refresh_cache-f457e6ea-8d06-46c6-be2e-f650e28a3b69" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:26:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a28c3864-e496-4d9a-9124-58af69bc4c2e req-9daa823e-ecc2-459a-8b67-facfecec892c service nova] Acquired lock "refresh_cache-f457e6ea-8d06-46c6-be2e-f650e28a3b69" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:26:54 user nova-compute[72351]: DEBUG nova.network.neutron [req-a28c3864-e496-4d9a-9124-58af69bc4c2e req-9daa823e-ecc2-459a-8b67-facfecec892c service nova] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Refreshing network info cache for port 9bf76200-4b93-40dc-bd6f-0200cec7933a {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:26:54 user nova-compute[72351]: INFO nova.scheduler.client.report [None req-f0fba6ea-3795-40eb-acec-2897ecefecaf tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Deleted allocations for instance 1d772c10-1c8a-4101-9a61-f8a60bb19c66
Aug 09 02:26:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f0fba6ea-3795-40eb-acec-2897ecefecaf tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "1d772c10-1c8a-4101-9a61-f8a60bb19c66" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 6.583s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1fa2042e-4a2d-4778-bb20-240b20e19cbb tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Acquiring lock "f457e6ea-8d06-46c6-be2e-f650e28a3b69" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1fa2042e-4a2d-4778-bb20-240b20e19cbb tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Lock "f457e6ea-8d06-46c6-be2e-f650e28a3b69" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:26:55 user nova-compute[72351]: INFO nova.compute.manager [None req-1fa2042e-4a2d-4778-bb20-240b20e19cbb tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Attaching volume 11ec2ffe-3db9-4d44-8f09-316c97ad7b80 to /dev/vdb
Aug 09 02:26:55 user nova-compute[72351]: DEBUG os_brick.utils [None req-1fa2042e-4a2d-4778-bb20-240b20e19cbb tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.015s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[0a02af15-7421-477c-8dc0-6460ed62af40]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:a52269602ff4\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.016s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d7c9b1bd-232f-40f4-945c-f41a4235940f]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.016s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[fec5dd4c-06be-43ad-91c6-21d087904efd]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[1ef519a2-b071-466f-86b8-eeda6f374298]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-1fa2042e-4a2d-4778-bb20-240b20e19cbb tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Running cmd (subprocess): nvme version {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-1fa2042e-4a2d-4778-bb20-240b20e19cbb tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] 'nvme version' failed. Not Retrying. {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.nvmeof [None req-1fa2042e-4a2d-4778-bb20-240b20e19cbb tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] nvme not present on system {{(pid=72351) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:26:55 user nova-compute[72351]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Aug 09 02:26:55 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[456d7fea-47c9-458f-91b4-a34a8966e236]: (4, '') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-1fa2042e-4a2d-4778-bb20-240b20e19cbb tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72351) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-1fa2042e-4a2d-4778-bb20-240b20e19cbb tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-1fa2042e-4a2d-4778-bb20-240b20e19cbb tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] LIGHTOS: no hostnqn found. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG os_brick.utils [None req-1fa2042e-4a2d-4778-bb20-240b20e19cbb tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] <== get_connector_properties: return (137ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:a52269602ff4', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG nova.virt.block_device [None req-1fa2042e-4a2d-4778-bb20-240b20e19cbb tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Updating existing volume attachment record: 33356b02-1bba-4b72-91ec-7a2e14fd4a3b {{(pid=72351) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Received event <DeviceRemovedEvent: 1691548015.2521648, 433e6729-fdd4-4467-971c-08ca962d887d => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=72351) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2360}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-884142d8-9560-4d7c-93f1-388d5b9fede8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 433e6729-fdd4-4467-971c-08ca962d887d {{(pid=72351) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2589}}
Aug 09 02:26:55 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-884142d8-9560-4d7c-93f1-388d5b9fede8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Successfully detached device vdb from instance 433e6729-fdd4-4467-971c-08ca962d887d from the live domain config.
Aug 09 02:26:55 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-884142d8-9560-4d7c-93f1-388d5b9fede8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-884142d8-9560-4d7c-93f1-388d5b9fede8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'HGSEFVOU', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb'}, None), 'kwargs': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:26:55 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-884142d8-9560-4d7c-93f1-388d5b9fede8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:26:55 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-884142d8-9560-4d7c-93f1-388d5b9fede8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-884142d8-9560-4d7c-93f1-388d5b9fede8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-884142d8-9560-4d7c-93f1-388d5b9fede8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[98d57f4c-d7c8-4e10-8d0b-0d024f9b5e41]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.032s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[95e8ca00-1fc2-4cd9-a989-aea8b85624ed]: (4, ('tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)\ntcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b (non-flash)\ntcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)\ntcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)\ntcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-884142d8-9560-4d7c-93f1-388d5b9fede8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:26:55 user nova-compute[72351]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b (non-flash)
Aug 09 02:26:55 user nova-compute[72351]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)
Aug 09 02:26:55 user nova-compute[72351]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)
Aug 09 02:26:55 user nova-compute[72351]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:26:55 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-884142d8-9560-4d7c-93f1-388d5b9fede8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] iscsi session list stdout=tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:26:55 user nova-compute[72351]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b (non-flash)
Aug 09 02:26:55 user nova-compute[72351]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)
Aug 09 02:26:55 user nova-compute[72351]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)
Aug 09 02:26:55 user nova-compute[72351]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:26:55 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-884142d8-9560-4d7c-93f1-388d5b9fede8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fd00b3d6200>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b'): ({'sdb'}, set())}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-884142d8-9560-4d7c-93f1-388d5b9fede8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Removing single pathed devices sdb {{(pid=72351) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[dfc3c900-6dc7-4a99-8d71-41e91dd7f7d8]: (4, ('path checker states:\nup                  4\npending             1\n\npaths: 0\nbusy: False\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[626e89ca-b059-4137-943e-2afec2e2f277]: (4, ('ok\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-884142d8-9560-4d7c-93f1-388d5b9fede8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Flushing IO for device /dev/sdb {{(pid=72351) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.012s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[b94dfe24-46b6-405c-9c7d-b3e42c09d3ae]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-884142d8-9560-4d7c-93f1-388d5b9fede8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=72351) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG nova.network.neutron [req-a28c3864-e496-4d9a-9124-58af69bc4c2e req-9daa823e-ecc2-459a-8b67-facfecec892c service nova] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Updated VIF entry in instance network info cache for port 9bf76200-4b93-40dc-bd6f-0200cec7933a. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG nova.network.neutron [req-a28c3864-e496-4d9a-9124-58af69bc4c2e req-9daa823e-ecc2-459a-8b67-facfecec892c service nova] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Updating instance_info_cache with network_info: [{"id": "dc244cf9-b0f6-4906-b9f3-53634b1847e3", "address": "fa:16:3e:0e:77:88", "network": {"id": "3fdd9499-7552-4ecc-a897-ed4c58be9519", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-969934765-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.20", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bde0634670914ae8bfbbc6b31c09d6dd", "mtu": null, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdc244cf9-b0", "ovs_interfaceid": "dc244cf9-b0f6-4906-b9f3-53634b1847e3", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "9bf76200-4b93-40dc-bd6f-0200cec7933a", "address": "fa:16:3e:30:6e:79", "network": {"id": "ea588e91-dd01-4f9c-ad88-30f0848e6839", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-596948871", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.51", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "bde0634670914ae8bfbbc6b31c09d6dd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9bf76200-4b", "ovs_interfaceid": "9bf76200-4b93-40dc-bd6f-0200cec7933a", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.049s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[f92a6440-a0c5-4951-9c12-17013c842598]: (4, ('1', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-884142d8-9560-4d7c-93f1-388d5b9fede8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-884142d8-9560-4d7c-93f1-388d5b9fede8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] SCSI volumes sdb have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-884142d8-9560-4d7c-93f1-388d5b9fede8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b')] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a28c3864-e496-4d9a-9124-58af69bc4c2e req-9daa823e-ecc2-459a-8b67-facfecec892c service nova] Releasing lock "refresh_cache-f457e6ea-8d06-46c6-be2e-f650e28a3b69" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG nova.compute.manager [req-a28c3864-e496-4d9a-9124-58af69bc4c2e req-9daa823e-ecc2-459a-8b67-facfecec892c service nova] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Received event network-vif-plugged-9bf76200-4b93-40dc-bd6f-0200cec7933a {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a28c3864-e496-4d9a-9124-58af69bc4c2e req-9daa823e-ecc2-459a-8b67-facfecec892c service nova] Acquiring lock "f457e6ea-8d06-46c6-be2e-f650e28a3b69-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a28c3864-e496-4d9a-9124-58af69bc4c2e req-9daa823e-ecc2-459a-8b67-facfecec892c service nova] Lock "f457e6ea-8d06-46c6-be2e-f650e28a3b69-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a28c3864-e496-4d9a-9124-58af69bc4c2e req-9daa823e-ecc2-459a-8b67-facfecec892c service nova] Lock "f457e6ea-8d06-46c6-be2e-f650e28a3b69-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG nova.compute.manager [req-a28c3864-e496-4d9a-9124-58af69bc4c2e req-9daa823e-ecc2-459a-8b67-facfecec892c service nova] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] No waiting events found dispatching network-vif-plugged-9bf76200-4b93-40dc-bd6f-0200cec7933a {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:26:55 user nova-compute[72351]: WARNING nova.compute.manager [req-a28c3864-e496-4d9a-9124-58af69bc4c2e req-9daa823e-ecc2-459a-8b67-facfecec892c service nova] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Received unexpected event network-vif-plugged-9bf76200-4b93-40dc-bd6f-0200cec7933a for instance with vm_state active and task_state None.
Aug 09 02:26:55 user nova-compute[72351]: DEBUG nova.compute.manager [req-a28c3864-e496-4d9a-9124-58af69bc4c2e req-9daa823e-ecc2-459a-8b67-facfecec892c service nova] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Received event network-vif-plugged-9bf76200-4b93-40dc-bd6f-0200cec7933a {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a28c3864-e496-4d9a-9124-58af69bc4c2e req-9daa823e-ecc2-459a-8b67-facfecec892c service nova] Acquiring lock "f457e6ea-8d06-46c6-be2e-f650e28a3b69-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a28c3864-e496-4d9a-9124-58af69bc4c2e req-9daa823e-ecc2-459a-8b67-facfecec892c service nova] Lock "f457e6ea-8d06-46c6-be2e-f650e28a3b69-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a28c3864-e496-4d9a-9124-58af69bc4c2e req-9daa823e-ecc2-459a-8b67-facfecec892c service nova] Lock "f457e6ea-8d06-46c6-be2e-f650e28a3b69-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG nova.compute.manager [req-a28c3864-e496-4d9a-9124-58af69bc4c2e req-9daa823e-ecc2-459a-8b67-facfecec892c service nova] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] No waiting events found dispatching network-vif-plugged-9bf76200-4b93-40dc-bd6f-0200cec7933a {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:26:55 user nova-compute[72351]: WARNING nova.compute.manager [req-a28c3864-e496-4d9a-9124-58af69bc4c2e req-9daa823e-ecc2-459a-8b67-facfecec892c service nova] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Received unexpected event network-vif-plugged-9bf76200-4b93-40dc-bd6f-0200cec7933a for instance with vm_state active and task_state None.
Aug 09 02:26:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a7fdc95f-bd32-4eba-a84c-d650be39a367]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-884142d8-9560-4d7c-93f1-388d5b9fede8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b -p 172.16.0.220:3260 --logout {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b -p 172.16.0.220:3260 --logout" returned: 0 in 0.083s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[553b34d7-f6f3-4df5-a965-a9643eaf5eff]: (4, ('Logging out of session [sid: 22, target: iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b, portal: 172.16.0.220,3260]\nLogout of [sid: 22, target: iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-884142d8-9560-4d7c-93f1-388d5b9fede8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 22, target: iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b, portal: 172.16.0.220,3260]
Aug 09 02:26:55 user nova-compute[72351]: Logout of [sid: 22, target: iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b, portal: 172.16.0.220,3260] successful.
Aug 09 02:26:55 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b -p 172.16.0.220:3260 --op delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b -p 172.16.0.220:3260 --op delete" returned: 0 in 0.027s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d4cfd2f3-c7d2-4b3a-9366-f59cad46c23f]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-884142d8-9560-4d7c-93f1-388d5b9fede8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-884142d8-9560-4d7c-93f1-388d5b9fede8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.325s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-884142d8-9560-4d7c-93f1-388d5b9fede8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] <== disconnect_volume: return (327ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:26:55 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-884142d8-9560-4d7c-93f1-388d5b9fede8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 02:26:56 user nova-compute[72351]: DEBUG nova.objects.instance [None req-884142d8-9560-4d7c-93f1-388d5b9fede8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lazy-loading 'flavor' on Instance uuid 433e6729-fdd4-4467-971c-08ca962d887d {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:26:56 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-884142d8-9560-4d7c-93f1-388d5b9fede8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lock "433e6729-fdd4-4467-971c-08ca962d887d" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.871s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:26:56 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-afe1e9d0-2188-414b-9f8f-f1de9d143ef8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lock "433e6729-fdd4-4467-971c-08ca962d887d" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 1.737s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:26:56 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-afe1e9d0-2188-414b-9f8f-f1de9d143ef8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Acquiring lock "433e6729-fdd4-4467-971c-08ca962d887d-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:26:56 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-afe1e9d0-2188-414b-9f8f-f1de9d143ef8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lock "433e6729-fdd4-4467-971c-08ca962d887d-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:26:56 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-afe1e9d0-2188-414b-9f8f-f1de9d143ef8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lock "433e6729-fdd4-4467-971c-08ca962d887d-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:26:56 user nova-compute[72351]: INFO nova.compute.manager [None req-afe1e9d0-2188-414b-9f8f-f1de9d143ef8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Terminating instance
Aug 09 02:26:56 user nova-compute[72351]: DEBUG nova.compute.manager [None req-afe1e9d0-2188-414b-9f8f-f1de9d143ef8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Start destroying the instance on the hypervisor. {{(pid=72351) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3124}}
Aug 09 02:26:56 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:56 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:56 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:57 user nova-compute[72351]: DEBUG nova.compute.manager [req-74c44ab1-d4fc-4e58-a851-8f51084d040b req-aca099f4-c0d3-495a-b7cc-8a6d5b18478c service nova] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Received event network-vif-unplugged-7f4f4044-2549-41b0-b1b8-9154419a8b54 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:26:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-74c44ab1-d4fc-4e58-a851-8f51084d040b req-aca099f4-c0d3-495a-b7cc-8a6d5b18478c service nova] Acquiring lock "433e6729-fdd4-4467-971c-08ca962d887d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:26:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-74c44ab1-d4fc-4e58-a851-8f51084d040b req-aca099f4-c0d3-495a-b7cc-8a6d5b18478c service nova] Lock "433e6729-fdd4-4467-971c-08ca962d887d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:26:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-74c44ab1-d4fc-4e58-a851-8f51084d040b req-aca099f4-c0d3-495a-b7cc-8a6d5b18478c service nova] Lock "433e6729-fdd4-4467-971c-08ca962d887d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:26:57 user nova-compute[72351]: DEBUG nova.compute.manager [req-74c44ab1-d4fc-4e58-a851-8f51084d040b req-aca099f4-c0d3-495a-b7cc-8a6d5b18478c service nova] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] No waiting events found dispatching network-vif-unplugged-7f4f4044-2549-41b0-b1b8-9154419a8b54 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:26:57 user nova-compute[72351]: DEBUG nova.compute.manager [req-74c44ab1-d4fc-4e58-a851-8f51084d040b req-aca099f4-c0d3-495a-b7cc-8a6d5b18478c service nova] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Received event network-vif-unplugged-7f4f4044-2549-41b0-b1b8-9154419a8b54 for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:26:57 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Instance destroyed successfully.
Aug 09 02:26:57 user nova-compute[72351]: DEBUG nova.objects.instance [None req-afe1e9d0-2188-414b-9f8f-f1de9d143ef8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lazy-loading 'resources' on Instance uuid 433e6729-fdd4-4467-971c-08ca962d887d {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:26:57 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-afe1e9d0-2188-414b-9f8f-f1de9d143ef8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:24:06Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-VolumesSnapshotTestJSON-instance-1098155920',display_name='tempest-VolumesSnapshotTestJSON-instance-1098155920',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-volumessnapshottestjson-instance-1098155920',id=22,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBCAKJgeFJGn197soLO4TKV3jbPbEAz0hnZRWzRGb1+UBTzihAr6/WvY3HiLk3OFxEU+jj/jJC5OiHDoEcaue5hFL+F0PeY1vCaX7fDZVIgbnkzaBADdLVYiBoPx//f/J0g==',key_name='tempest-keypair-1954238676',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:24:13Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='1c8ff52319444079b754f79843c67b2b',ramdisk_id='',reservation_id='r-a3jy1i1h',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-VolumesSnapshotTestJSON-343261326',owner_user_name='tempest-VolumesSnapshotTestJSON-343261326-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:24:14Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='97267b9f897e4034b71627a698210071',uuid=433e6729-fdd4-4467-971c-08ca962d887d,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "7f4f4044-2549-41b0-b1b8-9154419a8b54", "address": "fa:16:3e:04:53:28", "network": {"id": "96018a29-3b2e-44de-b2c4-67abc92e3a30", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-924549970-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.25", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "1c8ff52319444079b754f79843c67b2b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7f4f4044-25", "ovs_interfaceid": "7f4f4044-2549-41b0-b1b8-9154419a8b54", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:26:57 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-afe1e9d0-2188-414b-9f8f-f1de9d143ef8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Converting VIF {"id": "7f4f4044-2549-41b0-b1b8-9154419a8b54", "address": "fa:16:3e:04:53:28", "network": {"id": "96018a29-3b2e-44de-b2c4-67abc92e3a30", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-924549970-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.25", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "1c8ff52319444079b754f79843c67b2b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7f4f4044-25", "ovs_interfaceid": "7f4f4044-2549-41b0-b1b8-9154419a8b54", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:26:57 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-afe1e9d0-2188-414b-9f8f-f1de9d143ef8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:04:53:28,bridge_name='br-int',has_traffic_filtering=True,id=7f4f4044-2549-41b0-b1b8-9154419a8b54,network=Network(96018a29-3b2e-44de-b2c4-67abc92e3a30),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7f4f4044-25') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:26:57 user nova-compute[72351]: DEBUG os_vif [None req-afe1e9d0-2188-414b-9f8f-f1de9d143ef8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:04:53:28,bridge_name='br-int',has_traffic_filtering=True,id=7f4f4044-2549-41b0-b1b8-9154419a8b54,network=Network(96018a29-3b2e-44de-b2c4-67abc92e3a30),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7f4f4044-25') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:26:57 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:57 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap7f4f4044-25, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:26:57 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:57 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:57 user nova-compute[72351]: INFO os_vif [None req-afe1e9d0-2188-414b-9f8f-f1de9d143ef8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:04:53:28,bridge_name='br-int',has_traffic_filtering=True,id=7f4f4044-2549-41b0-b1b8-9154419a8b54,network=Network(96018a29-3b2e-44de-b2c4-67abc92e3a30),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7f4f4044-25')
Aug 09 02:26:57 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-afe1e9d0-2188-414b-9f8f-f1de9d143ef8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 02:26:57 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-afe1e9d0-2188-414b-9f8f-f1de9d143ef8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'HGSEFVOU', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb'}, None), 'kwargs': True}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:26:57 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-afe1e9d0-2188-414b-9f8f-f1de9d143ef8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:26:57 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-afe1e9d0-2188-414b-9f8f-f1de9d143ef8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:26:57 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-afe1e9d0-2188-414b-9f8f-f1de9d143ef8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:26:57 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-afe1e9d0-2188-414b-9f8f-f1de9d143ef8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:f19fe8f8-5857-44ff-bf31-b2f4a7671a6b', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 02:26:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:57 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[fca6fff1-626d-49e0-b22c-c25ae74d1f13]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:26:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:26:57 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[1aac962e-ddf1-4d2a-8c3a-839844088544]: (4, ('tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)\ntcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)\ntcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)\ntcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:26:57 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-afe1e9d0-2188-414b-9f8f-f1de9d143ef8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:26:57 user nova-compute[72351]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)
Aug 09 02:26:57 user nova-compute[72351]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)
Aug 09 02:26:57 user nova-compute[72351]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:26:57 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:26:57 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-afe1e9d0-2188-414b-9f8f-f1de9d143ef8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] iscsi session list stdout=tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:26:57 user nova-compute[72351]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)
Aug 09 02:26:57 user nova-compute[72351]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)
Aug 09 02:26:57 user nova-compute[72351]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:26:57 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:26:57 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-afe1e9d0-2188-414b-9f8f-f1de9d143ef8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fcfdc423a30>, {}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 02:26:57 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-afe1e9d0-2188-414b-9f8f-f1de9d143ef8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Disconnecting from: [] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 02:26:57 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-afe1e9d0-2188-414b-9f8f-f1de9d143ef8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.051s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:26:57 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-afe1e9d0-2188-414b-9f8f-f1de9d143ef8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] <== disconnect_volume: return (52ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:26:57 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-afe1e9d0-2188-414b-9f8f-f1de9d143ef8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 02:26:57 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-afe1e9d0-2188-414b-9f8f-f1de9d143ef8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Deleting instance files /opt/stack/data/nova/instances/433e6729-fdd4-4467-971c-08ca962d887d_del
Aug 09 02:26:57 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-afe1e9d0-2188-414b-9f8f-f1de9d143ef8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Deletion of /opt/stack/data/nova/instances/433e6729-fdd4-4467-971c-08ca962d887d_del complete
Aug 09 02:26:57 user nova-compute[72351]: INFO nova.compute.manager [None req-afe1e9d0-2188-414b-9f8f-f1de9d143ef8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Took 0.92 seconds to destroy the instance on the hypervisor.
Aug 09 02:26:57 user nova-compute[72351]: DEBUG oslo.service.loopingcall [None req-afe1e9d0-2188-414b-9f8f-f1de9d143ef8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72351) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Aug 09 02:26:57 user nova-compute[72351]: DEBUG nova.compute.manager [-] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Deallocating network for instance {{(pid=72351) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2263}}
Aug 09 02:26:57 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] deallocate_for_instance() {{(pid=72351) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1795}}
Aug 09 02:26:57 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:26:58 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:26:58 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Took 1.12 seconds to deallocate network for instance.
Aug 09 02:26:58 user nova-compute[72351]: WARNING nova.volume.cinder [None req-afe1e9d0-2188-414b-9f8f-f1de9d143ef8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Attachment 844fc477-8700-4494-9ebc-7f4f361f6fce does not exist. Ignoring.: cinderclient.exceptions.NotFound: Volume attachment could not be found with filter: attachment_id = 844fc477-8700-4494-9ebc-7f4f361f6fce. (HTTP 404) (Request-ID: req-1844a5fd-b11e-425e-b489-ef61182568ec)
Aug 09 02:26:58 user nova-compute[72351]: INFO nova.compute.manager [None req-afe1e9d0-2188-414b-9f8f-f1de9d143ef8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Took 0.11 seconds to detach 1 volumes for instance.
Aug 09 02:26:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-afe1e9d0-2188-414b-9f8f-f1de9d143ef8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:26:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-afe1e9d0-2188-414b-9f8f-f1de9d143ef8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:26:58 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:26:58 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:26:58 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72351) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10461}}
Aug 09 02:26:59 user nova-compute[72351]: DEBUG nova.compute.manager [req-b737d97f-a476-4438-8258-170f61619c9a req-4bfc8939-e758-4cf2-acc5-c169f4826053 service nova] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Received event network-vif-plugged-7f4f4044-2549-41b0-b1b8-9154419a8b54 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:26:59 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-b737d97f-a476-4438-8258-170f61619c9a req-4bfc8939-e758-4cf2-acc5-c169f4826053 service nova] Acquiring lock "433e6729-fdd4-4467-971c-08ca962d887d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:26:59 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-b737d97f-a476-4438-8258-170f61619c9a req-4bfc8939-e758-4cf2-acc5-c169f4826053 service nova] Lock "433e6729-fdd4-4467-971c-08ca962d887d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:26:59 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-b737d97f-a476-4438-8258-170f61619c9a req-4bfc8939-e758-4cf2-acc5-c169f4826053 service nova] Lock "433e6729-fdd4-4467-971c-08ca962d887d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:26:59 user nova-compute[72351]: DEBUG nova.compute.manager [req-b737d97f-a476-4438-8258-170f61619c9a req-4bfc8939-e758-4cf2-acc5-c169f4826053 service nova] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] No waiting events found dispatching network-vif-plugged-7f4f4044-2549-41b0-b1b8-9154419a8b54 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:26:59 user nova-compute[72351]: WARNING nova.compute.manager [req-b737d97f-a476-4438-8258-170f61619c9a req-4bfc8939-e758-4cf2-acc5-c169f4826053 service nova] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Received unexpected event network-vif-plugged-7f4f4044-2549-41b0-b1b8-9154419a8b54 for instance with vm_state deleted and task_state None.
Aug 09 02:26:59 user nova-compute[72351]: DEBUG nova.compute.manager [req-b737d97f-a476-4438-8258-170f61619c9a req-4bfc8939-e758-4cf2-acc5-c169f4826053 service nova] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Received event network-vif-deleted-7f4f4044-2549-41b0-b1b8-9154419a8b54 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:26:59 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-afe1e9d0-2188-414b-9f8f-f1de9d143ef8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:26:59 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-afe1e9d0-2188-414b-9f8f-f1de9d143ef8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:26:59 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-afe1e9d0-2188-414b-9f8f-f1de9d143ef8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.359s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:26:59 user nova-compute[72351]: INFO nova.scheduler.client.report [None req-afe1e9d0-2188-414b-9f8f-f1de9d143ef8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Deleted allocations for instance 433e6729-fdd4-4467-971c-08ca962d887d
Aug 09 02:26:59 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-afe1e9d0-2188-414b-9f8f-f1de9d143ef8 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lock "433e6729-fdd4-4467-971c-08ca962d887d" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.775s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:26:59 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:26:59 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:26:59 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:26:59 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:26:59 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Auditing locally available compute resources for user (node: user) {{(pid=72351) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Aug 09 02:27:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json" returned: 0 in 0.307s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json" returned: 0 in 0.183s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:00 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdf (vdb) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:27:00 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdg (vda) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:27:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/47afce95-e5f7-4703-a99f-7ef8b28c4bd2/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/47afce95-e5f7-4703-a99f-7ef8b28c4bd2/disk --force-share --output=json" returned: 0 in 0.142s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/47afce95-e5f7-4703-a99f-7ef8b28c4bd2/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/47afce95-e5f7-4703-a99f-7ef8b28c4bd2/disk --force-share --output=json" returned: 0 in 0.148s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.rescue --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.rescue --force-share --output=json" returned: 0 in 0.161s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.rescue --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:01 user nova-compute[72351]: DEBUG nova.network.neutron [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Port be4ec483-8532-45f7-bb9f-0b50952ce94b binding to destination host user is already ACTIVE {{(pid=72351) migrate_instance_start /opt/stack/nova/nova/network/neutron.py:3163}}
Aug 09 02:27:01 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Acquiring lock "refresh_cache-75bbbead-8628-448b-9484-19bf4ab922ff" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:27:01 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Acquired lock "refresh_cache-75bbbead-8628-448b-9484-19bf4ab922ff" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:27:01 user nova-compute[72351]: DEBUG nova.network.neutron [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:27:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.rescue --force-share --output=json" returned: 0 in 0.158s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json" returned: 0 in 0.145s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json" returned: 0 in 0.140s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f457e6ea-8d06-46c6-be2e-f650e28a3b69/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:01 user nova-compute[72351]: DEBUG nova.network.neutron [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Updating instance_info_cache with network_info: [{"id": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "address": "fa:16:3e:8b:b5:2e", "network": {"id": "8231f2c2-237e-41c6-8892-720b950c1453", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherB-1917380761-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.30", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fc90e94bbcb54fb6b86a9ac45edb596a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbe4ec483-85", "ovs_interfaceid": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:27:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f457e6ea-8d06-46c6-be2e-f650e28a3b69/disk --force-share --output=json" returned: 0 in 0.152s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f457e6ea-8d06-46c6-be2e-f650e28a3b69/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:01 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Releasing lock "refresh_cache-75bbbead-8628-448b-9484-19bf4ab922ff" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:27:01 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Creating tmpfile /opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff/tmpeijb7l0a to verify with other compute node that the instance is on the same shared storage. {{(pid=72351) check_instance_shared_storage_local /opt/stack/nova/nova/virt/libvirt/driver.py:9622}}
Aug 09 02:27:01 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f457e6ea-8d06-46c6-be2e-f650e28a3b69/disk --force-share --output=json" returned: 0 in 0.193s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:01 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk.rescue --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:01 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:01 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:02 user nova-compute[72351]: DEBUG nova.compute.manager [req-f9ba863c-2271-404c-9ec4-75f0fa796a02 req-390640ff-dcb8-45a9-851a-596b8bc10164 service nova] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Received event network-vif-unplugged-be4ec483-8532-45f7-bb9f-0b50952ce94b {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:27:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-f9ba863c-2271-404c-9ec4-75f0fa796a02 req-390640ff-dcb8-45a9-851a-596b8bc10164 service nova] Acquiring lock "75bbbead-8628-448b-9484-19bf4ab922ff-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:27:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-f9ba863c-2271-404c-9ec4-75f0fa796a02 req-390640ff-dcb8-45a9-851a-596b8bc10164 service nova] Lock "75bbbead-8628-448b-9484-19bf4ab922ff-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:27:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-f9ba863c-2271-404c-9ec4-75f0fa796a02 req-390640ff-dcb8-45a9-851a-596b8bc10164 service nova] Lock "75bbbead-8628-448b-9484-19bf4ab922ff-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:27:02 user nova-compute[72351]: DEBUG nova.compute.manager [req-f9ba863c-2271-404c-9ec4-75f0fa796a02 req-390640ff-dcb8-45a9-851a-596b8bc10164 service nova] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] No waiting events found dispatching network-vif-unplugged-be4ec483-8532-45f7-bb9f-0b50952ce94b {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:27:02 user nova-compute[72351]: WARNING nova.compute.manager [req-f9ba863c-2271-404c-9ec4-75f0fa796a02 req-390640ff-dcb8-45a9-851a-596b8bc10164 service nova] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Received unexpected event network-vif-unplugged-be4ec483-8532-45f7-bb9f-0b50952ce94b for instance with vm_state resized and task_state resize_reverting.
Aug 09 02:27:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk.rescue --force-share --output=json" returned: 0 in 0.179s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk.rescue --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk.rescue --force-share --output=json" returned: 0 in 0.142s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:02 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json" returned: 0 in 0.140s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json" returned: 0 in 0.162s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/16c1e558-baa1-4c2d-b13b-50585f74d587/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:02 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Instance destroyed successfully.
Aug 09 02:27:02 user nova-compute[72351]: DEBUG nova.objects.instance [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Lazy-loading 'resources' on Instance uuid 75bbbead-8628-448b-9484-19bf4ab922ff {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:27:02 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:21:09Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestOtherB-server-634605118',display_name='tempest-ServerActionsTestOtherB-server-634605118',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(7),hidden=False,host='user',hostname='tempest-serveractionstestotherb-server-634605118',id=15,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=7,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLdtpWQKmHav2tGxZemUVMEaT9TXyD7Duhy5QV84PPI+5KrgLVkbxVzWgryocScOebEprbcJ80bjBPnYhLtQJYSe5MXXz6GOQHLb+tJCZajhrDoMbn41/bPAnB5B3oFwCw==',key_name='tempest-keypair-644333596',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:25:31Z,launched_on='user',locked=False,locked_by=None,memory_mb=2048,metadata={},migration_context=<?>,new_flavor=Flavor(7),node='user',numa_topology=None,old_flavor=Flavor(6),os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='fc90e94bbcb54fb6b86a9ac45edb596a',ramdisk_id='',reservation_id='r-ozz8gdfj',resources=None,root_device_name='/dev/vda',root_gb=20,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',old_vm_state='active',owner_project_name='tempest-ServerActionsTestOtherB-1726991791',owner_user_name='tempest-ServerActionsTestOtherB-1726991791-project-member'},tags=<?>,task_state='resize_reverting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:25:31Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='19668b0c790d49dd830c39e3c2be29bf',uuid=75bbbead-8628-448b-9484-19bf4ab922ff,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='resized') vif={"id": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "address": "fa:16:3e:8b:b5:2e", "network": {"id": "8231f2c2-237e-41c6-8892-720b950c1453", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherB-1917380761-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.30", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fc90e94bbcb54fb6b86a9ac45edb596a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbe4ec483-85", "ovs_interfaceid": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:27:02 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Converting VIF {"id": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "address": "fa:16:3e:8b:b5:2e", "network": {"id": "8231f2c2-237e-41c6-8892-720b950c1453", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherB-1917380761-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.30", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fc90e94bbcb54fb6b86a9ac45edb596a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbe4ec483-85", "ovs_interfaceid": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:27:02 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:8b:b5:2e,bridge_name='br-int',has_traffic_filtering=True,id=be4ec483-8532-45f7-bb9f-0b50952ce94b,network=Network(8231f2c2-237e-41c6-8892-720b950c1453),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapbe4ec483-85') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:27:02 user nova-compute[72351]: DEBUG os_vif [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:8b:b5:2e,bridge_name='br-int',has_traffic_filtering=True,id=be4ec483-8532-45f7-bb9f-0b50952ce94b,network=Network(8231f2c2-237e-41c6-8892-720b950c1453),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapbe4ec483-85') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:27:02 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:02 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapbe4ec483-85, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:27:02 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:27:02 user nova-compute[72351]: INFO os_vif [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:8b:b5:2e,bridge_name='br-int',has_traffic_filtering=True,id=be4ec483-8532-45f7-bb9f-0b50952ce94b,network=Network(8231f2c2-237e-41c6-8892-720b950c1453),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapbe4ec483-85')
Aug 09 02:27:02 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 02:27:02 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'oTnweUhv', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc'}, None), 'kwargs': True}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:27:02 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:27:02 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:27:02 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:27:02 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 02:27:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:02 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[34acdfeb-bf79-4248-8f7a-762bb914bc37]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/16c1e558-baa1-4c2d-b13b-50585f74d587/disk --force-share --output=json" returned: 0 in 0.155s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/16c1e558-baa1-4c2d-b13b-50585f74d587/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:02 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[b7673bb4-20a8-4ec9-8ef1-371340f99bb9]: (4, ('tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)\ntcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)\ntcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)\ntcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:02 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:27:02 user nova-compute[72351]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)
Aug 09 02:27:02 user nova-compute[72351]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)
Aug 09 02:27:02 user nova-compute[72351]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:27:02 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:27:02 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] iscsi session list stdout=tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:27:02 user nova-compute[72351]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)
Aug 09 02:27:02 user nova-compute[72351]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)
Aug 09 02:27:02 user nova-compute[72351]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:27:02 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:27:02 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fd00b3d43a0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd'): ({'sdc'}, set())}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 02:27:02 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Removing single pathed devices sdc {{(pid=72351) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Aug 09 02:27:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:02 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[dbbe47d8-6b87-4452-8fc7-1fbd9a9ad26c]: (4, ('path checker states:\nup                  4\n\npaths: 0\nbusy: False\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdc {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdc" returned: 0 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:02 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[883d2305-2907-4d27-8251-70ecfa216b03]: (4, ('ok\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:02 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Flushing IO for device /dev/sdc {{(pid=72351) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Aug 09 02:27:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdc {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdc" returned: 0 in 0.014s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:02 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[92d83cd4-ba1a-4e54-89e3-a21014c40485]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:02 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Remove SCSI device /dev/sdc with /sys/block/sdc/device/delete {{(pid=72351) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Aug 09 02:27:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdc/device/delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/16c1e558-baa1-4c2d-b13b-50585f74d587/disk --force-share --output=json" returned: 0 in 0.145s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdc/device/delete" returned: 0 in 0.041s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:02 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[3d24e004-6785-49f1-9a41-6923f372e5ca]: (4, ('1', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:02 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Checking to see if SCSI volumes sdc have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Aug 09 02:27:02 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] SCSI volumes sdc have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Aug 09 02:27:02 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd')] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 02:27:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:02 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[682f95c7-ace7-4bbe-94ff-f52cea9f8ff5]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:02 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:27:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 --logout {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:02 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json" returned: 0 in 0.140s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 --logout" returned: 0 in 0.084s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:03 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[6d5f5922-895c-4a10-87a2-def8b818bf2a]: (4, ('Logging out of session [sid: 21, target: iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd, portal: 172.16.0.220,3260]\nLogout of [sid: 21, target: iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:03 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 21, target: iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd, portal: 172.16.0.220,3260]
Aug 09 02:27:03 user nova-compute[72351]: Logout of [sid: 21, target: iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd, portal: 172.16.0.220,3260] successful.
Aug 09 02:27:03 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:27:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 --op delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 --op delete" returned: 0 in 0.025s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:03 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[1d9103ab-6497-470c-8385-3a00d816e285]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:03 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:27:03 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.429s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:27:03 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] <== disconnect_volume: return (431ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:27:03 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 02:27:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json" returned: 0 in 0.158s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:03 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Error from libvirt while getting description of instance-0000000f: [Error Code 42] Domain not found: no domain with matching uuid '75bbbead-8628-448b-9484-19bf4ab922ff' (instance-0000000f): libvirt.libvirtError: Domain not found: no domain with matching uuid '75bbbead-8628-448b-9484-19bf4ab922ff' (instance-0000000f)
Aug 09 02:27:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/831c864a-75d9-42b3-b952-749f4201e40e/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/831c864a-75d9-42b3-b952-749f4201e40e/disk --force-share --output=json" returned: 0 in 0.130s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/831c864a-75d9-42b3-b952-749f4201e40e/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/831c864a-75d9-42b3-b952-749f4201e40e/disk --force-share --output=json" returned: 0 in 0.156s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:03 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sde (vdb) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:27:04 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:27:04 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:27:04 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Hypervisor/Node resource view: name=user free_ram=6977MB free_disk=25.336444854736328GB free_vcpus=2 pci_devices=[{"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72351) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Aug 09 02:27:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:27:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:27:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.drop_move_claim_at_dest" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:27:04 user nova-compute[72351]: DEBUG nova.compute.manager [req-99952975-e759-4fc1-bae6-5883f469590e req-1920753a-0c2a-42f6-ba76-4b19dd868a72 service nova] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Received event network-vif-plugged-be4ec483-8532-45f7-bb9f-0b50952ce94b {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:27:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-99952975-e759-4fc1-bae6-5883f469590e req-1920753a-0c2a-42f6-ba76-4b19dd868a72 service nova] Acquiring lock "75bbbead-8628-448b-9484-19bf4ab922ff-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:27:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-99952975-e759-4fc1-bae6-5883f469590e req-1920753a-0c2a-42f6-ba76-4b19dd868a72 service nova] Lock "75bbbead-8628-448b-9484-19bf4ab922ff-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:27:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-99952975-e759-4fc1-bae6-5883f469590e req-1920753a-0c2a-42f6-ba76-4b19dd868a72 service nova] Lock "75bbbead-8628-448b-9484-19bf4ab922ff-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:27:04 user nova-compute[72351]: DEBUG nova.compute.manager [req-99952975-e759-4fc1-bae6-5883f469590e req-1920753a-0c2a-42f6-ba76-4b19dd868a72 service nova] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] No waiting events found dispatching network-vif-plugged-be4ec483-8532-45f7-bb9f-0b50952ce94b {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:27:04 user nova-compute[72351]: WARNING nova.compute.manager [req-99952975-e759-4fc1-bae6-5883f469590e req-1920753a-0c2a-42f6-ba76-4b19dd868a72 service nova] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Received unexpected event network-vif-plugged-be4ec483-8532-45f7-bb9f-0b50952ce94b for instance with vm_state resized and task_state resize_reverting.
Aug 09 02:27:04 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Applying migration context for instance 75bbbead-8628-448b-9484-19bf4ab922ff as it has an incoming, in-progress migration 161f097f-81e6-4201-8af0-05f72c1ba9e0. Migration status is reverting {{(pid=72351) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1016}}
Aug 09 02:27:04 user nova-compute[72351]: INFO nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Updating resource usage from migration 161f097f-81e6-4201-8af0-05f72c1ba9e0
Aug 09 02:27:04 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:27:04 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance eb86cb49-2018-42c2-bd97-c662b829775d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:27:04 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:27:04 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance a7f3d076-864b-4b68-9207-214ce2c21444 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:27:04 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 831c864a-75d9-42b3-b952-749f4201e40e actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:27:04 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance f457e6ea-8d06-46c6-be2e-f650e28a3b69 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:27:04 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Migration 161f097f-81e6-4201-8af0-05f72c1ba9e0 is active on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1709}}
Aug 09 02:27:04 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 75bbbead-8628-448b-9484-19bf4ab922ff actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 20, 'MEMORY_MB': 2048, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:27:04 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 47afce95-e5f7-4703-a99f-7ef8b28c4bd2 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:27:04 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 16c1e558-baa1-4c2d-b13b-50585f74d587 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:27:04 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 957c1510-1320-4492-9052-a58ff540319d actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:27:04 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Total usable vcpus: 12, total allocated vcpus: 11 {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Aug 09 02:27:04 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Final resource view: name=user phys_ram=16011MB used_ram=7680MB phys_disk=40GB used_disk=29GB total_vcpus=12 used_vcpus=11 pci_stats=[] {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Aug 09 02:27:04 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691548009.2776005, 1d772c10-1c8a-4101-9a61-f8a60bb19c66 => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:27:04 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] VM Stopped (Lifecycle Event)
Aug 09 02:27:04 user nova-compute[72351]: DEBUG nova.compute.manager [None req-39fdd25c-5b71-4c47-9c47-24a6b9f83da6 None None] [instance: 1d772c10-1c8a-4101-9a61-f8a60bb19c66] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:27:04 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:27:04 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:27:04 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Compute_service record updated for user:user {{(pid=72351) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Aug 09 02:27:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.500s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:27:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.drop_move_claim_at_dest" :: waited 0.470s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:27:04 user nova-compute[72351]: DEBUG nova.objects.instance [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Lazy-loading 'migration_context' on Instance uuid 75bbbead-8628-448b-9484-19bf4ab922ff {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:27:04 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:27:04 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:27:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.drop_move_claim_at_dest" :: held 0.438s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:27:05 user nova-compute[72351]: INFO nova.compute.manager [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Swapping old allocation on dict_keys(['8eae9b37-1236-4e6d-91a5-b650de27b009']) held by migration 161f097f-81e6-4201-8af0-05f72c1ba9e0 for instance
Aug 09 02:27:05 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Overwriting current allocation {'allocations': {'8eae9b37-1236-4e6d-91a5-b650de27b009': {'resources': {'VCPU': 1, 'MEMORY_MB': 2048, 'DISK_GB': 20}, 'generation': 32}}, 'project_id': 'fc90e94bbcb54fb6b86a9ac45edb596a', 'user_id': '19668b0c790d49dd830c39e3c2be29bf', 'consumer_generation': 1} on consumer 75bbbead-8628-448b-9484-19bf4ab922ff {{(pid=72351) move_allocations /opt/stack/nova/nova/scheduler/client/report.py:2018}}
Aug 09 02:27:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Acquiring lock "refresh_cache-75bbbead-8628-448b-9484-19bf4ab922ff" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:27:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Acquired lock "refresh_cache-75bbbead-8628-448b-9484-19bf4ab922ff" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:27:05 user nova-compute[72351]: DEBUG nova.network.neutron [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:27:05 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:27:05 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:27:05 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:27:05 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Starting heal instance info cache {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9842}}
Aug 09 02:27:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "refresh_cache-585c81a5-ac24-4d12-a8a6-1e7b12a2d53a" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:27:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquired lock "refresh_cache-585c81a5-ac24-4d12-a8a6-1e7b12a2d53a" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:27:05 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Forcefully refreshing network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1996}}
Aug 09 02:27:05 user nova-compute[72351]: DEBUG nova.network.neutron [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Updating instance_info_cache with network_info: [{"id": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "address": "fa:16:3e:8b:b5:2e", "network": {"id": "8231f2c2-237e-41c6-8892-720b950c1453", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherB-1917380761-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.30", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fc90e94bbcb54fb6b86a9ac45edb596a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbe4ec483-85", "ovs_interfaceid": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:27:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Releasing lock "refresh_cache-75bbbead-8628-448b-9484-19bf4ab922ff" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:27:05 user nova-compute[72351]: DEBUG os_brick.utils [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:27:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:05 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[0a773f60-12a1-48c2-8758-7853ee93374f]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:a52269602ff4\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.015s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:06 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[14a93e31-c68e-4bed-829c-46438b72e0a2]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.015s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:06 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[f3ac8e1b-b4a7-450c-8230-76ea96635721]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:06 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[c04991df-9d5a-45bd-a71c-e4276dba28eb]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Running cmd (subprocess): nvme version {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] 'nvme version' failed. Not Retrying. {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:27:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.nvmeof [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] nvme not present on system {{(pid=72351) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Aug 09 02:27:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:27:06 user nova-compute[72351]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Aug 09 02:27:06 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[588efd64-8b47-4aaa-a687-fa2727b1733a]: (4, '') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72351) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Aug 09 02:27:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Aug 09 02:27:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] LIGHTOS: no hostnqn found. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Aug 09 02:27:06 user nova-compute[72351]: DEBUG os_brick.utils [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] <== get_connector_properties: return (125ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:a52269602ff4', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:27:06 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Updating instance_info_cache with network_info: [{"id": "e521fae4-093d-4c92-a32d-684c1a5137f5", "address": "fa:16:3e:e5:b0:10", "network": {"id": "b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-210689817-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.41", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "69d584c230b94ff8aae722eb2f364c5c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape521fae4-09", "ovs_interfaceid": "e521fae4-093d-4c92-a32d-684c1a5137f5", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:27:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Releasing lock "refresh_cache-585c81a5-ac24-4d12-a8a6-1e7b12a2d53a" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:27:06 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Updated the network info_cache for instance {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9913}}
Aug 09 02:27:06 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:27:06 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:27:06 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:27:06 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:27:07 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:07 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:08 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-1fa2042e-4a2d-4778-bb20-240b20e19cbb tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 02:27:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1fa2042e-4a2d-4778-bb20-240b20e19cbb tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:11ec2ffe-3db9-4d44-8f09-316c97ad7b80'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'eEVXgmdc', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:27:08 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-1fa2042e-4a2d-4778-bb20-240b20e19cbb tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:27:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-1fa2042e-4a2d-4778-bb20-240b20e19cbb tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:27:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-1fa2042e-4a2d-4778-bb20-240b20e19cbb tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:27:08 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-1fa2042e-4a2d-4778-bb20-240b20e19cbb tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:27:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-1fa2042e-4a2d-4778-bb20-240b20e19cbb tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:11ec2ffe-3db9-4d44-8f09-316c97ad7b80" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:27:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-1fa2042e-4a2d-4778-bb20-240b20e19cbb tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:11ec2ffe-3db9-4d44-8f09-316c97ad7b80" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:27:08 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-1fa2042e-4a2d-4778-bb20-240b20e19cbb tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 02:27:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:11ec2ffe-3db9-4d44-8f09-316c97ad7b80 -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:11ec2ffe-3db9-4d44-8f09-316c97ad7b80 -p 172.16.0.220:3260" returned: 21 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:08 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[8407dd6d-0ab3-4fc5-8e57-f1d65d0de23c]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1fa2042e-4a2d-4778-bb20-240b20e19cbb tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Aug 09 02:27:08 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:27:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:11ec2ffe-3db9-4d44-8f09-316c97ad7b80 -p 172.16.0.220:3260 --interface default --op new {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:11ec2ffe-3db9-4d44-8f09-316c97ad7b80 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:08 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[f35aedd2-6b92-46f7-92da-73854e65207d]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:11ec2ffe-3db9-4d44-8f09-316c97ad7b80] added\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1fa2042e-4a2d-4778-bb20-240b20e19cbb tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:11ec2ffe-3db9-4d44-8f09-316c97ad7b80] added
Aug 09 02:27:08 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:27:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:11ec2ffe-3db9-4d44-8f09-316c97ad7b80 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:11ec2ffe-3db9-4d44-8f09-316c97ad7b80 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:08 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[cb1b0a20-37ca-44ee-8ad4-48a7e478a222]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1fa2042e-4a2d-4778-bb20-240b20e19cbb tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:27:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:11ec2ffe-3db9-4d44-8f09-316c97ad7b80 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:11ec2ffe-3db9-4d44-8f09-316c97ad7b80 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:08 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d6dfb708-aec1-42e4-b3e3-30a708d8d671]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1fa2042e-4a2d-4778-bb20-240b20e19cbb tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:27:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:11ec2ffe-3db9-4d44-8f09-316c97ad7b80 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v eEVXgmdc {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:11ec2ffe-3db9-4d44-8f09-316c97ad7b80 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v eEVXgmdc" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:08 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[77473cb9-4129-462d-b813-d9e4981c8663]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1fa2042e-4a2d-4778-bb20-240b20e19cbb tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'eEVXgmdc'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:27:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:11ec2ffe-3db9-4d44-8f09-316c97ad7b80 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:11ec2ffe-3db9-4d44-8f09-316c97ad7b80 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:08 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[c48e6afa-f52e-4cb5-b03b-c408e1581509]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1fa2042e-4a2d-4778-bb20-240b20e19cbb tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:27:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.026s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:08 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[1df6fc50-6f08-40f2-9289-c96a92f6168c]: (4, ('tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)\ntcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)\ntcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1fa2042e-4a2d-4778-bb20-240b20e19cbb tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)
Aug 09 02:27:08 user nova-compute[72351]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)
Aug 09 02:27:08 user nova-compute[72351]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:27:08 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:27:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1fa2042e-4a2d-4778-bb20-240b20e19cbb tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] iscsi session list stdout=tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)
Aug 09 02:27:08 user nova-compute[72351]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)
Aug 09 02:27:08 user nova-compute[72351]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:27:08 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:27:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:11ec2ffe-3db9-4d44-8f09-316c97ad7b80 -p 172.16.0.220:3260 --login {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:11ec2ffe-3db9-4d44-8f09-316c97ad7b80 -p 172.16.0.220:3260 --login" returned: 0 in 0.095s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:08 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[098262a6-7ad2-4b78-a93e-a2a3e087f7bf]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:11ec2ffe-3db9-4d44-8f09-316c97ad7b80, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:11ec2ffe-3db9-4d44-8f09-316c97ad7b80, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1fa2042e-4a2d-4778-bb20-240b20e19cbb tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:11ec2ffe-3db9-4d44-8f09-316c97ad7b80, portal: 172.16.0.220,3260]
Aug 09 02:27:08 user nova-compute[72351]: Login to [iface: default, target: iqn.2010-10.org.openstack:11ec2ffe-3db9-4d44-8f09-316c97ad7b80, portal: 172.16.0.220,3260] successful.
Aug 09 02:27:08 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:27:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:11ec2ffe-3db9-4d44-8f09-316c97ad7b80 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:11ec2ffe-3db9-4d44-8f09-316c97ad7b80 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:08 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[09e29957-8a51-4df7-911a-cf41f56afa9b]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1fa2042e-4a2d-4778-bb20-240b20e19cbb tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:27:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:08 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[02a2b4e9-0914-4389-aea1-ac7cef664b38]: (4, ('tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)\ntcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)\ntcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:11ec2ffe-3db9-4d44-8f09-316c97ad7b80 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1fa2042e-4a2d-4778-bb20-240b20e19cbb tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)
Aug 09 02:27:08 user nova-compute[72351]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)
Aug 09 02:27:08 user nova-compute[72351]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:27:08 user nova-compute[72351]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:11ec2ffe-3db9-4d44-8f09-316c97ad7b80 (non-flash)
Aug 09 02:27:08 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:27:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1fa2042e-4a2d-4778-bb20-240b20e19cbb tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] iscsi session list stdout=tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)
Aug 09 02:27:08 user nova-compute[72351]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)
Aug 09 02:27:08 user nova-compute[72351]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:27:08 user nova-compute[72351]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:11ec2ffe-3db9-4d44-8f09-316c97ad7b80 (non-flash)
Aug 09 02:27:08 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:27:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-1fa2042e-4a2d-4778-bb20-240b20e19cbb tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:11ec2ffe-3db9-4d44-8f09-316c97ad7b80" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.310s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:27:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1fa2042e-4a2d-4778-bb20-240b20e19cbb tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 02:27:08 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-1fa2042e-4a2d-4778-bb20-240b20e19cbb tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] HCTL ('33', '-', '-', 0) found on session 27 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 02:27:08 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-1fa2042e-4a2d-4778-bb20-240b20e19cbb tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 02:27:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.016s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:08 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[7050d0a5-e822-4cdc-8440-cc592e4e4898]: (4, ('- - 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:08 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-1fa2042e-4a2d-4778-bb20-240b20e19cbb tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Searching for a device in session 27 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:27:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1fa2042e-4a2d-4778-bb20-240b20e19cbb tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Connected to sdb using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'eEVXgmdc', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 02:27:09 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-1fa2042e-4a2d-4778-bb20-240b20e19cbb tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.339s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:27:09 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1fa2042e-4a2d-4778-bb20-240b20e19cbb tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] <== connect_volume: return (1341ms) {'type': 'block', 'scsi_wwn': '26630336635653562', 'path': '/dev/sdb'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:27:09 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-1fa2042e-4a2d-4778-bb20-240b20e19cbb tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26630336635653562', 'path': '/dev/sdb'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 02:27:09 user nova-compute[72351]: DEBUG nova.objects.instance [None req-1fa2042e-4a2d-4778-bb20-240b20e19cbb tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Lazy-loading 'flavor' on Instance uuid f457e6ea-8d06-46c6-be2e-f650e28a3b69 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:27:09 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-1fa2042e-4a2d-4778-bb20-240b20e19cbb tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] attach device xml: <disk type="block" device="disk">
Aug 09 02:27:09 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:27:09 user nova-compute[72351]:   <source dev="/dev/sdb"/>
Aug 09 02:27:09 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:27:09 user nova-compute[72351]:   <serial>11ec2ffe-3db9-4d44-8f09-316c97ad7b80</serial>
Aug 09 02:27:09 user nova-compute[72351]: </disk>
Aug 09 02:27:09 user nova-compute[72351]:  {{(pid=72351) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Aug 09 02:27:10 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-1fa2042e-4a2d-4778-bb20-240b20e19cbb tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:27:10 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-1fa2042e-4a2d-4778-bb20-240b20e19cbb tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] No BDM found with device name hda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:27:10 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-1fa2042e-4a2d-4778-bb20-240b20e19cbb tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] No VIF found with MAC fa:16:3e:0e:77:88, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:27:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1fa2042e-4a2d-4778-bb20-240b20e19cbb tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Lock "f457e6ea-8d06-46c6-be2e-f650e28a3b69" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 15.460s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:27:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-217686eb-8008-44b6-a40e-07cae3a7bdb2 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Acquiring lock "f457e6ea-8d06-46c6-be2e-f650e28a3b69" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:27:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-217686eb-8008-44b6-a40e-07cae3a7bdb2 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Lock "f457e6ea-8d06-46c6-be2e-f650e28a3b69" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:27:11 user nova-compute[72351]: INFO nova.compute.manager [None req-217686eb-8008-44b6-a40e-07cae3a7bdb2 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Detaching volume 11ec2ffe-3db9-4d44-8f09-316c97ad7b80
Aug 09 02:27:11 user nova-compute[72351]: INFO nova.virt.block_device [None req-217686eb-8008-44b6-a40e-07cae3a7bdb2 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Attempting to driver detach volume 11ec2ffe-3db9-4d44-8f09-316c97ad7b80 from mountpoint /dev/vdb
Aug 09 02:27:11 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-217686eb-8008-44b6-a40e-07cae3a7bdb2 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Attempting to detach device vdb from instance f457e6ea-8d06-46c6-be2e-f650e28a3b69 from the persistent domain config. {{(pid=72351) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2477}}
Aug 09 02:27:11 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-217686eb-8008-44b6-a40e-07cae3a7bdb2 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 02:27:11 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:27:11 user nova-compute[72351]:   <source dev="/dev/sdb"/>
Aug 09 02:27:11 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:27:11 user nova-compute[72351]:   <serial>11ec2ffe-3db9-4d44-8f09-316c97ad7b80</serial>
Aug 09 02:27:11 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x08" function="0x0"/>
Aug 09 02:27:11 user nova-compute[72351]: </disk>
Aug 09 02:27:11 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 02:27:11 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-217686eb-8008-44b6-a40e-07cae3a7bdb2 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Successfully detached device vdb from instance f457e6ea-8d06-46c6-be2e-f650e28a3b69 from the persistent domain config.
Aug 09 02:27:11 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-217686eb-8008-44b6-a40e-07cae3a7bdb2 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance f457e6ea-8d06-46c6-be2e-f650e28a3b69 from the live domain config. {{(pid=72351) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2513}}
Aug 09 02:27:11 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-217686eb-8008-44b6-a40e-07cae3a7bdb2 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 02:27:11 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:27:11 user nova-compute[72351]:   <source dev="/dev/sdb"/>
Aug 09 02:27:11 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:27:11 user nova-compute[72351]:   <serial>11ec2ffe-3db9-4d44-8f09-316c97ad7b80</serial>
Aug 09 02:27:11 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x08" function="0x0"/>
Aug 09 02:27:11 user nova-compute[72351]: </disk>
Aug 09 02:27:11 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 02:27:12 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691548017.2813244, 433e6729-fdd4-4467-971c-08ca962d887d => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:27:12 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] VM Stopped (Lifecycle Event)
Aug 09 02:27:12 user nova-compute[72351]: DEBUG nova.compute.manager [None req-a4b027c6-529a-42da-8dcf-1837ee1c45bc None None] [instance: 433e6729-fdd4-4467-971c-08ca962d887d] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:27:12 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Received event <DeviceRemovedEvent: 1691548032.3948684, f457e6ea-8d06-46c6-be2e-f650e28a3b69 => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=72351) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2360}}
Aug 09 02:27:12 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-217686eb-8008-44b6-a40e-07cae3a7bdb2 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance f457e6ea-8d06-46c6-be2e-f650e28a3b69 {{(pid=72351) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2589}}
Aug 09 02:27:12 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-217686eb-8008-44b6-a40e-07cae3a7bdb2 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Successfully detached device vdb from instance f457e6ea-8d06-46c6-be2e-f650e28a3b69 from the live domain config.
Aug 09 02:27:12 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-217686eb-8008-44b6-a40e-07cae3a7bdb2 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 02:27:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-217686eb-8008-44b6-a40e-07cae3a7bdb2 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:11ec2ffe-3db9-4d44-8f09-316c97ad7b80'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'eEVXgmdc', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb'}, None), 'kwargs': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:27:12 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-217686eb-8008-44b6-a40e-07cae3a7bdb2 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:27:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-217686eb-8008-44b6-a40e-07cae3a7bdb2 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:27:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-217686eb-8008-44b6-a40e-07cae3a7bdb2 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:27:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-217686eb-8008-44b6-a40e-07cae3a7bdb2 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:11ec2ffe-3db9-4d44-8f09-316c97ad7b80', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 02:27:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:12 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[0a3f8d3c-1b1f-4e7f-8791-6631d01747be]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:11ec2ffe-3db9-4d44-8f09-316c97ad7b80\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.026s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:12 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[db034976-8495-4f3d-97b1-72ac54c66915]: (4, ('tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)\ntcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)\ntcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:11ec2ffe-3db9-4d44-8f09-316c97ad7b80 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-217686eb-8008-44b6-a40e-07cae3a7bdb2 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)
Aug 09 02:27:12 user nova-compute[72351]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)
Aug 09 02:27:12 user nova-compute[72351]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:27:12 user nova-compute[72351]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:11ec2ffe-3db9-4d44-8f09-316c97ad7b80 (non-flash)
Aug 09 02:27:12 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:27:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-217686eb-8008-44b6-a40e-07cae3a7bdb2 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] iscsi session list stdout=tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)
Aug 09 02:27:12 user nova-compute[72351]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)
Aug 09 02:27:12 user nova-compute[72351]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:27:12 user nova-compute[72351]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:11ec2ffe-3db9-4d44-8f09-316c97ad7b80 (non-flash)
Aug 09 02:27:12 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:27:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-217686eb-8008-44b6-a40e-07cae3a7bdb2 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fcfdc422950>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:11ec2ffe-3db9-4d44-8f09-316c97ad7b80'): ({'sdb'}, set())}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 02:27:12 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-217686eb-8008-44b6-a40e-07cae3a7bdb2 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Removing single pathed devices sdb {{(pid=72351) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Aug 09 02:27:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:12 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[12f485f1-c23e-4698-a13f-9886fa7bdaf3]: (4, ('path checker states:\nup                  4\n\npaths: 0\nbusy: False\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:12 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[8e7cbe24-d49b-424c-bceb-bbc74b1270e7]: (4, ('ok\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:12 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-217686eb-8008-44b6-a40e-07cae3a7bdb2 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Flushing IO for device /dev/sdb {{(pid=72351) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Aug 09 02:27:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.013s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:12 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[8fe83a74-8e01-4421-87b7-b19c231e01ba]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:12 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-217686eb-8008-44b6-a40e-07cae3a7bdb2 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=72351) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Aug 09 02:27:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.032s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:12 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[0d70a0a1-86c8-41b9-a69d-3798fa0f95f6]: (4, ('1', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:12 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-217686eb-8008-44b6-a40e-07cae3a7bdb2 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Aug 09 02:27:12 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-217686eb-8008-44b6-a40e-07cae3a7bdb2 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] SCSI volumes sdb have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Aug 09 02:27:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-217686eb-8008-44b6-a40e-07cae3a7bdb2 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:11ec2ffe-3db9-4d44-8f09-316c97ad7b80')] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 02:27:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:11ec2ffe-3db9-4d44-8f09-316c97ad7b80 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:11ec2ffe-3db9-4d44-8f09-316c97ad7b80 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:12 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e972b68c-a17a-4f35-a2e3-dd1e245bcd50]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-217686eb-8008-44b6-a40e-07cae3a7bdb2 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:27:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:11ec2ffe-3db9-4d44-8f09-316c97ad7b80 -p 172.16.0.220:3260 --logout {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:11ec2ffe-3db9-4d44-8f09-316c97ad7b80 -p 172.16.0.220:3260 --logout" returned: 0 in 0.051s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:12 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[c077179c-8eaf-4295-93e1-81902726c15e]: (4, ('Logging out of session [sid: 27, target: iqn.2010-10.org.openstack:11ec2ffe-3db9-4d44-8f09-316c97ad7b80, portal: 172.16.0.220,3260]\nLogout of [sid: 27, target: iqn.2010-10.org.openstack:11ec2ffe-3db9-4d44-8f09-316c97ad7b80, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-217686eb-8008-44b6-a40e-07cae3a7bdb2 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 27, target: iqn.2010-10.org.openstack:11ec2ffe-3db9-4d44-8f09-316c97ad7b80, portal: 172.16.0.220,3260]
Aug 09 02:27:12 user nova-compute[72351]: Logout of [sid: 27, target: iqn.2010-10.org.openstack:11ec2ffe-3db9-4d44-8f09-316c97ad7b80, portal: 172.16.0.220,3260] successful.
Aug 09 02:27:12 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:27:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:11ec2ffe-3db9-4d44-8f09-316c97ad7b80 -p 172.16.0.220:3260 --op delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:12 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:11ec2ffe-3db9-4d44-8f09-316c97ad7b80 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.033s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:12 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[6d6c8044-8afe-4c21-ad2c-156831932d51]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-217686eb-8008-44b6-a40e-07cae3a7bdb2 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:27:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-217686eb-8008-44b6-a40e-07cae3a7bdb2 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.268s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:27:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-217686eb-8008-44b6-a40e-07cae3a7bdb2 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] <== disconnect_volume: return (270ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:27:12 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-217686eb-8008-44b6-a40e-07cae3a7bdb2 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 02:27:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:13 user nova-compute[72351]: DEBUG nova.objects.instance [None req-217686eb-8008-44b6-a40e-07cae3a7bdb2 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Lazy-loading 'flavor' on Instance uuid f457e6ea-8d06-46c6-be2e-f650e28a3b69 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:27:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-217686eb-8008-44b6-a40e-07cae3a7bdb2 tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Lock "f457e6ea-8d06-46c6-be2e-f650e28a3b69" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.738s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:27:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-573693b1-fef0-4819-9ee7-6be13c801b1c tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Acquiring lock "interface-f457e6ea-8d06-46c6-be2e-f650e28a3b69-9bf76200-4b93-40dc-bd6f-0200cec7933a" by "nova.compute.manager.ComputeManager.detach_interface.<locals>.do_detach_interface" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:27:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-573693b1-fef0-4819-9ee7-6be13c801b1c tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Lock "interface-f457e6ea-8d06-46c6-be2e-f650e28a3b69-9bf76200-4b93-40dc-bd6f-0200cec7933a" acquired by "nova.compute.manager.ComputeManager.detach_interface.<locals>.do_detach_interface" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:27:14 user nova-compute[72351]: DEBUG nova.objects.instance [None req-573693b1-fef0-4819-9ee7-6be13c801b1c tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Lazy-loading 'flavor' on Instance uuid f457e6ea-8d06-46c6-be2e-f650e28a3b69 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:27:14 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-573693b1-fef0-4819-9ee7-6be13c801b1c tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-08-09T02:24:58Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=InstanceDeviceMetadata,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-443615500',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-443615500',id=25,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBK8nrHk7zghGHoCUEI3vExNw2oXJc2YwDe7QrMxgC2DNUP/PkVzpLwy9DWnyTLCCR4ddGxe1t7xIbXts5zbiafpc3PmU6by4kyzLopbWZ1BYcJyQcmguVSz4dz8HMbCR9g==',key_name='tempest-keypair-1705746290',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:25:05Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=<?>,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='bde0634670914ae8bfbbc6b31c09d6dd',ramdisk_id='',reservation_id='r-5xxup5ze',resources=<?>,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedAttachmentsTest-480515778',owner_user_name='tempest-TaggedAttachmentsTest-480515778-project-member'},tags=<?>,task_state=None,terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:25:06Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='8159c9e78af04e01a1faf9b3cfc3591e',uuid=f457e6ea-8d06-46c6-be2e-f650e28a3b69,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "9bf76200-4b93-40dc-bd6f-0200cec7933a", "address": "fa:16:3e:30:6e:79", "network": {"id": "ea588e91-dd01-4f9c-ad88-30f0848e6839", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-596948871", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.51", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "bde0634670914ae8bfbbc6b31c09d6dd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9bf76200-4b", "ovs_interfaceid": "9bf76200-4b93-40dc-bd6f-0200cec7933a", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:27:14 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-573693b1-fef0-4819-9ee7-6be13c801b1c tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Converting VIF {"id": "9bf76200-4b93-40dc-bd6f-0200cec7933a", "address": "fa:16:3e:30:6e:79", "network": {"id": "ea588e91-dd01-4f9c-ad88-30f0848e6839", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-596948871", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.51", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "bde0634670914ae8bfbbc6b31c09d6dd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9bf76200-4b", "ovs_interfaceid": "9bf76200-4b93-40dc-bd6f-0200cec7933a", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:27:14 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-573693b1-fef0-4819-9ee7-6be13c801b1c tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:30:6e:79,bridge_name='br-int',has_traffic_filtering=True,id=9bf76200-4b93-40dc-bd6f-0200cec7933a,network=Network(ea588e91-dd01-4f9c-ad88-30f0848e6839),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap9bf76200-4b') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:27:14 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-573693b1-fef0-4819-9ee7-6be13c801b1c tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] looking for interface given config: <interface type="ethernet"><mac address="fa:16:3e:30:6e:79"/><model type="virtio"/><mtu size="1442"/><target dev="tap9bf76200-4b"/></interface> {{(pid=72351) get_interface_by_cfg /opt/stack/nova/nova/virt/libvirt/guest.py:257}}
Aug 09 02:27:14 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-573693b1-fef0-4819-9ee7-6be13c801b1c tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] looking for interface given config: <interface type="ethernet"><mac address="fa:16:3e:30:6e:79"/><model type="virtio"/><mtu size="1442"/><target dev="tap9bf76200-4b"/></interface> {{(pid=72351) get_interface_by_cfg /opt/stack/nova/nova/virt/libvirt/guest.py:257}}
Aug 09 02:27:14 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-573693b1-fef0-4819-9ee7-6be13c801b1c tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Attempting to detach device tap9bf76200-4b from instance f457e6ea-8d06-46c6-be2e-f650e28a3b69 from the persistent domain config. {{(pid=72351) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2477}}
Aug 09 02:27:14 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-573693b1-fef0-4819-9ee7-6be13c801b1c tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] detach device xml: <interface type="ethernet">
Aug 09 02:27:14 user nova-compute[72351]:   <mac address="fa:16:3e:30:6e:79"/>
Aug 09 02:27:14 user nova-compute[72351]:   <model type="virtio"/>
Aug 09 02:27:14 user nova-compute[72351]:   <mtu size="1442"/>
Aug 09 02:27:14 user nova-compute[72351]:   <target dev="tap9bf76200-4b"/>
Aug 09 02:27:14 user nova-compute[72351]: </interface>
Aug 09 02:27:14 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 02:27:14 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-573693b1-fef0-4819-9ee7-6be13c801b1c tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] looking for interface given config: <interface type="ethernet"><mac address="fa:16:3e:30:6e:79"/><model type="virtio"/><mtu size="1442"/><target dev="tap9bf76200-4b"/></interface> {{(pid=72351) get_interface_by_cfg /opt/stack/nova/nova/virt/libvirt/guest.py:257}}
Aug 09 02:27:14 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-573693b1-fef0-4819-9ee7-6be13c801b1c tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] interface for config: <interface type="ethernet"><mac address="fa:16:3e:30:6e:79"/><model type="virtio"/><mtu size="1442"/><target dev="tap9bf76200-4b"/></interface>not found in domain: <domain type='kvm' id='29'>
Aug 09 02:27:14 user nova-compute[72351]:   <name>instance-00000019</name>
Aug 09 02:27:14 user nova-compute[72351]:   <uuid>f457e6ea-8d06-46c6-be2e-f650e28a3b69</uuid>
Aug 09 02:27:14 user nova-compute[72351]:   <metadata>
Aug 09 02:27:14 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1" xmlns:instance="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:27:14 user nova-compute[72351]:   <nova:package version="0.0.0"/>
Aug 09 02:27:14 user nova-compute[72351]:   <nova:name>tempest-device-tagging-server-443615500</nova:name>
Aug 09 02:27:14 user nova-compute[72351]:   <nova:creationTime>2023-08-09 02:26:54</nova:creationTime>
Aug 09 02:27:14 user nova-compute[72351]:   <nova:flavor name="m1.tiny">
Aug 09 02:27:14 user nova-compute[72351]:     <nova:memory>512</nova:memory>
Aug 09 02:27:14 user nova-compute[72351]:     <nova:disk>1</nova:disk>
Aug 09 02:27:14 user nova-compute[72351]:     <nova:swap>0</nova:swap>
Aug 09 02:27:14 user nova-compute[72351]:     <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:27:14 user nova-compute[72351]:     <nova:vcpus>1</nova:vcpus>
Aug 09 02:27:14 user nova-compute[72351]:   </nova:flavor>
Aug 09 02:27:14 user nova-compute[72351]:   <nova:owner>
Aug 09 02:27:14 user nova-compute[72351]:     <nova:user uuid="8159c9e78af04e01a1faf9b3cfc3591e">tempest-TaggedAttachmentsTest-480515778-project-member</nova:user>
Aug 09 02:27:14 user nova-compute[72351]:     <nova:project uuid="bde0634670914ae8bfbbc6b31c09d6dd">tempest-TaggedAttachmentsTest-480515778</nova:project>
Aug 09 02:27:14 user nova-compute[72351]:   </nova:owner>
Aug 09 02:27:14 user nova-compute[72351]:   <nova:root type="image" uuid="18ccda7f-31d3-4978-ab9d-4ed2f74eff94"/>
Aug 09 02:27:14 user nova-compute[72351]:   <nova:ports>
Aug 09 02:27:14 user nova-compute[72351]:     <nova:port uuid="dc244cf9-b0f6-4906-b9f3-53634b1847e3">
Aug 09 02:27:14 user nova-compute[72351]:       <nova:ip type="fixed" address="10.0.0.9" ipVersion="4"/>
Aug 09 02:27:14 user nova-compute[72351]:     </nova:port>
Aug 09 02:27:14 user nova-compute[72351]:     <nova:port uuid="9bf76200-4b93-40dc-bd6f-0200cec7933a">
Aug 09 02:27:14 user nova-compute[72351]:       <nova:ip type="fixed" address="10.10.10.51" ipVersion="4"/>
Aug 09 02:27:14 user nova-compute[72351]:     </nova:port>
Aug 09 02:27:14 user nova-compute[72351]:   </nova:ports>
Aug 09 02:27:14 user nova-compute[72351]: </nova:instance>
Aug 09 02:27:14 user nova-compute[72351]:   </metadata>
Aug 09 02:27:14 user nova-compute[72351]:   <memory unit='KiB'>524288</memory>
Aug 09 02:27:14 user nova-compute[72351]:   <currentMemory unit='KiB'>524288</currentMemory>
Aug 09 02:27:14 user nova-compute[72351]:   <vcpu placement='static'>1</vcpu>
Aug 09 02:27:14 user nova-compute[72351]:   <resource>
Aug 09 02:27:14 user nova-compute[72351]:     <partition>/machine</partition>
Aug 09 02:27:14 user nova-compute[72351]:   </resource>
Aug 09 02:27:14 user nova-compute[72351]:   <sysinfo type='smbios'>
Aug 09 02:27:14 user nova-compute[72351]:     <system>
Aug 09 02:27:14 user nova-compute[72351]:       <entry name='manufacturer'>OpenStack Foundation</entry>
Aug 09 02:27:14 user nova-compute[72351]:       <entry name='product'>OpenStack Nova</entry>
Aug 09 02:27:14 user nova-compute[72351]:       <entry name='version'>0.0.0</entry>
Aug 09 02:27:14 user nova-compute[72351]:       <entry name='serial'>f457e6ea-8d06-46c6-be2e-f650e28a3b69</entry>
Aug 09 02:27:14 user nova-compute[72351]:       <entry name='uuid'>f457e6ea-8d06-46c6-be2e-f650e28a3b69</entry>
Aug 09 02:27:14 user nova-compute[72351]:       <entry name='family'>Virtual Machine</entry>
Aug 09 02:27:14 user nova-compute[72351]:     </system>
Aug 09 02:27:14 user nova-compute[72351]:   </sysinfo>
Aug 09 02:27:14 user nova-compute[72351]:   <os>
Aug 09 02:27:14 user nova-compute[72351]:     <type arch='x86_64' machine='pc-i440fx-6.2'>hvm</type>
Aug 09 02:27:14 user nova-compute[72351]:     <boot dev='hd'/>
Aug 09 02:27:14 user nova-compute[72351]:     <smbios mode='sysinfo'/>
Aug 09 02:27:14 user nova-compute[72351]:   </os>
Aug 09 02:27:14 user nova-compute[72351]:   <features>
Aug 09 02:27:14 user nova-compute[72351]:     <acpi/>
Aug 09 02:27:14 user nova-compute[72351]:     <apic/>
Aug 09 02:27:14 user nova-compute[72351]:     <vmcoreinfo state='on'/>
Aug 09 02:27:14 user nova-compute[72351]:   </features>
Aug 09 02:27:14 user nova-compute[72351]:   <cpu mode='custom' match='exact' check='full'>
Aug 09 02:27:14 user nova-compute[72351]:     <model fallback='forbid'>Nehalem</model>
Aug 09 02:27:14 user nova-compute[72351]:     <topology sockets='1' dies='1' cores='1' threads='1'/>
Aug 09 02:27:14 user nova-compute[72351]:     <feature policy='require' name='vme'/>
Aug 09 02:27:14 user nova-compute[72351]:     <feature policy='require' name='x2apic'/>
Aug 09 02:27:14 user nova-compute[72351]:     <feature policy='require' name='hypervisor'/>
Aug 09 02:27:14 user nova-compute[72351]:   </cpu>
Aug 09 02:27:14 user nova-compute[72351]:   <clock offset='utc'>
Aug 09 02:27:14 user nova-compute[72351]:     <timer name='pit' tickpolicy='delay'/>
Aug 09 02:27:14 user nova-compute[72351]:     <timer name='rtc' tickpolicy='catchup'/>
Aug 09 02:27:14 user nova-compute[72351]:     <timer name='hpet' present='no'/>
Aug 09 02:27:14 user nova-compute[72351]:   </clock>
Aug 09 02:27:14 user nova-compute[72351]:   <on_poweroff>destroy</on_poweroff>
Aug 09 02:27:14 user nova-compute[72351]:   <on_reboot>restart</on_reboot>
Aug 09 02:27:14 user nova-compute[72351]:   <on_crash>destroy</on_crash>
Aug 09 02:27:14 user nova-compute[72351]:   <devices>
Aug 09 02:27:14 user nova-compute[72351]:     <emulator>/usr/bin/qemu-system-x86_64</emulator>
Aug 09 02:27:14 user nova-compute[72351]:     <disk type='file' device='disk'>
Aug 09 02:27:14 user nova-compute[72351]:       <driver name='qemu' type='qcow2' cache='none'/>
Aug 09 02:27:14 user nova-compute[72351]:       <source file='/opt/stack/data/nova/instances/f457e6ea-8d06-46c6-be2e-f650e28a3b69/disk' index='2'/>
Aug 09 02:27:14 user nova-compute[72351]:       <backingStore type='file' index='3'>
Aug 09 02:27:14 user nova-compute[72351]:         <format type='raw'/>
Aug 09 02:27:14 user nova-compute[72351]:         <source file='/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd'/>
Aug 09 02:27:14 user nova-compute[72351]:         <backingStore/>
Aug 09 02:27:14 user nova-compute[72351]:       </backingStore>
Aug 09 02:27:14 user nova-compute[72351]:       <target dev='vda' bus='virtio'/>
Aug 09 02:27:14 user nova-compute[72351]:       <alias name='virtio-disk0'/>
Aug 09 02:27:14 user nova-compute[72351]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
Aug 09 02:27:14 user nova-compute[72351]:     </disk>
Aug 09 02:27:14 user nova-compute[72351]:     <disk type='file' device='cdrom'>
Aug 09 02:27:14 user nova-compute[72351]:       <driver name='qemu' type='raw' cache='none'/>
Aug 09 02:27:14 user nova-compute[72351]:       <source file='/opt/stack/data/nova/instances/f457e6ea-8d06-46c6-be2e-f650e28a3b69/disk.config' index='1'/>
Aug 09 02:27:14 user nova-compute[72351]:       <backingStore/>
Aug 09 02:27:14 user nova-compute[72351]:       <target dev='hda' bus='ide'/>
Aug 09 02:27:14 user nova-compute[72351]:       <readonly/>
Aug 09 02:27:14 user nova-compute[72351]:       <alias name='ide0-0-0'/>
Aug 09 02:27:14 user nova-compute[72351]:       <address type='drive' controller='0' bus='0' target='0' unit='0'/>
Aug 09 02:27:14 user nova-compute[72351]:     </disk>
Aug 09 02:27:14 user nova-compute[72351]:     <controller type='usb' index='0' model='none'>
Aug 09 02:27:14 user nova-compute[72351]:       <alias name='usb'/>
Aug 09 02:27:14 user nova-compute[72351]:     </controller>
Aug 09 02:27:14 user nova-compute[72351]:     <controller type='pci' index='0' model='pci-root'>
Aug 09 02:27:14 user nova-compute[72351]:       <alias name='pci.0'/>
Aug 09 02:27:14 user nova-compute[72351]:     </controller>
Aug 09 02:27:14 user nova-compute[72351]:     <controller type='ide' index='0'>
Aug 09 02:27:14 user nova-compute[72351]:       <alias name='ide'/>
Aug 09 02:27:14 user nova-compute[72351]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
Aug 09 02:27:14 user nova-compute[72351]:     </controller>
Aug 09 02:27:14 user nova-compute[72351]:     <interface type='ethernet'>
Aug 09 02:27:14 user nova-compute[72351]:       <mac address='fa:16:3e:0e:77:88'/>
Aug 09 02:27:14 user nova-compute[72351]:       <target dev='tapdc244cf9-b0'/>
Aug 09 02:27:14 user nova-compute[72351]:       <model type='virtio'/>
Aug 09 02:27:14 user nova-compute[72351]:       <mtu size='1442'/>
Aug 09 02:27:14 user nova-compute[72351]:       <alias name='net0'/>
Aug 09 02:27:14 user nova-compute[72351]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
Aug 09 02:27:14 user nova-compute[72351]:     </interface>
Aug 09 02:27:14 user nova-compute[72351]:     <interface type='ethernet'>
Aug 09 02:27:14 user nova-compute[72351]:       <mac address='fa:16:3e:30:6e:79'/>
Aug 09 02:27:14 user nova-compute[72351]:       <target dev='tap9bf76200-4b'/>
Aug 09 02:27:14 user nova-compute[72351]:       <model type='virtio'/>
Aug 09 02:27:14 user nova-compute[72351]:       <mtu size='1442'/>
Aug 09 02:27:14 user nova-compute[72351]:       <alias name='net1'/>
Aug 09 02:27:14 user nova-compute[72351]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
Aug 09 02:27:14 user nova-compute[72351]:     </interface>
Aug 09 02:27:14 user nova-compute[72351]:     <serial type='pty'>
Aug 09 02:27:14 user nova-compute[72351]:       <source path='/dev/pts/10'/>
Aug 09 02:27:14 user nova-compute[72351]:       <log file='/opt/stack/data/nova/instances/f457e6ea-8d06-46c6-be2e-f650e28a3b69/console.log' append='off'/>
Aug 09 02:27:14 user nova-compute[72351]:       <target type='isa-serial' port='0'>
Aug 09 02:27:14 user nova-compute[72351]:         <model name='isa-serial'/>
Aug 09 02:27:14 user nova-compute[72351]:       </target>
Aug 09 02:27:14 user nova-compute[72351]:       <alias name='serial0'/>
Aug 09 02:27:14 user nova-compute[72351]:     </serial>
Aug 09 02:27:14 user nova-compute[72351]:     <console type='pty' tty='/dev/pts/10'>
Aug 09 02:27:14 user nova-compute[72351]:       <source path='/dev/pts/10'/>
Aug 09 02:27:14 user nova-compute[72351]:       <log file='/opt/stack/data/nova/instances/f457e6ea-8d06-46c6-be2e-f650e28a3b69/console.log' append='off'/>
Aug 09 02:27:14 user nova-compute[72351]:       <target type='serial' port='0'/>
Aug 09 02:27:14 user nova-compute[72351]:       <alias name='serial0'/>
Aug 09 02:27:14 user nova-compute[72351]:     </console>
Aug 09 02:27:14 user nova-compute[72351]:     <input type='mouse' bus='ps2'>
Aug 09 02:27:14 user nova-compute[72351]:       <alias name='input0'/>
Aug 09 02:27:14 user nova-compute[72351]:     </input>
Aug 09 02:27:14 user nova-compute[72351]:     <input type='keyboard' bus='ps2'>
Aug 09 02:27:14 user nova-compute[72351]:       <alias name='input1'/>
Aug 09 02:27:14 user nova-compute[72351]:     </input>
Aug 09 02:27:14 user nova-compute[72351]:     <graphics type='vnc' port='5910' autoport='yes' listen='0.0.0.0'>
Aug 09 02:27:14 user nova-compute[72351]:       <listen type='address' address='0.0.0.0'/>
Aug 09 02:27:14 user nova-compute[72351]:     </graphics>
Aug 09 02:27:14 user nova-compute[72351]:     <audio id='1' type='none'/>
Aug 09 02:27:14 user nova-compute[72351]:     <video>
Aug 09 02:27:14 user nova-compute[72351]:       <model type='virtio' heads='1' primary='yes'/>
Aug 09 02:27:14 user nova-compute[72351]:       <alias name='video0'/>
Aug 09 02:27:14 user nova-compute[72351]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
Aug 09 02:27:14 user nova-compute[72351]:     </video>
Aug 09 02:27:14 user nova-compute[72351]:     <memballoon model='virtio'>
Aug 09 02:27:14 user nova-compute[72351]:       <stats period='10'/>
Aug 09 02:27:14 user nova-compute[72351]:       <alias name='balloon0'/>
Aug 09 02:27:14 user nova-compute[72351]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
Aug 09 02:27:14 user nova-compute[72351]:     </memballoon>
Aug 09 02:27:14 user nova-compute[72351]:     <rng model='virtio'>
Aug 09 02:27:14 user nova-compute[72351]:       <backend model='random'>/dev/urandom</backend>
Aug 09 02:27:14 user nova-compute[72351]:       <alias name='rng0'/>
Aug 09 02:27:14 user nova-compute[72351]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
Aug 09 02:27:14 user nova-compute[72351]:     </rng>
Aug 09 02:27:14 user nova-compute[72351]:   </devices>
Aug 09 02:27:14 user nova-compute[72351]:   <seclabel type='dynamic' model='apparmor' relabel='yes'>
Aug 09 02:27:14 user nova-compute[72351]:     <label>libvirt-f457e6ea-8d06-46c6-be2e-f650e28a3b69</label>
Aug 09 02:27:14 user nova-compute[72351]:     <imagelabel>libvirt-f457e6ea-8d06-46c6-be2e-f650e28a3b69</imagelabel>
Aug 09 02:27:14 user nova-compute[72351]:   </seclabel>
Aug 09 02:27:14 user nova-compute[72351]:   <seclabel type='dynamic' model='dac' relabel='yes'>
Aug 09 02:27:14 user nova-compute[72351]:     <label>+64055:+108</label>
Aug 09 02:27:14 user nova-compute[72351]:     <imagelabel>+64055:+108</imagelabel>
Aug 09 02:27:14 user nova-compute[72351]:   </seclabel>
Aug 09 02:27:14 user nova-compute[72351]: </domain>
Aug 09 02:27:14 user nova-compute[72351]:  {{(pid=72351) get_interface_by_cfg /opt/stack/nova/nova/virt/libvirt/guest.py:282}}
Aug 09 02:27:14 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-573693b1-fef0-4819-9ee7-6be13c801b1c tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Successfully detached device tap9bf76200-4b from instance f457e6ea-8d06-46c6-be2e-f650e28a3b69 from the persistent domain config.
Aug 09 02:27:14 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-573693b1-fef0-4819-9ee7-6be13c801b1c tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] (1/8): Attempting to detach device tap9bf76200-4b with device alias net1 from instance f457e6ea-8d06-46c6-be2e-f650e28a3b69 from the live domain config. {{(pid=72351) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2513}}
Aug 09 02:27:14 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-573693b1-fef0-4819-9ee7-6be13c801b1c tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] detach device xml: <interface type="ethernet">
Aug 09 02:27:14 user nova-compute[72351]:   <mac address="fa:16:3e:30:6e:79"/>
Aug 09 02:27:14 user nova-compute[72351]:   <model type="virtio"/>
Aug 09 02:27:14 user nova-compute[72351]:   <mtu size="1442"/>
Aug 09 02:27:14 user nova-compute[72351]:   <target dev="tap9bf76200-4b"/>
Aug 09 02:27:14 user nova-compute[72351]: </interface>
Aug 09 02:27:14 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 02:27:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:14 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Received event <DeviceRemovedEvent: 1691548034.401292, f457e6ea-8d06-46c6-be2e-f650e28a3b69 => net1> from libvirt while the driver is waiting for it; dispatched. {{(pid=72351) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2360}}
Aug 09 02:27:14 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-573693b1-fef0-4819-9ee7-6be13c801b1c tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Start waiting for the detach event from libvirt for device tap9bf76200-4b with device alias net1 for instance f457e6ea-8d06-46c6-be2e-f650e28a3b69 {{(pid=72351) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2589}}
Aug 09 02:27:14 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-573693b1-fef0-4819-9ee7-6be13c801b1c tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] looking for interface given config: <interface type="ethernet"><mac address="fa:16:3e:30:6e:79"/><model type="virtio"/><mtu size="1442"/><target dev="tap9bf76200-4b"/></interface> {{(pid=72351) get_interface_by_cfg /opt/stack/nova/nova/virt/libvirt/guest.py:257}}
Aug 09 02:27:14 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-573693b1-fef0-4819-9ee7-6be13c801b1c tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] interface for config: <interface type="ethernet"><mac address="fa:16:3e:30:6e:79"/><model type="virtio"/><mtu size="1442"/><target dev="tap9bf76200-4b"/></interface>not found in domain: <domain type='kvm' id='29'>
Aug 09 02:27:14 user nova-compute[72351]:   <name>instance-00000019</name>
Aug 09 02:27:14 user nova-compute[72351]:   <uuid>f457e6ea-8d06-46c6-be2e-f650e28a3b69</uuid>
Aug 09 02:27:14 user nova-compute[72351]:   <metadata>
Aug 09 02:27:14 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1" xmlns:instance="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:27:14 user nova-compute[72351]:   <nova:package version="0.0.0"/>
Aug 09 02:27:14 user nova-compute[72351]:   <nova:name>tempest-device-tagging-server-443615500</nova:name>
Aug 09 02:27:14 user nova-compute[72351]:   <nova:creationTime>2023-08-09 02:26:54</nova:creationTime>
Aug 09 02:27:14 user nova-compute[72351]:   <nova:flavor name="m1.tiny">
Aug 09 02:27:14 user nova-compute[72351]:     <nova:memory>512</nova:memory>
Aug 09 02:27:14 user nova-compute[72351]:     <nova:disk>1</nova:disk>
Aug 09 02:27:14 user nova-compute[72351]:     <nova:swap>0</nova:swap>
Aug 09 02:27:14 user nova-compute[72351]:     <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:27:14 user nova-compute[72351]:     <nova:vcpus>1</nova:vcpus>
Aug 09 02:27:14 user nova-compute[72351]:   </nova:flavor>
Aug 09 02:27:14 user nova-compute[72351]:   <nova:owner>
Aug 09 02:27:14 user nova-compute[72351]:     <nova:user uuid="8159c9e78af04e01a1faf9b3cfc3591e">tempest-TaggedAttachmentsTest-480515778-project-member</nova:user>
Aug 09 02:27:14 user nova-compute[72351]:     <nova:project uuid="bde0634670914ae8bfbbc6b31c09d6dd">tempest-TaggedAttachmentsTest-480515778</nova:project>
Aug 09 02:27:14 user nova-compute[72351]:   </nova:owner>
Aug 09 02:27:14 user nova-compute[72351]:   <nova:root type="image" uuid="18ccda7f-31d3-4978-ab9d-4ed2f74eff94"/>
Aug 09 02:27:14 user nova-compute[72351]:   <nova:ports>
Aug 09 02:27:14 user nova-compute[72351]:     <nova:port uuid="dc244cf9-b0f6-4906-b9f3-53634b1847e3">
Aug 09 02:27:14 user nova-compute[72351]:       <nova:ip type="fixed" address="10.0.0.9" ipVersion="4"/>
Aug 09 02:27:14 user nova-compute[72351]:     </nova:port>
Aug 09 02:27:14 user nova-compute[72351]:     <nova:port uuid="9bf76200-4b93-40dc-bd6f-0200cec7933a">
Aug 09 02:27:14 user nova-compute[72351]:       <nova:ip type="fixed" address="10.10.10.51" ipVersion="4"/>
Aug 09 02:27:14 user nova-compute[72351]:     </nova:port>
Aug 09 02:27:14 user nova-compute[72351]:   </nova:ports>
Aug 09 02:27:14 user nova-compute[72351]: </nova:instance>
Aug 09 02:27:14 user nova-compute[72351]:   </metadata>
Aug 09 02:27:14 user nova-compute[72351]:   <memory unit='KiB'>524288</memory>
Aug 09 02:27:14 user nova-compute[72351]:   <currentMemory unit='KiB'>524288</currentMemory>
Aug 09 02:27:14 user nova-compute[72351]:   <vcpu placement='static'>1</vcpu>
Aug 09 02:27:14 user nova-compute[72351]:   <resource>
Aug 09 02:27:14 user nova-compute[72351]:     <partition>/machine</partition>
Aug 09 02:27:14 user nova-compute[72351]:   </resource>
Aug 09 02:27:14 user nova-compute[72351]:   <sysinfo type='smbios'>
Aug 09 02:27:14 user nova-compute[72351]:     <system>
Aug 09 02:27:14 user nova-compute[72351]:       <entry name='manufacturer'>OpenStack Foundation</entry>
Aug 09 02:27:14 user nova-compute[72351]:       <entry name='product'>OpenStack Nova</entry>
Aug 09 02:27:14 user nova-compute[72351]:       <entry name='version'>0.0.0</entry>
Aug 09 02:27:14 user nova-compute[72351]:       <entry name='serial'>f457e6ea-8d06-46c6-be2e-f650e28a3b69</entry>
Aug 09 02:27:14 user nova-compute[72351]:       <entry name='uuid'>f457e6ea-8d06-46c6-be2e-f650e28a3b69</entry>
Aug 09 02:27:14 user nova-compute[72351]:       <entry name='family'>Virtual Machine</entry>
Aug 09 02:27:14 user nova-compute[72351]:     </system>
Aug 09 02:27:14 user nova-compute[72351]:   </sysinfo>
Aug 09 02:27:14 user nova-compute[72351]:   <os>
Aug 09 02:27:14 user nova-compute[72351]:     <type arch='x86_64' machine='pc-i440fx-6.2'>hvm</type>
Aug 09 02:27:14 user nova-compute[72351]:     <boot dev='hd'/>
Aug 09 02:27:14 user nova-compute[72351]:     <smbios mode='sysinfo'/>
Aug 09 02:27:14 user nova-compute[72351]:   </os>
Aug 09 02:27:14 user nova-compute[72351]:   <features>
Aug 09 02:27:14 user nova-compute[72351]:     <acpi/>
Aug 09 02:27:14 user nova-compute[72351]:     <apic/>
Aug 09 02:27:14 user nova-compute[72351]:     <vmcoreinfo state='on'/>
Aug 09 02:27:14 user nova-compute[72351]:   </features>
Aug 09 02:27:14 user nova-compute[72351]:   <cpu mode='custom' match='exact' check='full'>
Aug 09 02:27:14 user nova-compute[72351]:     <model fallback='forbid'>Nehalem</model>
Aug 09 02:27:14 user nova-compute[72351]:     <topology sockets='1' dies='1' cores='1' threads='1'/>
Aug 09 02:27:14 user nova-compute[72351]:     <feature policy='require' name='vme'/>
Aug 09 02:27:14 user nova-compute[72351]:     <feature policy='require' name='x2apic'/>
Aug 09 02:27:14 user nova-compute[72351]:     <feature policy='require' name='hypervisor'/>
Aug 09 02:27:14 user nova-compute[72351]:   </cpu>
Aug 09 02:27:14 user nova-compute[72351]:   <clock offset='utc'>
Aug 09 02:27:14 user nova-compute[72351]:     <timer name='pit' tickpolicy='delay'/>
Aug 09 02:27:14 user nova-compute[72351]:     <timer name='rtc' tickpolicy='catchup'/>
Aug 09 02:27:14 user nova-compute[72351]:     <timer name='hpet' present='no'/>
Aug 09 02:27:14 user nova-compute[72351]:   </clock>
Aug 09 02:27:14 user nova-compute[72351]:   <on_poweroff>destroy</on_poweroff>
Aug 09 02:27:14 user nova-compute[72351]:   <on_reboot>restart</on_reboot>
Aug 09 02:27:14 user nova-compute[72351]:   <on_crash>destroy</on_crash>
Aug 09 02:27:14 user nova-compute[72351]:   <devices>
Aug 09 02:27:14 user nova-compute[72351]:     <emulator>/usr/bin/qemu-system-x86_64</emulator>
Aug 09 02:27:14 user nova-compute[72351]:     <disk type='file' device='disk'>
Aug 09 02:27:14 user nova-compute[72351]:       <driver name='qemu' type='qcow2' cache='none'/>
Aug 09 02:27:14 user nova-compute[72351]:       <source file='/opt/stack/data/nova/instances/f457e6ea-8d06-46c6-be2e-f650e28a3b69/disk' index='2'/>
Aug 09 02:27:14 user nova-compute[72351]:       <backingStore type='file' index='3'>
Aug 09 02:27:14 user nova-compute[72351]:         <format type='raw'/>
Aug 09 02:27:14 user nova-compute[72351]:         <source file='/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd'/>
Aug 09 02:27:14 user nova-compute[72351]:         <backingStore/>
Aug 09 02:27:14 user nova-compute[72351]:       </backingStore>
Aug 09 02:27:14 user nova-compute[72351]:       <target dev='vda' bus='virtio'/>
Aug 09 02:27:14 user nova-compute[72351]:       <alias name='virtio-disk0'/>
Aug 09 02:27:14 user nova-compute[72351]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
Aug 09 02:27:14 user nova-compute[72351]:     </disk>
Aug 09 02:27:14 user nova-compute[72351]:     <disk type='file' device='cdrom'>
Aug 09 02:27:14 user nova-compute[72351]:       <driver name='qemu' type='raw' cache='none'/>
Aug 09 02:27:14 user nova-compute[72351]:       <source file='/opt/stack/data/nova/instances/f457e6ea-8d06-46c6-be2e-f650e28a3b69/disk.config' index='1'/>
Aug 09 02:27:14 user nova-compute[72351]:       <backingStore/>
Aug 09 02:27:14 user nova-compute[72351]:       <target dev='hda' bus='ide'/>
Aug 09 02:27:14 user nova-compute[72351]:       <readonly/>
Aug 09 02:27:14 user nova-compute[72351]:       <alias name='ide0-0-0'/>
Aug 09 02:27:14 user nova-compute[72351]:       <address type='drive' controller='0' bus='0' target='0' unit='0'/>
Aug 09 02:27:14 user nova-compute[72351]:     </disk>
Aug 09 02:27:14 user nova-compute[72351]:     <controller type='usb' index='0' model='none'>
Aug 09 02:27:14 user nova-compute[72351]:       <alias name='usb'/>
Aug 09 02:27:14 user nova-compute[72351]:     </controller>
Aug 09 02:27:14 user nova-compute[72351]:     <controller type='pci' index='0' model='pci-root'>
Aug 09 02:27:14 user nova-compute[72351]:       <alias name='pci.0'/>
Aug 09 02:27:14 user nova-compute[72351]:     </controller>
Aug 09 02:27:14 user nova-compute[72351]:     <controller type='ide' index='0'>
Aug 09 02:27:14 user nova-compute[72351]:       <alias name='ide'/>
Aug 09 02:27:14 user nova-compute[72351]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
Aug 09 02:27:14 user nova-compute[72351]:     </controller>
Aug 09 02:27:14 user nova-compute[72351]:     <interface type='ethernet'>
Aug 09 02:27:14 user nova-compute[72351]:       <mac address='fa:16:3e:0e:77:88'/>
Aug 09 02:27:14 user nova-compute[72351]:       <target dev='tapdc244cf9-b0'/>
Aug 09 02:27:14 user nova-compute[72351]:       <model type='virtio'/>
Aug 09 02:27:14 user nova-compute[72351]:       <mtu size='1442'/>
Aug 09 02:27:14 user nova-compute[72351]:       <alias name='net0'/>
Aug 09 02:27:14 user nova-compute[72351]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
Aug 09 02:27:14 user nova-compute[72351]:     </interface>
Aug 09 02:27:14 user nova-compute[72351]:     <serial type='pty'>
Aug 09 02:27:14 user nova-compute[72351]:       <source path='/dev/pts/10'/>
Aug 09 02:27:14 user nova-compute[72351]:       <log file='/opt/stack/data/nova/instances/f457e6ea-8d06-46c6-be2e-f650e28a3b69/console.log' append='off'/>
Aug 09 02:27:14 user nova-compute[72351]:       <target type='isa-serial' port='0'>
Aug 09 02:27:14 user nova-compute[72351]:         <model name='isa-serial'/>
Aug 09 02:27:14 user nova-compute[72351]:       </target>
Aug 09 02:27:14 user nova-compute[72351]:       <alias name='serial0'/>
Aug 09 02:27:14 user nova-compute[72351]:     </serial>
Aug 09 02:27:14 user nova-compute[72351]:     <console type='pty' tty='/dev/pts/10'>
Aug 09 02:27:14 user nova-compute[72351]:       <source path='/dev/pts/10'/>
Aug 09 02:27:14 user nova-compute[72351]:       <log file='/opt/stack/data/nova/instances/f457e6ea-8d06-46c6-be2e-f650e28a3b69/console.log' append='off'/>
Aug 09 02:27:14 user nova-compute[72351]:       <target type='serial' port='0'/>
Aug 09 02:27:14 user nova-compute[72351]:       <alias name='serial0'/>
Aug 09 02:27:14 user nova-compute[72351]:     </console>
Aug 09 02:27:14 user nova-compute[72351]:     <input type='mouse' bus='ps2'>
Aug 09 02:27:14 user nova-compute[72351]:       <alias name='input0'/>
Aug 09 02:27:14 user nova-compute[72351]:     </input>
Aug 09 02:27:14 user nova-compute[72351]:     <input type='keyboard' bus='ps2'>
Aug 09 02:27:14 user nova-compute[72351]:       <alias name='input1'/>
Aug 09 02:27:14 user nova-compute[72351]:     </input>
Aug 09 02:27:14 user nova-compute[72351]:     <graphics type='vnc' port='5910' autoport='yes' listen='0.0.0.0'>
Aug 09 02:27:14 user nova-compute[72351]:       <listen type='address' address='0.0.0.0'/>
Aug 09 02:27:14 user nova-compute[72351]:     </graphics>
Aug 09 02:27:14 user nova-compute[72351]:     <audio id='1' type='none'/>
Aug 09 02:27:14 user nova-compute[72351]:     <video>
Aug 09 02:27:14 user nova-compute[72351]:       <model type='virtio' heads='1' primary='yes'/>
Aug 09 02:27:14 user nova-compute[72351]:       <alias name='video0'/>
Aug 09 02:27:14 user nova-compute[72351]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
Aug 09 02:27:14 user nova-compute[72351]:     </video>
Aug 09 02:27:14 user nova-compute[72351]:     <memballoon model='virtio'>
Aug 09 02:27:14 user nova-compute[72351]:       <stats period='10'/>
Aug 09 02:27:14 user nova-compute[72351]:       <alias name='balloon0'/>
Aug 09 02:27:14 user nova-compute[72351]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
Aug 09 02:27:14 user nova-compute[72351]:     </memballoon>
Aug 09 02:27:14 user nova-compute[72351]:     <rng model='virtio'>
Aug 09 02:27:14 user nova-compute[72351]:       <backend model='random'>/dev/urandom</backend>
Aug 09 02:27:14 user nova-compute[72351]:       <alias name='rng0'/>
Aug 09 02:27:14 user nova-compute[72351]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
Aug 09 02:27:14 user nova-compute[72351]:     </rng>
Aug 09 02:27:14 user nova-compute[72351]:   </devices>
Aug 09 02:27:14 user nova-compute[72351]:   <seclabel type='dynamic' model='apparmor' relabel='yes'>
Aug 09 02:27:14 user nova-compute[72351]:     <label>libvirt-f457e6ea-8d06-46c6-be2e-f650e28a3b69</label>
Aug 09 02:27:14 user nova-compute[72351]:     <imagelabel>libvirt-f457e6ea-8d06-46c6-be2e-f650e28a3b69</imagelabel>
Aug 09 02:27:14 user nova-compute[72351]:   </seclabel>
Aug 09 02:27:14 user nova-compute[72351]:   <seclabel type='dynamic' model='dac' relabel='yes'>
Aug 09 02:27:14 user nova-compute[72351]:     <label>+64055:+108</label>
Aug 09 02:27:14 user nova-compute[72351]:     <imagelabel>+64055:+108</imagelabel>
Aug 09 02:27:14 user nova-compute[72351]:   </seclabel>
Aug 09 02:27:14 user nova-compute[72351]: </domain>
Aug 09 02:27:14 user nova-compute[72351]:  {{(pid=72351) get_interface_by_cfg /opt/stack/nova/nova/virt/libvirt/guest.py:282}}
Aug 09 02:27:14 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-573693b1-fef0-4819-9ee7-6be13c801b1c tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Successfully detached device tap9bf76200-4b from instance f457e6ea-8d06-46c6-be2e-f650e28a3b69 from the live domain config.
Aug 09 02:27:14 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-573693b1-fef0-4819-9ee7-6be13c801b1c tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-08-09T02:24:58Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=InstanceDeviceMetadata,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-443615500',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-443615500',id=25,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBK8nrHk7zghGHoCUEI3vExNw2oXJc2YwDe7QrMxgC2DNUP/PkVzpLwy9DWnyTLCCR4ddGxe1t7xIbXts5zbiafpc3PmU6by4kyzLopbWZ1BYcJyQcmguVSz4dz8HMbCR9g==',key_name='tempest-keypair-1705746290',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:25:05Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=<?>,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='bde0634670914ae8bfbbc6b31c09d6dd',ramdisk_id='',reservation_id='r-5xxup5ze',resources=<?>,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedAttachmentsTest-480515778',owner_user_name='tempest-TaggedAttachmentsTest-480515778-project-member'},tags=<?>,task_state=None,terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:25:06Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='8159c9e78af04e01a1faf9b3cfc3591e',uuid=f457e6ea-8d06-46c6-be2e-f650e28a3b69,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "9bf76200-4b93-40dc-bd6f-0200cec7933a", "address": "fa:16:3e:30:6e:79", "network": {"id": "ea588e91-dd01-4f9c-ad88-30f0848e6839", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-596948871", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.51", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "bde0634670914ae8bfbbc6b31c09d6dd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9bf76200-4b", "ovs_interfaceid": "9bf76200-4b93-40dc-bd6f-0200cec7933a", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:27:14 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-573693b1-fef0-4819-9ee7-6be13c801b1c tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Converting VIF {"id": "9bf76200-4b93-40dc-bd6f-0200cec7933a", "address": "fa:16:3e:30:6e:79", "network": {"id": "ea588e91-dd01-4f9c-ad88-30f0848e6839", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-596948871", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.51", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "bde0634670914ae8bfbbc6b31c09d6dd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9bf76200-4b", "ovs_interfaceid": "9bf76200-4b93-40dc-bd6f-0200cec7933a", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:27:14 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-573693b1-fef0-4819-9ee7-6be13c801b1c tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:30:6e:79,bridge_name='br-int',has_traffic_filtering=True,id=9bf76200-4b93-40dc-bd6f-0200cec7933a,network=Network(ea588e91-dd01-4f9c-ad88-30f0848e6839),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap9bf76200-4b') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:27:14 user nova-compute[72351]: DEBUG os_vif [None req-573693b1-fef0-4819-9ee7-6be13c801b1c tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:30:6e:79,bridge_name='br-int',has_traffic_filtering=True,id=9bf76200-4b93-40dc-bd6f-0200cec7933a,network=Network(ea588e91-dd01-4f9c-ad88-30f0848e6839),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap9bf76200-4b') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:27:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap9bf76200-4b, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:27:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:27:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:14 user nova-compute[72351]: INFO os_vif [None req-573693b1-fef0-4819-9ee7-6be13c801b1c tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:30:6e:79,bridge_name='br-int',has_traffic_filtering=True,id=9bf76200-4b93-40dc-bd6f-0200cec7933a,network=Network(ea588e91-dd01-4f9c-ad88-30f0848e6839),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap9bf76200-4b')
Aug 09 02:27:14 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-573693b1-fef0-4819-9ee7-6be13c801b1c tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] set metadata xml: <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:27:14 user nova-compute[72351]:   <nova:package version="0.0.0"/>
Aug 09 02:27:14 user nova-compute[72351]:   <nova:name>tempest-device-tagging-server-443615500</nova:name>
Aug 09 02:27:14 user nova-compute[72351]:   <nova:creationTime>2023-08-09 02:27:14</nova:creationTime>
Aug 09 02:27:14 user nova-compute[72351]:   <nova:flavor name="m1.tiny">
Aug 09 02:27:14 user nova-compute[72351]:     <nova:memory>512</nova:memory>
Aug 09 02:27:14 user nova-compute[72351]:     <nova:disk>1</nova:disk>
Aug 09 02:27:14 user nova-compute[72351]:     <nova:swap>0</nova:swap>
Aug 09 02:27:14 user nova-compute[72351]:     <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:27:14 user nova-compute[72351]:     <nova:vcpus>1</nova:vcpus>
Aug 09 02:27:14 user nova-compute[72351]:   </nova:flavor>
Aug 09 02:27:14 user nova-compute[72351]:   <nova:owner>
Aug 09 02:27:14 user nova-compute[72351]:     <nova:user uuid="8159c9e78af04e01a1faf9b3cfc3591e">tempest-TaggedAttachmentsTest-480515778-project-member</nova:user>
Aug 09 02:27:14 user nova-compute[72351]:     <nova:project uuid="bde0634670914ae8bfbbc6b31c09d6dd">tempest-TaggedAttachmentsTest-480515778</nova:project>
Aug 09 02:27:14 user nova-compute[72351]:   </nova:owner>
Aug 09 02:27:14 user nova-compute[72351]:   <nova:root type="image" uuid="18ccda7f-31d3-4978-ab9d-4ed2f74eff94"/>
Aug 09 02:27:14 user nova-compute[72351]:   <nova:ports>
Aug 09 02:27:14 user nova-compute[72351]:     <nova:port uuid="dc244cf9-b0f6-4906-b9f3-53634b1847e3">
Aug 09 02:27:14 user nova-compute[72351]:       <nova:ip type="fixed" address="10.0.0.9" ipVersion="4"/>
Aug 09 02:27:14 user nova-compute[72351]:     </nova:port>
Aug 09 02:27:14 user nova-compute[72351]:   </nova:ports>
Aug 09 02:27:14 user nova-compute[72351]: </nova:instance>
Aug 09 02:27:14 user nova-compute[72351]:  {{(pid=72351) set_metadata /opt/stack/nova/nova/virt/libvirt/guest.py:359}}
Aug 09 02:27:14 user nova-compute[72351]: DEBUG nova.compute.manager [req-08178058-fe48-469a-85fa-545087bcc746 req-9cf414cf-6bae-4f84-a775-86cb8e1b323b service nova] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Received event network-vif-unplugged-9bf76200-4b93-40dc-bd6f-0200cec7933a {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:27:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-08178058-fe48-469a-85fa-545087bcc746 req-9cf414cf-6bae-4f84-a775-86cb8e1b323b service nova] Acquiring lock "f457e6ea-8d06-46c6-be2e-f650e28a3b69-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:27:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-08178058-fe48-469a-85fa-545087bcc746 req-9cf414cf-6bae-4f84-a775-86cb8e1b323b service nova] Lock "f457e6ea-8d06-46c6-be2e-f650e28a3b69-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:27:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-08178058-fe48-469a-85fa-545087bcc746 req-9cf414cf-6bae-4f84-a775-86cb8e1b323b service nova] Lock "f457e6ea-8d06-46c6-be2e-f650e28a3b69-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:27:14 user nova-compute[72351]: DEBUG nova.compute.manager [req-08178058-fe48-469a-85fa-545087bcc746 req-9cf414cf-6bae-4f84-a775-86cb8e1b323b service nova] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] No waiting events found dispatching network-vif-unplugged-9bf76200-4b93-40dc-bd6f-0200cec7933a {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:27:14 user nova-compute[72351]: WARNING nova.compute.manager [req-08178058-fe48-469a-85fa-545087bcc746 req-9cf414cf-6bae-4f84-a775-86cb8e1b323b service nova] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Received unexpected event network-vif-unplugged-9bf76200-4b93-40dc-bd6f-0200cec7933a for instance with vm_state active and task_state None.
Aug 09 02:27:15 user nova-compute[72351]: DEBUG nova.compute.manager [req-add8da7b-7146-453b-8b63-db836f77e9cd req-486a1c34-d40f-4811-8f7e-3e14fb9a9963 service nova] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Received event network-vif-deleted-9bf76200-4b93-40dc-bd6f-0200cec7933a {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:27:15 user nova-compute[72351]: INFO nova.compute.manager [req-add8da7b-7146-453b-8b63-db836f77e9cd req-486a1c34-d40f-4811-8f7e-3e14fb9a9963 service nova] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Neutron deleted interface 9bf76200-4b93-40dc-bd6f-0200cec7933a; detaching it from the instance and deleting it from the info cache
Aug 09 02:27:15 user nova-compute[72351]: DEBUG nova.network.neutron [req-add8da7b-7146-453b-8b63-db836f77e9cd req-486a1c34-d40f-4811-8f7e-3e14fb9a9963 service nova] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Updating instance_info_cache with network_info: [{"id": "dc244cf9-b0f6-4906-b9f3-53634b1847e3", "address": "fa:16:3e:0e:77:88", "network": {"id": "3fdd9499-7552-4ecc-a897-ed4c58be9519", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-969934765-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.20", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bde0634670914ae8bfbbc6b31c09d6dd", "mtu": null, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdc244cf9-b0", "ovs_interfaceid": "dc244cf9-b0f6-4906-b9f3-53634b1847e3", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-573693b1-fef0-4819-9ee7-6be13c801b1c tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Acquiring lock "refresh_cache-f457e6ea-8d06-46c6-be2e-f650e28a3b69" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-573693b1-fef0-4819-9ee7-6be13c801b1c tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Acquired lock "refresh_cache-f457e6ea-8d06-46c6-be2e-f650e28a3b69" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG nova.network.neutron [None req-573693b1-fef0-4819-9ee7-6be13c801b1c tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG nova.objects.instance [req-add8da7b-7146-453b-8b63-db836f77e9cd req-486a1c34-d40f-4811-8f7e-3e14fb9a9963 service nova] Lazy-loading 'system_metadata' on Instance uuid f457e6ea-8d06-46c6-be2e-f650e28a3b69 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Starting finish_revert_migration {{(pid=72351) finish_revert_migration /opt/stack/nova/nova/virt/libvirt/driver.py:11819}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG nova.objects.instance [req-add8da7b-7146-453b-8b63-db836f77e9cd req-486a1c34-d40f-4811-8f7e-3e14fb9a9963 service nova] Lazy-loading 'flavor' on Instance uuid f457e6ea-8d06-46c6-be2e-f650e28a3b69 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Start _get_guest_xml network_info=[{"id": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "address": "fa:16:3e:8b:b5:2e", "network": {"id": "8231f2c2-237e-41c6-8892-720b950c1453", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherB-1917380761-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.30", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fc90e94bbcb54fb6b86a9ac45edb596a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbe4ec483-85", "ovs_interfaceid": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vdb': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} image_meta=ImageMeta(checksum=<?>,container_format='bare',created_at=<?>,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=1,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=<?>,status=<?>,tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'size': 0, 'device_type': 'disk', 'encryption_secret_uuid': None, 'guest_format': None, 'encryption_options': None, 'disk_bus': 'virtio', 'encrypted': False, 'encryption_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'image_id': '18ccda7f-31d3-4978-ab9d-4ed2f74eff94'}], 'ephemerals': [], 'block_device_mapping': [{'device_type': 'disk', 'guest_format': None, 'disk_bus': 'virtio', 'mount_device': '/dev/vdb', 'boot_index': None, 'attachment_id': '09d3bdb7-1738-402a-af19-21d027385bf7', 'delete_on_termination': False, 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqns': ['iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'oTnweUhv', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'attaching', 'instance': '75bbbead-8628-448b-9484-19bf4ab922ff', 'attached_at': '2023-08-09T02:27:15.000000', 'detached_at': '', 'volume_id': '2db51b4f-d032-4367-bb6f-8964563eb8fd', 'serial': '2db51b4f-d032-4367-bb6f-8964563eb8fd'}, 'volume_type': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:27:15 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:27:15 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:27:15 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [req-add8da7b-7146-453b-8b63-db836f77e9cd req-486a1c34-d40f-4811-8f7e-3e14fb9a9963 service nova] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-08-09T02:24:58Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-443615500',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-443615500',id=25,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBK8nrHk7zghGHoCUEI3vExNw2oXJc2YwDe7QrMxgC2DNUP/PkVzpLwy9DWnyTLCCR4ddGxe1t7xIbXts5zbiafpc3PmU6by4kyzLopbWZ1BYcJyQcmguVSz4dz8HMbCR9g==',key_name='tempest-keypair-1705746290',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:25:05Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata=<?>,migration_context=None,new_flavor=None,node='user',numa_topology=<?>,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='bde0634670914ae8bfbbc6b31c09d6dd',ramdisk_id='',reservation_id='r-5xxup5ze',resources=<?>,root_device_name='/dev/vda',root_gb=1,security_groups=<?>,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedAttachmentsTest-480515778',owner_user_name='tempest-TaggedAttachmentsTest-480515778-project-member'},tags=<?>,task_state=None,terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:25:06Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='8159c9e78af04e01a1faf9b3cfc3591e',uuid=f457e6ea-8d06-46c6-be2e-f650e28a3b69,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "9bf76200-4b93-40dc-bd6f-0200cec7933a", "address": "fa:16:3e:30:6e:79", "network": {"id": "ea588e91-dd01-4f9c-ad88-30f0848e6839", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-596948871", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.51", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "bde0634670914ae8bfbbc6b31c09d6dd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9bf76200-4b", "ovs_interfaceid": "9bf76200-4b93-40dc-bd6f-0200cec7933a", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG nova.network.os_vif_util [req-add8da7b-7146-453b-8b63-db836f77e9cd req-486a1c34-d40f-4811-8f7e-3e14fb9a9963 service nova] Converting VIF {"id": "9bf76200-4b93-40dc-bd6f-0200cec7933a", "address": "fa:16:3e:30:6e:79", "network": {"id": "ea588e91-dd01-4f9c-ad88-30f0848e6839", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-596948871", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.51", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "bde0634670914ae8bfbbc6b31c09d6dd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9bf76200-4b", "ovs_interfaceid": "9bf76200-4b93-40dc-bd6f-0200cec7933a", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG nova.network.os_vif_util [req-add8da7b-7146-453b-8b63-db836f77e9cd req-486a1c34-d40f-4811-8f7e-3e14fb9a9963 service nova] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:30:6e:79,bridge_name='br-int',has_traffic_filtering=True,id=9bf76200-4b93-40dc-bd6f-0200cec7933a,network=Network(ea588e91-dd01-4f9c-ad88-30f0848e6839),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap9bf76200-4b') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [req-add8da7b-7146-453b-8b63-db836f77e9cd req-486a1c34-d40f-4811-8f7e-3e14fb9a9963 service nova] looking for interface given config: <interface type="ethernet"><mac address="fa:16:3e:30:6e:79"/><model type="virtio"/><mtu size="1442"/><target dev="tap9bf76200-4b"/></interface> {{(pid=72351) get_interface_by_cfg /opt/stack/nova/nova/virt/libvirt/guest.py:257}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format='bare',created_at=<?>,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=1,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=<?>,status=<?>,tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG nova.objects.instance [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Lazy-loading 'vcpu_model' on Instance uuid 75bbbead-8628-448b-9484-19bf4ab922ff {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [req-add8da7b-7146-453b-8b63-db836f77e9cd req-486a1c34-d40f-4811-8f7e-3e14fb9a9963 service nova] interface for config: <interface type="ethernet"><mac address="fa:16:3e:30:6e:79"/><model type="virtio"/><mtu size="1442"/><target dev="tap9bf76200-4b"/></interface>not found in domain: <domain type='kvm' id='29'>
Aug 09 02:27:15 user nova-compute[72351]:   <name>instance-00000019</name>
Aug 09 02:27:15 user nova-compute[72351]:   <uuid>f457e6ea-8d06-46c6-be2e-f650e28a3b69</uuid>
Aug 09 02:27:15 user nova-compute[72351]:   <metadata>
Aug 09 02:27:15 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1" xmlns:instance="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:27:15 user nova-compute[72351]:   <nova:package version="0.0.0"/>
Aug 09 02:27:15 user nova-compute[72351]:   <nova:name>tempest-device-tagging-server-443615500</nova:name>
Aug 09 02:27:15 user nova-compute[72351]:   <nova:creationTime>2023-08-09 02:27:14</nova:creationTime>
Aug 09 02:27:15 user nova-compute[72351]:   <nova:flavor name="m1.tiny">
Aug 09 02:27:15 user nova-compute[72351]:     <nova:memory>512</nova:memory>
Aug 09 02:27:15 user nova-compute[72351]:     <nova:disk>1</nova:disk>
Aug 09 02:27:15 user nova-compute[72351]:     <nova:swap>0</nova:swap>
Aug 09 02:27:15 user nova-compute[72351]:     <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:27:15 user nova-compute[72351]:     <nova:vcpus>1</nova:vcpus>
Aug 09 02:27:15 user nova-compute[72351]:   </nova:flavor>
Aug 09 02:27:15 user nova-compute[72351]:   <nova:owner>
Aug 09 02:27:15 user nova-compute[72351]:     <nova:user uuid="8159c9e78af04e01a1faf9b3cfc3591e">tempest-TaggedAttachmentsTest-480515778-project-member</nova:user>
Aug 09 02:27:15 user nova-compute[72351]:     <nova:project uuid="bde0634670914ae8bfbbc6b31c09d6dd">tempest-TaggedAttachmentsTest-480515778</nova:project>
Aug 09 02:27:15 user nova-compute[72351]:   </nova:owner>
Aug 09 02:27:15 user nova-compute[72351]:   <nova:root type="image" uuid="18ccda7f-31d3-4978-ab9d-4ed2f74eff94"/>
Aug 09 02:27:15 user nova-compute[72351]:   <nova:ports>
Aug 09 02:27:15 user nova-compute[72351]:     <nova:port uuid="dc244cf9-b0f6-4906-b9f3-53634b1847e3">
Aug 09 02:27:15 user nova-compute[72351]:       <nova:ip type="fixed" address="10.0.0.9" ipVersion="4"/>
Aug 09 02:27:15 user nova-compute[72351]:     </nova:port>
Aug 09 02:27:15 user nova-compute[72351]:   </nova:ports>
Aug 09 02:27:15 user nova-compute[72351]: </nova:instance>
Aug 09 02:27:15 user nova-compute[72351]:   </metadata>
Aug 09 02:27:15 user nova-compute[72351]:   <memory unit='KiB'>524288</memory>
Aug 09 02:27:15 user nova-compute[72351]:   <currentMemory unit='KiB'>524288</currentMemory>
Aug 09 02:27:15 user nova-compute[72351]:   <vcpu placement='static'>1</vcpu>
Aug 09 02:27:15 user nova-compute[72351]:   <resource>
Aug 09 02:27:15 user nova-compute[72351]:     <partition>/machine</partition>
Aug 09 02:27:15 user nova-compute[72351]:   </resource>
Aug 09 02:27:15 user nova-compute[72351]:   <sysinfo type='smbios'>
Aug 09 02:27:15 user nova-compute[72351]:     <system>
Aug 09 02:27:15 user nova-compute[72351]:       <entry name='manufacturer'>OpenStack Foundation</entry>
Aug 09 02:27:15 user nova-compute[72351]:       <entry name='product'>OpenStack Nova</entry>
Aug 09 02:27:15 user nova-compute[72351]:       <entry name='version'>0.0.0</entry>
Aug 09 02:27:15 user nova-compute[72351]:       <entry name='serial'>f457e6ea-8d06-46c6-be2e-f650e28a3b69</entry>
Aug 09 02:27:15 user nova-compute[72351]:       <entry name='uuid'>f457e6ea-8d06-46c6-be2e-f650e28a3b69</entry>
Aug 09 02:27:15 user nova-compute[72351]:       <entry name='family'>Virtual Machine</entry>
Aug 09 02:27:15 user nova-compute[72351]:     </system>
Aug 09 02:27:15 user nova-compute[72351]:   </sysinfo>
Aug 09 02:27:15 user nova-compute[72351]:   <os>
Aug 09 02:27:15 user nova-compute[72351]:     <type arch='x86_64' machine='pc-i440fx-6.2'>hvm</type>
Aug 09 02:27:15 user nova-compute[72351]:     <boot dev='hd'/>
Aug 09 02:27:15 user nova-compute[72351]:     <smbios mode='sysinfo'/>
Aug 09 02:27:15 user nova-compute[72351]:   </os>
Aug 09 02:27:15 user nova-compute[72351]:   <features>
Aug 09 02:27:15 user nova-compute[72351]:     <acpi/>
Aug 09 02:27:15 user nova-compute[72351]:     <apic/>
Aug 09 02:27:15 user nova-compute[72351]:     <vmcoreinfo state='on'/>
Aug 09 02:27:15 user nova-compute[72351]:   </features>
Aug 09 02:27:15 user nova-compute[72351]:   <cpu mode='custom' match='exact' check='full'>
Aug 09 02:27:15 user nova-compute[72351]:     <model fallback='forbid'>Nehalem</model>
Aug 09 02:27:15 user nova-compute[72351]:     <topology sockets='1' dies='1' cores='1' threads='1'/>
Aug 09 02:27:15 user nova-compute[72351]:     <feature policy='require' name='vme'/>
Aug 09 02:27:15 user nova-compute[72351]:     <feature policy='require' name='x2apic'/>
Aug 09 02:27:15 user nova-compute[72351]:     <feature policy='require' name='hypervisor'/>
Aug 09 02:27:15 user nova-compute[72351]:   </cpu>
Aug 09 02:27:15 user nova-compute[72351]:   <clock offset='utc'>
Aug 09 02:27:15 user nova-compute[72351]:     <timer name='pit' tickpolicy='delay'/>
Aug 09 02:27:15 user nova-compute[72351]:     <timer name='rtc' tickpolicy='catchup'/>
Aug 09 02:27:15 user nova-compute[72351]:     <timer name='hpet' present='no'/>
Aug 09 02:27:15 user nova-compute[72351]:   </clock>
Aug 09 02:27:15 user nova-compute[72351]:   <on_poweroff>destroy</on_poweroff>
Aug 09 02:27:15 user nova-compute[72351]:   <on_reboot>restart</on_reboot>
Aug 09 02:27:15 user nova-compute[72351]:   <on_crash>destroy</on_crash>
Aug 09 02:27:15 user nova-compute[72351]:   <devices>
Aug 09 02:27:15 user nova-compute[72351]:     <emulator>/usr/bin/qemu-system-x86_64</emulator>
Aug 09 02:27:15 user nova-compute[72351]:     <disk type='file' device='disk'>
Aug 09 02:27:15 user nova-compute[72351]:       <driver name='qemu' type='qcow2' cache='none'/>
Aug 09 02:27:15 user nova-compute[72351]:       <source file='/opt/stack/data/nova/instances/f457e6ea-8d06-46c6-be2e-f650e28a3b69/disk' index='2'/>
Aug 09 02:27:15 user nova-compute[72351]:       <backingStore type='file' index='3'>
Aug 09 02:27:15 user nova-compute[72351]:         <format type='raw'/>
Aug 09 02:27:15 user nova-compute[72351]:         <source file='/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd'/>
Aug 09 02:27:15 user nova-compute[72351]:         <backingStore/>
Aug 09 02:27:15 user nova-compute[72351]:       </backingStore>
Aug 09 02:27:15 user nova-compute[72351]:       <target dev='vda' bus='virtio'/>
Aug 09 02:27:15 user nova-compute[72351]:       <alias name='virtio-disk0'/>
Aug 09 02:27:15 user nova-compute[72351]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
Aug 09 02:27:15 user nova-compute[72351]:     </disk>
Aug 09 02:27:15 user nova-compute[72351]:     <disk type='file' device='cdrom'>
Aug 09 02:27:15 user nova-compute[72351]:       <driver name='qemu' type='raw' cache='none'/>
Aug 09 02:27:15 user nova-compute[72351]:       <source file='/opt/stack/data/nova/instances/f457e6ea-8d06-46c6-be2e-f650e28a3b69/disk.config' index='1'/>
Aug 09 02:27:15 user nova-compute[72351]:       <backingStore/>
Aug 09 02:27:15 user nova-compute[72351]:       <target dev='hda' bus='ide'/>
Aug 09 02:27:15 user nova-compute[72351]:       <readonly/>
Aug 09 02:27:15 user nova-compute[72351]:       <alias name='ide0-0-0'/>
Aug 09 02:27:15 user nova-compute[72351]:       <address type='drive' controller='0' bus='0' target='0' unit='0'/>
Aug 09 02:27:15 user nova-compute[72351]:     </disk>
Aug 09 02:27:15 user nova-compute[72351]:     <controller type='usb' index='0' model='none'>
Aug 09 02:27:15 user nova-compute[72351]:       <alias name='usb'/>
Aug 09 02:27:15 user nova-compute[72351]:     </controller>
Aug 09 02:27:15 user nova-compute[72351]:     <controller type='pci' index='0' model='pci-root'>
Aug 09 02:27:15 user nova-compute[72351]:       <alias name='pci.0'/>
Aug 09 02:27:15 user nova-compute[72351]:     </controller>
Aug 09 02:27:15 user nova-compute[72351]:     <controller type='ide' index='0'>
Aug 09 02:27:15 user nova-compute[72351]:       <alias name='ide'/>
Aug 09 02:27:15 user nova-compute[72351]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
Aug 09 02:27:15 user nova-compute[72351]:     </controller>
Aug 09 02:27:15 user nova-compute[72351]:     <interface type='ethernet'>
Aug 09 02:27:15 user nova-compute[72351]:       <mac address='fa:16:3e:0e:77:88'/>
Aug 09 02:27:15 user nova-compute[72351]:       <target dev='tapdc244cf9-b0'/>
Aug 09 02:27:15 user nova-compute[72351]:       <model type='virtio'/>
Aug 09 02:27:15 user nova-compute[72351]:       <mtu size='1442'/>
Aug 09 02:27:15 user nova-compute[72351]:       <alias name='net0'/>
Aug 09 02:27:15 user nova-compute[72351]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
Aug 09 02:27:15 user nova-compute[72351]:     </interface>
Aug 09 02:27:15 user nova-compute[72351]:     <serial type='pty'>
Aug 09 02:27:15 user nova-compute[72351]:       <source path='/dev/pts/10'/>
Aug 09 02:27:15 user nova-compute[72351]:       <log file='/opt/stack/data/nova/instances/f457e6ea-8d06-46c6-be2e-f650e28a3b69/console.log' append='off'/>
Aug 09 02:27:15 user nova-compute[72351]:       <target type='isa-serial' port='0'>
Aug 09 02:27:15 user nova-compute[72351]:         <model name='isa-serial'/>
Aug 09 02:27:15 user nova-compute[72351]:       </target>
Aug 09 02:27:15 user nova-compute[72351]:       <alias name='serial0'/>
Aug 09 02:27:15 user nova-compute[72351]:     </serial>
Aug 09 02:27:15 user nova-compute[72351]:     <console type='pty' tty='/dev/pts/10'>
Aug 09 02:27:15 user nova-compute[72351]:       <source path='/dev/pts/10'/>
Aug 09 02:27:15 user nova-compute[72351]:       <log file='/opt/stack/data/nova/instances/f457e6ea-8d06-46c6-be2e-f650e28a3b69/console.log' append='off'/>
Aug 09 02:27:15 user nova-compute[72351]:       <target type='serial' port='0'/>
Aug 09 02:27:15 user nova-compute[72351]:       <alias name='serial0'/>
Aug 09 02:27:15 user nova-compute[72351]:     </console>
Aug 09 02:27:15 user nova-compute[72351]:     <input type='mouse' bus='ps2'>
Aug 09 02:27:15 user nova-compute[72351]:       <alias name='input0'/>
Aug 09 02:27:15 user nova-compute[72351]:     </input>
Aug 09 02:27:15 user nova-compute[72351]:     <input type='keyboard' bus='ps2'>
Aug 09 02:27:15 user nova-compute[72351]:       <alias name='input1'/>
Aug 09 02:27:15 user nova-compute[72351]:     </input>
Aug 09 02:27:15 user nova-compute[72351]:     <graphics type='vnc' port='5910' autoport='yes' listen='0.0.0.0'>
Aug 09 02:27:15 user nova-compute[72351]:       <listen type='address' address='0.0.0.0'/>
Aug 09 02:27:15 user nova-compute[72351]:     </graphics>
Aug 09 02:27:15 user nova-compute[72351]:     <audio id='1' type='none'/>
Aug 09 02:27:15 user nova-compute[72351]:     <video>
Aug 09 02:27:15 user nova-compute[72351]:       <model type='virtio' heads='1' primary='yes'/>
Aug 09 02:27:15 user nova-compute[72351]:       <alias name='video0'/>
Aug 09 02:27:15 user nova-compute[72351]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
Aug 09 02:27:15 user nova-compute[72351]:     </video>
Aug 09 02:27:15 user nova-compute[72351]:     <memballoon model='virtio'>
Aug 09 02:27:15 user nova-compute[72351]:       <stats period='10'/>
Aug 09 02:27:15 user nova-compute[72351]:       <alias name='balloon0'/>
Aug 09 02:27:15 user nova-compute[72351]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
Aug 09 02:27:15 user nova-compute[72351]:     </memballoon>
Aug 09 02:27:15 user nova-compute[72351]:     <rng model='virtio'>
Aug 09 02:27:15 user nova-compute[72351]:       <backend model='random'>/dev/urandom</backend>
Aug 09 02:27:15 user nova-compute[72351]:       <alias name='rng0'/>
Aug 09 02:27:15 user nova-compute[72351]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
Aug 09 02:27:15 user nova-compute[72351]:     </rng>
Aug 09 02:27:15 user nova-compute[72351]:   </devices>
Aug 09 02:27:15 user nova-compute[72351]:   <seclabel type='dynamic' model='apparmor' relabel='yes'>
Aug 09 02:27:15 user nova-compute[72351]:     <label>libvirt-f457e6ea-8d06-46c6-be2e-f650e28a3b69</label>
Aug 09 02:27:15 user nova-compute[72351]:     <imagelabel>libvirt-f457e6ea-8d06-46c6-be2e-f650e28a3b69</imagelabel>
Aug 09 02:27:15 user nova-compute[72351]:   </seclabel>
Aug 09 02:27:15 user nova-compute[72351]:   <seclabel type='dynamic' model='dac' relabel='yes'>
Aug 09 02:27:15 user nova-compute[72351]:     <label>+64055:+108</label>
Aug 09 02:27:15 user nova-compute[72351]:     <imagelabel>+64055:+108</imagelabel>
Aug 09 02:27:15 user nova-compute[72351]:   </seclabel>
Aug 09 02:27:15 user nova-compute[72351]: </domain>
Aug 09 02:27:15 user nova-compute[72351]:  {{(pid=72351) get_interface_by_cfg /opt/stack/nova/nova/virt/libvirt/guest.py:282}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [req-add8da7b-7146-453b-8b63-db836f77e9cd req-486a1c34-d40f-4811-8f7e-3e14fb9a9963 service nova] looking for interface given config: <interface type="ethernet"><mac address="fa:16:3e:30:6e:79"/><model type="virtio"/><mtu size="1442"/><target dev="tap9bf76200-4b"/></interface> {{(pid=72351) get_interface_by_cfg /opt/stack/nova/nova/virt/libvirt/guest.py:257}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'oTnweUhv', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:27:15 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:27:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:27:15 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:27:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:27:15 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 02:27:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [req-add8da7b-7146-453b-8b63-db836f77e9cd req-486a1c34-d40f-4811-8f7e-3e14fb9a9963 service nova] interface for config: <interface type="ethernet"><mac address="fa:16:3e:30:6e:79"/><model type="virtio"/><mtu size="1442"/><target dev="tap9bf76200-4b"/></interface>not found in domain: <domain type='kvm' id='29'>
Aug 09 02:27:15 user nova-compute[72351]:   <name>instance-00000019</name>
Aug 09 02:27:15 user nova-compute[72351]:   <uuid>f457e6ea-8d06-46c6-be2e-f650e28a3b69</uuid>
Aug 09 02:27:15 user nova-compute[72351]:   <metadata>
Aug 09 02:27:15 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1" xmlns:instance="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:27:15 user nova-compute[72351]:   <nova:package version="0.0.0"/>
Aug 09 02:27:15 user nova-compute[72351]:   <nova:name>tempest-device-tagging-server-443615500</nova:name>
Aug 09 02:27:15 user nova-compute[72351]:   <nova:creationTime>2023-08-09 02:27:14</nova:creationTime>
Aug 09 02:27:15 user nova-compute[72351]:   <nova:flavor name="m1.tiny">
Aug 09 02:27:15 user nova-compute[72351]:     <nova:memory>512</nova:memory>
Aug 09 02:27:15 user nova-compute[72351]:     <nova:disk>1</nova:disk>
Aug 09 02:27:15 user nova-compute[72351]:     <nova:swap>0</nova:swap>
Aug 09 02:27:15 user nova-compute[72351]:     <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:27:15 user nova-compute[72351]:     <nova:vcpus>1</nova:vcpus>
Aug 09 02:27:15 user nova-compute[72351]:   </nova:flavor>
Aug 09 02:27:15 user nova-compute[72351]:   <nova:owner>
Aug 09 02:27:15 user nova-compute[72351]:     <nova:user uuid="8159c9e78af04e01a1faf9b3cfc3591e">tempest-TaggedAttachmentsTest-480515778-project-member</nova:user>
Aug 09 02:27:15 user nova-compute[72351]:     <nova:project uuid="bde0634670914ae8bfbbc6b31c09d6dd">tempest-TaggedAttachmentsTest-480515778</nova:project>
Aug 09 02:27:15 user nova-compute[72351]:   </nova:owner>
Aug 09 02:27:15 user nova-compute[72351]:   <nova:root type="image" uuid="18ccda7f-31d3-4978-ab9d-4ed2f74eff94"/>
Aug 09 02:27:15 user nova-compute[72351]:   <nova:ports>
Aug 09 02:27:15 user nova-compute[72351]:     <nova:port uuid="dc244cf9-b0f6-4906-b9f3-53634b1847e3">
Aug 09 02:27:15 user nova-compute[72351]:       <nova:ip type="fixed" address="10.0.0.9" ipVersion="4"/>
Aug 09 02:27:15 user nova-compute[72351]:     </nova:port>
Aug 09 02:27:15 user nova-compute[72351]:   </nova:ports>
Aug 09 02:27:15 user nova-compute[72351]: </nova:instance>
Aug 09 02:27:15 user nova-compute[72351]:   </metadata>
Aug 09 02:27:15 user nova-compute[72351]:   <memory unit='KiB'>524288</memory>
Aug 09 02:27:15 user nova-compute[72351]:   <currentMemory unit='KiB'>524288</currentMemory>
Aug 09 02:27:15 user nova-compute[72351]:   <vcpu placement='static'>1</vcpu>
Aug 09 02:27:15 user nova-compute[72351]:   <resource>
Aug 09 02:27:15 user nova-compute[72351]:     <partition>/machine</partition>
Aug 09 02:27:15 user nova-compute[72351]:   </resource>
Aug 09 02:27:15 user nova-compute[72351]:   <sysinfo type='smbios'>
Aug 09 02:27:15 user nova-compute[72351]:     <system>
Aug 09 02:27:15 user nova-compute[72351]:       <entry name='manufacturer'>OpenStack Foundation</entry>
Aug 09 02:27:15 user nova-compute[72351]:       <entry name='product'>OpenStack Nova</entry>
Aug 09 02:27:15 user nova-compute[72351]:       <entry name='version'>0.0.0</entry>
Aug 09 02:27:15 user nova-compute[72351]:       <entry name='serial'>f457e6ea-8d06-46c6-be2e-f650e28a3b69</entry>
Aug 09 02:27:15 user nova-compute[72351]:       <entry name='uuid'>f457e6ea-8d06-46c6-be2e-f650e28a3b69</entry>
Aug 09 02:27:15 user nova-compute[72351]:       <entry name='family'>Virtual Machine</entry>
Aug 09 02:27:15 user nova-compute[72351]:     </system>
Aug 09 02:27:15 user nova-compute[72351]:   </sysinfo>
Aug 09 02:27:15 user nova-compute[72351]:   <os>
Aug 09 02:27:15 user nova-compute[72351]:     <type arch='x86_64' machine='pc-i440fx-6.2'>hvm</type>
Aug 09 02:27:15 user nova-compute[72351]:     <boot dev='hd'/>
Aug 09 02:27:15 user nova-compute[72351]:     <smbios mode='sysinfo'/>
Aug 09 02:27:15 user nova-compute[72351]:   </os>
Aug 09 02:27:15 user nova-compute[72351]:   <features>
Aug 09 02:27:15 user nova-compute[72351]:     <acpi/>
Aug 09 02:27:15 user nova-compute[72351]:     <apic/>
Aug 09 02:27:15 user nova-compute[72351]:     <vmcoreinfo state='on'/>
Aug 09 02:27:15 user nova-compute[72351]:   </features>
Aug 09 02:27:15 user nova-compute[72351]:   <cpu mode='custom' match='exact' check='full'>
Aug 09 02:27:15 user nova-compute[72351]:     <model fallback='forbid'>Nehalem</model>
Aug 09 02:27:15 user nova-compute[72351]:     <topology sockets='1' dies='1' cores='1' threads='1'/>
Aug 09 02:27:15 user nova-compute[72351]:     <feature policy='require' name='vme'/>
Aug 09 02:27:15 user nova-compute[72351]:     <feature policy='require' name='x2apic'/>
Aug 09 02:27:15 user nova-compute[72351]:     <feature policy='require' name='hypervisor'/>
Aug 09 02:27:15 user nova-compute[72351]:   </cpu>
Aug 09 02:27:15 user nova-compute[72351]:   <clock offset='utc'>
Aug 09 02:27:15 user nova-compute[72351]:     <timer name='pit' tickpolicy='delay'/>
Aug 09 02:27:15 user nova-compute[72351]:     <timer name='rtc' tickpolicy='catchup'/>
Aug 09 02:27:15 user nova-compute[72351]:     <timer name='hpet' present='no'/>
Aug 09 02:27:15 user nova-compute[72351]:   </clock>
Aug 09 02:27:15 user nova-compute[72351]:   <on_poweroff>destroy</on_poweroff>
Aug 09 02:27:15 user nova-compute[72351]:   <on_reboot>restart</on_reboot>
Aug 09 02:27:15 user nova-compute[72351]:   <on_crash>destroy</on_crash>
Aug 09 02:27:15 user nova-compute[72351]:   <devices>
Aug 09 02:27:15 user nova-compute[72351]:     <emulator>/usr/bin/qemu-system-x86_64</emulator>
Aug 09 02:27:15 user nova-compute[72351]:     <disk type='file' device='disk'>
Aug 09 02:27:15 user nova-compute[72351]:       <driver name='qemu' type='qcow2' cache='none'/>
Aug 09 02:27:15 user nova-compute[72351]:       <source file='/opt/stack/data/nova/instances/f457e6ea-8d06-46c6-be2e-f650e28a3b69/disk' index='2'/>
Aug 09 02:27:15 user nova-compute[72351]:       <backingStore type='file' index='3'>
Aug 09 02:27:15 user nova-compute[72351]:         <format type='raw'/>
Aug 09 02:27:15 user nova-compute[72351]:         <source file='/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd'/>
Aug 09 02:27:15 user nova-compute[72351]:         <backingStore/>
Aug 09 02:27:15 user nova-compute[72351]:       </backingStore>
Aug 09 02:27:15 user nova-compute[72351]:       <target dev='vda' bus='virtio'/>
Aug 09 02:27:15 user nova-compute[72351]:       <alias name='virtio-disk0'/>
Aug 09 02:27:15 user nova-compute[72351]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
Aug 09 02:27:15 user nova-compute[72351]:     </disk>
Aug 09 02:27:15 user nova-compute[72351]:     <disk type='file' device='cdrom'>
Aug 09 02:27:15 user nova-compute[72351]:       <driver name='qemu' type='raw' cache='none'/>
Aug 09 02:27:15 user nova-compute[72351]:       <source file='/opt/stack/data/nova/instances/f457e6ea-8d06-46c6-be2e-f650e28a3b69/disk.config' index='1'/>
Aug 09 02:27:15 user nova-compute[72351]:       <backingStore/>
Aug 09 02:27:15 user nova-compute[72351]:       <target dev='hda' bus='ide'/>
Aug 09 02:27:15 user nova-compute[72351]:       <readonly/>
Aug 09 02:27:15 user nova-compute[72351]:       <alias name='ide0-0-0'/>
Aug 09 02:27:15 user nova-compute[72351]:       <address type='drive' controller='0' bus='0' target='0' unit='0'/>
Aug 09 02:27:15 user nova-compute[72351]:     </disk>
Aug 09 02:27:15 user nova-compute[72351]:     <controller type='usb' index='0' model='none'>
Aug 09 02:27:15 user nova-compute[72351]:       <alias name='usb'/>
Aug 09 02:27:15 user nova-compute[72351]:     </controller>
Aug 09 02:27:15 user nova-compute[72351]:     <controller type='pci' index='0' model='pci-root'>
Aug 09 02:27:15 user nova-compute[72351]:       <alias name='pci.0'/>
Aug 09 02:27:15 user nova-compute[72351]:     </controller>
Aug 09 02:27:15 user nova-compute[72351]:     <controller type='ide' index='0'>
Aug 09 02:27:15 user nova-compute[72351]:       <alias name='ide'/>
Aug 09 02:27:15 user nova-compute[72351]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
Aug 09 02:27:15 user nova-compute[72351]:     </controller>
Aug 09 02:27:15 user nova-compute[72351]:     <interface type='ethernet'>
Aug 09 02:27:15 user nova-compute[72351]:       <mac address='fa:16:3e:0e:77:88'/>
Aug 09 02:27:15 user nova-compute[72351]:       <target dev='tapdc244cf9-b0'/>
Aug 09 02:27:15 user nova-compute[72351]:       <model type='virtio'/>
Aug 09 02:27:15 user nova-compute[72351]:       <mtu size='1442'/>
Aug 09 02:27:15 user nova-compute[72351]:       <alias name='net0'/>
Aug 09 02:27:15 user nova-compute[72351]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
Aug 09 02:27:15 user nova-compute[72351]:     </interface>
Aug 09 02:27:15 user nova-compute[72351]:     <serial type='pty'>
Aug 09 02:27:15 user nova-compute[72351]:       <source path='/dev/pts/10'/>
Aug 09 02:27:15 user nova-compute[72351]:       <log file='/opt/stack/data/nova/instances/f457e6ea-8d06-46c6-be2e-f650e28a3b69/console.log' append='off'/>
Aug 09 02:27:15 user nova-compute[72351]:       <target type='isa-serial' port='0'>
Aug 09 02:27:15 user nova-compute[72351]:         <model name='isa-serial'/>
Aug 09 02:27:15 user nova-compute[72351]:       </target>
Aug 09 02:27:15 user nova-compute[72351]:       <alias name='serial0'/>
Aug 09 02:27:15 user nova-compute[72351]:     </serial>
Aug 09 02:27:15 user nova-compute[72351]:     <console type='pty' tty='/dev/pts/10'>
Aug 09 02:27:15 user nova-compute[72351]:       <source path='/dev/pts/10'/>
Aug 09 02:27:15 user nova-compute[72351]:       <log file='/opt/stack/data/nova/instances/f457e6ea-8d06-46c6-be2e-f650e28a3b69/console.log' append='off'/>
Aug 09 02:27:15 user nova-compute[72351]:       <target type='serial' port='0'/>
Aug 09 02:27:15 user nova-compute[72351]:       <alias name='serial0'/>
Aug 09 02:27:15 user nova-compute[72351]:     </console>
Aug 09 02:27:15 user nova-compute[72351]:     <input type='mouse' bus='ps2'>
Aug 09 02:27:15 user nova-compute[72351]:       <alias name='input0'/>
Aug 09 02:27:15 user nova-compute[72351]:     </input>
Aug 09 02:27:15 user nova-compute[72351]:     <input type='keyboard' bus='ps2'>
Aug 09 02:27:15 user nova-compute[72351]:       <alias name='input1'/>
Aug 09 02:27:15 user nova-compute[72351]:     </input>
Aug 09 02:27:15 user nova-compute[72351]:     <graphics type='vnc' port='5910' autoport='yes' listen='0.0.0.0'>
Aug 09 02:27:15 user nova-compute[72351]:       <listen type='address' address='0.0.0.0'/>
Aug 09 02:27:15 user nova-compute[72351]:     </graphics>
Aug 09 02:27:15 user nova-compute[72351]:     <audio id='1' type='none'/>
Aug 09 02:27:15 user nova-compute[72351]:     <video>
Aug 09 02:27:15 user nova-compute[72351]:       <model type='virtio' heads='1' primary='yes'/>
Aug 09 02:27:15 user nova-compute[72351]:       <alias name='video0'/>
Aug 09 02:27:15 user nova-compute[72351]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
Aug 09 02:27:15 user nova-compute[72351]:     </video>
Aug 09 02:27:15 user nova-compute[72351]:     <memballoon model='virtio'>
Aug 09 02:27:15 user nova-compute[72351]:       <stats period='10'/>
Aug 09 02:27:15 user nova-compute[72351]:       <alias name='balloon0'/>
Aug 09 02:27:15 user nova-compute[72351]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
Aug 09 02:27:15 user nova-compute[72351]:     </memballoon>
Aug 09 02:27:15 user nova-compute[72351]:     <rng model='virtio'>
Aug 09 02:27:15 user nova-compute[72351]:       <backend model='random'>/dev/urandom</backend>
Aug 09 02:27:15 user nova-compute[72351]:       <alias name='rng0'/>
Aug 09 02:27:15 user nova-compute[72351]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
Aug 09 02:27:15 user nova-compute[72351]:     </rng>
Aug 09 02:27:15 user nova-compute[72351]:   </devices>
Aug 09 02:27:15 user nova-compute[72351]:   <seclabel type='dynamic' model='apparmor' relabel='yes'>
Aug 09 02:27:15 user nova-compute[72351]:     <label>libvirt-f457e6ea-8d06-46c6-be2e-f650e28a3b69</label>
Aug 09 02:27:15 user nova-compute[72351]:     <imagelabel>libvirt-f457e6ea-8d06-46c6-be2e-f650e28a3b69</imagelabel>
Aug 09 02:27:15 user nova-compute[72351]:   </seclabel>
Aug 09 02:27:15 user nova-compute[72351]:   <seclabel type='dynamic' model='dac' relabel='yes'>
Aug 09 02:27:15 user nova-compute[72351]:     <label>+64055:+108</label>
Aug 09 02:27:15 user nova-compute[72351]:     <imagelabel>+64055:+108</imagelabel>
Aug 09 02:27:15 user nova-compute[72351]:   </seclabel>
Aug 09 02:27:15 user nova-compute[72351]: </domain>
Aug 09 02:27:15 user nova-compute[72351]:  {{(pid=72351) get_interface_by_cfg /opt/stack/nova/nova/virt/libvirt/guest.py:282}}
Aug 09 02:27:15 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [req-add8da7b-7146-453b-8b63-db836f77e9cd req-486a1c34-d40f-4811-8f7e-3e14fb9a9963 service nova] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Detaching interface fa:16:3e:30:6e:79 failed because the device is no longer found on the guest.: nova.exception.DeviceNotFound: Device 'tap9bf76200-4b' not found.
Aug 09 02:27:15 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [req-add8da7b-7146-453b-8b63-db836f77e9cd req-486a1c34-d40f-4811-8f7e-3e14fb9a9963 service nova] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-08-09T02:24:58Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-443615500',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-443615500',id=25,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBK8nrHk7zghGHoCUEI3vExNw2oXJc2YwDe7QrMxgC2DNUP/PkVzpLwy9DWnyTLCCR4ddGxe1t7xIbXts5zbiafpc3PmU6by4kyzLopbWZ1BYcJyQcmguVSz4dz8HMbCR9g==',key_name='tempest-keypair-1705746290',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:25:05Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata=<?>,migration_context=None,new_flavor=None,node='user',numa_topology=<?>,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='bde0634670914ae8bfbbc6b31c09d6dd',ramdisk_id='',reservation_id='r-5xxup5ze',resources=<?>,root_device_name='/dev/vda',root_gb=1,security_groups=<?>,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedAttachmentsTest-480515778',owner_user_name='tempest-TaggedAttachmentsTest-480515778-project-member'},tags=<?>,task_state=None,terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:25:06Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='8159c9e78af04e01a1faf9b3cfc3591e',uuid=f457e6ea-8d06-46c6-be2e-f650e28a3b69,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "9bf76200-4b93-40dc-bd6f-0200cec7933a", "address": "fa:16:3e:30:6e:79", "network": {"id": "ea588e91-dd01-4f9c-ad88-30f0848e6839", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-596948871", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.51", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "bde0634670914ae8bfbbc6b31c09d6dd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9bf76200-4b", "ovs_interfaceid": "9bf76200-4b93-40dc-bd6f-0200cec7933a", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG nova.network.os_vif_util [req-add8da7b-7146-453b-8b63-db836f77e9cd req-486a1c34-d40f-4811-8f7e-3e14fb9a9963 service nova] Converting VIF {"id": "9bf76200-4b93-40dc-bd6f-0200cec7933a", "address": "fa:16:3e:30:6e:79", "network": {"id": "ea588e91-dd01-4f9c-ad88-30f0848e6839", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-596948871", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.51", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "bde0634670914ae8bfbbc6b31c09d6dd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9bf76200-4b", "ovs_interfaceid": "9bf76200-4b93-40dc-bd6f-0200cec7933a", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG nova.network.os_vif_util [req-add8da7b-7146-453b-8b63-db836f77e9cd req-486a1c34-d40f-4811-8f7e-3e14fb9a9963 service nova] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:30:6e:79,bridge_name='br-int',has_traffic_filtering=True,id=9bf76200-4b93-40dc-bd6f-0200cec7933a,network=Network(ea588e91-dd01-4f9c-ad88-30f0848e6839),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap9bf76200-4b') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG os_vif [req-add8da7b-7146-453b-8b63-db836f77e9cd req-486a1c34-d40f-4811-8f7e-3e14fb9a9963 service nova] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:30:6e:79,bridge_name='br-int',has_traffic_filtering=True,id=9bf76200-4b93-40dc-bd6f-0200cec7933a,network=Network(ea588e91-dd01-4f9c-ad88-30f0848e6839),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap9bf76200-4b') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap9bf76200-4b, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260" returned: 21 in 0.024s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d8a55ee9-2bfd-443b-8322-efe0612bf4f8]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Aug 09 02:27:15 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 --interface default --op new {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:15 user nova-compute[72351]: INFO os_vif [req-add8da7b-7146-453b-8b63-db836f77e9cd req-486a1c34-d40f-4811-8f7e-3e14fb9a9963 service nova] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:30:6e:79,bridge_name='br-int',has_traffic_filtering=True,id=9bf76200-4b93-40dc-bd6f-0200cec7933a,network=Network(ea588e91-dd01-4f9c-ad88-30f0848e6839),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap9bf76200-4b')
Aug 09 02:27:15 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [req-add8da7b-7146-453b-8b63-db836f77e9cd req-486a1c34-d40f-4811-8f7e-3e14fb9a9963 service nova] set metadata xml: <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:27:15 user nova-compute[72351]:   <nova:package version="0.0.0"/>
Aug 09 02:27:15 user nova-compute[72351]:   <nova:name>tempest-device-tagging-server-443615500</nova:name>
Aug 09 02:27:15 user nova-compute[72351]:   <nova:creationTime>2023-08-09 02:27:15</nova:creationTime>
Aug 09 02:27:15 user nova-compute[72351]:   <nova:flavor name="m1.tiny">
Aug 09 02:27:15 user nova-compute[72351]:     <nova:memory>512</nova:memory>
Aug 09 02:27:15 user nova-compute[72351]:     <nova:disk>1</nova:disk>
Aug 09 02:27:15 user nova-compute[72351]:     <nova:swap>0</nova:swap>
Aug 09 02:27:15 user nova-compute[72351]:     <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:27:15 user nova-compute[72351]:     <nova:vcpus>1</nova:vcpus>
Aug 09 02:27:15 user nova-compute[72351]:   </nova:flavor>
Aug 09 02:27:15 user nova-compute[72351]:   <nova:owner>
Aug 09 02:27:15 user nova-compute[72351]:     <nova:user uuid="8159c9e78af04e01a1faf9b3cfc3591e">tempest-TaggedAttachmentsTest-480515778-project-member</nova:user>
Aug 09 02:27:15 user nova-compute[72351]:     <nova:project uuid="bde0634670914ae8bfbbc6b31c09d6dd">tempest-TaggedAttachmentsTest-480515778</nova:project>
Aug 09 02:27:15 user nova-compute[72351]:   </nova:owner>
Aug 09 02:27:15 user nova-compute[72351]:   <nova:root type="image" uuid="18ccda7f-31d3-4978-ab9d-4ed2f74eff94"/>
Aug 09 02:27:15 user nova-compute[72351]:   <nova:ports>
Aug 09 02:27:15 user nova-compute[72351]:     <nova:port uuid="dc244cf9-b0f6-4906-b9f3-53634b1847e3">
Aug 09 02:27:15 user nova-compute[72351]:       <nova:ip type="fixed" address="10.0.0.9" ipVersion="4"/>
Aug 09 02:27:15 user nova-compute[72351]:     </nova:port>
Aug 09 02:27:15 user nova-compute[72351]:   </nova:ports>
Aug 09 02:27:15 user nova-compute[72351]: </nova:instance>
Aug 09 02:27:15 user nova-compute[72351]:  {{(pid=72351) set_metadata /opt/stack/nova/nova/virt/libvirt/guest.py:359}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.033s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[079e0b58-f7d1-4809-b2de-d9c5b8d13709]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd] added\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd] added
Aug 09 02:27:15 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ed1a0208-a88e-4c73-b420-510f3e00e7d5]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.033s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[959bd008-f488-4412-8cc0-2875a35e0938]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 --op update -n node.session.auth.username -v oTnweUhv {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 --op update -n node.session.auth.username -v oTnweUhv" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a1645da7-4c37-47f5-924c-dd2032e20492]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'oTnweUhv'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[edaa600e-7b86-4536-9c64-55793717dc5f]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[826b6ede-c4b2-4446-b389-c88135efd2e2]: (4, ('tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)\ntcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)\ntcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)
Aug 09 02:27:15 user nova-compute[72351]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)
Aug 09 02:27:15 user nova-compute[72351]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:27:15 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] iscsi session list stdout=tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)
Aug 09 02:27:15 user nova-compute[72351]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)
Aug 09 02:27:15 user nova-compute[72351]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:27:15 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 --login {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 --login" returned: 0 in 0.064s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ba79b585-2694-4222-a3b7-e7bb971e5c28]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd, portal: 172.16.0.220,3260]
Aug 09 02:27:15 user nova-compute[72351]: Login to [iface: default, target: iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd, portal: 172.16.0.220,3260] successful.
Aug 09 02:27:15 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ec9d509e-ad10-4958-8bc5-8e84ba591048]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[7d8b58e9-a678-484a-9c82-38733e8d75d7]: (4, ('tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)\ntcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)\ntcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)
Aug 09 02:27:15 user nova-compute[72351]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)
Aug 09 02:27:15 user nova-compute[72351]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:27:15 user nova-compute[72351]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:27:15 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] iscsi session list stdout=tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)
Aug 09 02:27:15 user nova-compute[72351]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)
Aug 09 02:27:15 user nova-compute[72351]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:27:15 user nova-compute[72351]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:27:15 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.346s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] HCTL ('33', '-', '-', 0) found on session 28 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.043s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[38e87ca1-921e-48e1-a59b-4a0850a03521]: (4, ('- - 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Searching for a device in session 28 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Connected to sdb using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'oTnweUhv', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 02:27:15 user nova-compute[72351]: INFO nova.network.neutron [None req-573693b1-fef0-4819-9ee7-6be13c801b1c tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Port 9bf76200-4b93-40dc-bd6f-0200cec7933a from network info_cache is no longer associated with instance in Neutron. Removing from network info_cache.
Aug 09 02:27:15 user nova-compute[72351]: DEBUG nova.network.neutron [None req-573693b1-fef0-4819-9ee7-6be13c801b1c tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Updating instance_info_cache with network_info: [{"id": "dc244cf9-b0f6-4906-b9f3-53634b1847e3", "address": "fa:16:3e:0e:77:88", "network": {"id": "3fdd9499-7552-4ecc-a897-ed4c58be9519", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-969934765-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.20", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bde0634670914ae8bfbbc6b31c09d6dd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdc244cf9-b0", "ovs_interfaceid": "dc244cf9-b0f6-4906-b9f3-53634b1847e3", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-573693b1-fef0-4819-9ee7-6be13c801b1c tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Releasing lock "refresh_cache-f457e6ea-8d06-46c6-be2e-f650e28a3b69" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:27:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-573693b1-fef0-4819-9ee7-6be13c801b1c tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Lock "interface-f457e6ea-8d06-46c6-be2e-f650e28a3b69-9bf76200-4b93-40dc-bd6f-0200cec7933a" "released" by "nova.compute.manager.ComputeManager.detach_interface.<locals>.do_detach_interface" :: held 1.689s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:27:16 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.409s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:27:16 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] <== connect_volume: return (1410ms) {'type': 'block', 'scsi_wwn': '23062643536626331', 'path': '/dev/sdb'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:27:16 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23062643536626331', 'path': '/dev/sdb'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 02:27:16 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:21:09Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestOtherB-server-634605118',display_name='tempest-ServerActionsTestOtherB-server-634605118',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serveractionstestotherb-server-634605118',id=15,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLdtpWQKmHav2tGxZemUVMEaT9TXyD7Duhy5QV84PPI+5KrgLVkbxVzWgryocScOebEprbcJ80bjBPnYhLtQJYSe5MXXz6GOQHLb+tJCZajhrDoMbn41/bPAnB5B3oFwCw==',key_name='tempest-keypair-644333596',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:25:31Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=MigrationContext,new_flavor=Flavor(7),node='user',numa_topology=None,old_flavor=Flavor(6),os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='fc90e94bbcb54fb6b86a9ac45edb596a',ramdisk_id='',reservation_id='r-ozz8gdfj',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',old_vm_state='active',owner_project_name='tempest-ServerActionsTestOtherB-1726991791',owner_user_name='tempest-ServerActionsTestOtherB-1726991791-project-member'},tags=<?>,task_state='resize_reverting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:27:01Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='19668b0c790d49dd830c39e3c2be29bf',uuid=75bbbead-8628-448b-9484-19bf4ab922ff,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='resized') vif={"id": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "address": "fa:16:3e:8b:b5:2e", "network": {"id": "8231f2c2-237e-41c6-8892-720b950c1453", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherB-1917380761-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.30", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fc90e94bbcb54fb6b86a9ac45edb596a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbe4ec483-85", "ovs_interfaceid": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:27:16 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Converting VIF {"id": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "address": "fa:16:3e:8b:b5:2e", "network": {"id": "8231f2c2-237e-41c6-8892-720b950c1453", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherB-1917380761-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.30", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fc90e94bbcb54fb6b86a9ac45edb596a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbe4ec483-85", "ovs_interfaceid": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:27:16 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:8b:b5:2e,bridge_name='br-int',has_traffic_filtering=True,id=be4ec483-8532-45f7-bb9f-0b50952ce94b,network=Network(8231f2c2-237e-41c6-8892-720b950c1453),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapbe4ec483-85') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:27:16 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:27:16 user nova-compute[72351]:   <uuid>75bbbead-8628-448b-9484-19bf4ab922ff</uuid>
Aug 09 02:27:16 user nova-compute[72351]:   <name>instance-0000000f</name>
Aug 09 02:27:16 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:27:16 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:27:16 user nova-compute[72351]:   <metadata>
Aug 09 02:27:16 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:27:16 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:27:16 user nova-compute[72351]:       <nova:name>tempest-ServerActionsTestOtherB-server-634605118</nova:name>
Aug 09 02:27:16 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:27:15</nova:creationTime>
Aug 09 02:27:16 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:27:16 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:27:16 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:27:16 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:27:16 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:27:16 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:27:16 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:27:16 user nova-compute[72351]:       <nova:owner>
Aug 09 02:27:16 user nova-compute[72351]:         <nova:user uuid="19668b0c790d49dd830c39e3c2be29bf">tempest-ServerActionsTestOtherB-1726991791-project-member</nova:user>
Aug 09 02:27:16 user nova-compute[72351]:         <nova:project uuid="fc90e94bbcb54fb6b86a9ac45edb596a">tempest-ServerActionsTestOtherB-1726991791</nova:project>
Aug 09 02:27:16 user nova-compute[72351]:       </nova:owner>
Aug 09 02:27:16 user nova-compute[72351]:       <nova:root type="image" uuid="18ccda7f-31d3-4978-ab9d-4ed2f74eff94"/>
Aug 09 02:27:16 user nova-compute[72351]:       <nova:ports>
Aug 09 02:27:16 user nova-compute[72351]:         <nova:port uuid="be4ec483-8532-45f7-bb9f-0b50952ce94b">
Aug 09 02:27:16 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.14" ipVersion="4"/>
Aug 09 02:27:16 user nova-compute[72351]:         </nova:port>
Aug 09 02:27:16 user nova-compute[72351]:       </nova:ports>
Aug 09 02:27:16 user nova-compute[72351]:     </nova:instance>
Aug 09 02:27:16 user nova-compute[72351]:   </metadata>
Aug 09 02:27:16 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:27:16 user nova-compute[72351]:     <system>
Aug 09 02:27:16 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:27:16 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:27:16 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:27:16 user nova-compute[72351]:       <entry name="serial">75bbbead-8628-448b-9484-19bf4ab922ff</entry>
Aug 09 02:27:16 user nova-compute[72351]:       <entry name="uuid">75bbbead-8628-448b-9484-19bf4ab922ff</entry>
Aug 09 02:27:16 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:27:16 user nova-compute[72351]:     </system>
Aug 09 02:27:16 user nova-compute[72351]:   </sysinfo>
Aug 09 02:27:16 user nova-compute[72351]:   <os>
Aug 09 02:27:16 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:27:16 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:27:16 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:27:16 user nova-compute[72351]:   </os>
Aug 09 02:27:16 user nova-compute[72351]:   <features>
Aug 09 02:27:16 user nova-compute[72351]:     <acpi/>
Aug 09 02:27:16 user nova-compute[72351]:     <apic/>
Aug 09 02:27:16 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:27:16 user nova-compute[72351]:   </features>
Aug 09 02:27:16 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:27:16 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:27:16 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:27:16 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:27:16 user nova-compute[72351]:   </clock>
Aug 09 02:27:16 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:27:16 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:27:16 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:27:16 user nova-compute[72351]:   </cpu>
Aug 09 02:27:16 user nova-compute[72351]:   <devices>
Aug 09 02:27:16 user nova-compute[72351]:     <disk type="file" device="disk">
Aug 09 02:27:16 user nova-compute[72351]:       <driver name="qemu" type="qcow2" cache="none"/>
Aug 09 02:27:16 user nova-compute[72351]:       <source file="/opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff/disk"/>
Aug 09 02:27:16 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:27:16 user nova-compute[72351]:     </disk>
Aug 09 02:27:16 user nova-compute[72351]:     <disk type="block" device="disk">
Aug 09 02:27:16 user nova-compute[72351]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:27:16 user nova-compute[72351]:       <source dev="/dev/sdb"/>
Aug 09 02:27:16 user nova-compute[72351]:       <target dev="vdb" bus="virtio"/>
Aug 09 02:27:16 user nova-compute[72351]:       <serial>2db51b4f-d032-4367-bb6f-8964563eb8fd</serial>
Aug 09 02:27:16 user nova-compute[72351]:     </disk>
Aug 09 02:27:16 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:27:16 user nova-compute[72351]:       <mac address="fa:16:3e:8b:b5:2e"/>
Aug 09 02:27:16 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:27:16 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:27:16 user nova-compute[72351]:       <target dev="tapbe4ec483-85"/>
Aug 09 02:27:16 user nova-compute[72351]:     </interface>
Aug 09 02:27:16 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:27:16 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff/console.log" append="off"/>
Aug 09 02:27:16 user nova-compute[72351]:     </serial>
Aug 09 02:27:16 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:27:16 user nova-compute[72351]:     <video>
Aug 09 02:27:16 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:27:16 user nova-compute[72351]:     </video>
Aug 09 02:27:16 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:27:16 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:27:16 user nova-compute[72351]:     </rng>
Aug 09 02:27:16 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:27:16 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:27:16 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:27:16 user nova-compute[72351]:     </memballoon>
Aug 09 02:27:16 user nova-compute[72351]:   </devices>
Aug 09 02:27:16 user nova-compute[72351]: </domain>
Aug 09 02:27:16 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:27:16 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:21:09Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestOtherB-server-634605118',display_name='tempest-ServerActionsTestOtherB-server-634605118',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serveractionstestotherb-server-634605118',id=15,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLdtpWQKmHav2tGxZemUVMEaT9TXyD7Duhy5QV84PPI+5KrgLVkbxVzWgryocScOebEprbcJ80bjBPnYhLtQJYSe5MXXz6GOQHLb+tJCZajhrDoMbn41/bPAnB5B3oFwCw==',key_name='tempest-keypair-644333596',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:25:31Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=MigrationContext,new_flavor=Flavor(7),node='user',numa_topology=None,old_flavor=Flavor(6),os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='fc90e94bbcb54fb6b86a9ac45edb596a',ramdisk_id='',reservation_id='r-ozz8gdfj',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',old_vm_state='active',owner_project_name='tempest-ServerActionsTestOtherB-1726991791',owner_user_name='tempest-ServerActionsTestOtherB-1726991791-project-member'},tags=<?>,task_state='resize_reverting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:27:01Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='19668b0c790d49dd830c39e3c2be29bf',uuid=75bbbead-8628-448b-9484-19bf4ab922ff,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='resized') vif={"id": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "address": "fa:16:3e:8b:b5:2e", "network": {"id": "8231f2c2-237e-41c6-8892-720b950c1453", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherB-1917380761-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.30", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fc90e94bbcb54fb6b86a9ac45edb596a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbe4ec483-85", "ovs_interfaceid": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:27:16 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Converting VIF {"id": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "address": "fa:16:3e:8b:b5:2e", "network": {"id": "8231f2c2-237e-41c6-8892-720b950c1453", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherB-1917380761-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.30", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fc90e94bbcb54fb6b86a9ac45edb596a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbe4ec483-85", "ovs_interfaceid": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:27:16 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:8b:b5:2e,bridge_name='br-int',has_traffic_filtering=True,id=be4ec483-8532-45f7-bb9f-0b50952ce94b,network=Network(8231f2c2-237e-41c6-8892-720b950c1453),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapbe4ec483-85') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:27:16 user nova-compute[72351]: DEBUG os_vif [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:8b:b5:2e,bridge_name='br-int',has_traffic_filtering=True,id=be4ec483-8532-45f7-bb9f-0b50952ce94b,network=Network(8231f2c2-237e-41c6-8892-720b950c1453),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapbe4ec483-85') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:27:16 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:16 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:27:16 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:27:16 user nova-compute[72351]: DEBUG nova.compute.manager [req-61daf558-df2f-4455-8099-925ed38b3b90 req-01ea75fe-69f9-4657-8418-100604376563 service nova] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Received event network-vif-plugged-9bf76200-4b93-40dc-bd6f-0200cec7933a {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:27:16 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-61daf558-df2f-4455-8099-925ed38b3b90 req-01ea75fe-69f9-4657-8418-100604376563 service nova] Acquiring lock "f457e6ea-8d06-46c6-be2e-f650e28a3b69-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:27:16 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-61daf558-df2f-4455-8099-925ed38b3b90 req-01ea75fe-69f9-4657-8418-100604376563 service nova] Lock "f457e6ea-8d06-46c6-be2e-f650e28a3b69-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:27:16 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-61daf558-df2f-4455-8099-925ed38b3b90 req-01ea75fe-69f9-4657-8418-100604376563 service nova] Lock "f457e6ea-8d06-46c6-be2e-f650e28a3b69-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:27:16 user nova-compute[72351]: DEBUG nova.compute.manager [req-61daf558-df2f-4455-8099-925ed38b3b90 req-01ea75fe-69f9-4657-8418-100604376563 service nova] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] No waiting events found dispatching network-vif-plugged-9bf76200-4b93-40dc-bd6f-0200cec7933a {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:27:16 user nova-compute[72351]: WARNING nova.compute.manager [req-61daf558-df2f-4455-8099-925ed38b3b90 req-01ea75fe-69f9-4657-8418-100604376563 service nova] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Received unexpected event network-vif-plugged-9bf76200-4b93-40dc-bd6f-0200cec7933a for instance with vm_state active and task_state None.
Aug 09 02:27:16 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:16 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapbe4ec483-85, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:27:16 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapbe4ec483-85, col_values=(('external_ids', {'iface-id': 'be4ec483-8532-45f7-bb9f-0b50952ce94b', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:8b:b5:2e', 'vm-uuid': '75bbbead-8628-448b-9484-19bf4ab922ff'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:27:16 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:16 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:27:16 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:16 user nova-compute[72351]: INFO os_vif [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:8b:b5:2e,bridge_name='br-int',has_traffic_filtering=True,id=be4ec483-8532-45f7-bb9f-0b50952ce94b,network=Network(8231f2c2-237e-41c6-8892-720b950c1453),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapbe4ec483-85')
Aug 09 02:27:17 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691548022.5911279, 75bbbead-8628-448b-9484-19bf4ab922ff => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:27:17 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] VM Stopped (Lifecycle Event)
Aug 09 02:27:17 user nova-compute[72351]: DEBUG nova.compute.manager [None req-05b11341-1c52-4581-9bee-0d8d4fc1b52c None None] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:27:17 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:18 user nova-compute[72351]: DEBUG nova.compute.manager [req-1c223bab-da2d-405e-a969-cc4ff9461766 req-8b69ca81-58a3-491c-8a1c-2e1fb7fe75ee service nova] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Received event network-vif-plugged-be4ec483-8532-45f7-bb9f-0b50952ce94b {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:27:18 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-1c223bab-da2d-405e-a969-cc4ff9461766 req-8b69ca81-58a3-491c-8a1c-2e1fb7fe75ee service nova] Acquiring lock "75bbbead-8628-448b-9484-19bf4ab922ff-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:27:18 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-1c223bab-da2d-405e-a969-cc4ff9461766 req-8b69ca81-58a3-491c-8a1c-2e1fb7fe75ee service nova] Lock "75bbbead-8628-448b-9484-19bf4ab922ff-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:27:18 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-1c223bab-da2d-405e-a969-cc4ff9461766 req-8b69ca81-58a3-491c-8a1c-2e1fb7fe75ee service nova] Lock "75bbbead-8628-448b-9484-19bf4ab922ff-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:27:18 user nova-compute[72351]: DEBUG nova.compute.manager [req-1c223bab-da2d-405e-a969-cc4ff9461766 req-8b69ca81-58a3-491c-8a1c-2e1fb7fe75ee service nova] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] No waiting events found dispatching network-vif-plugged-be4ec483-8532-45f7-bb9f-0b50952ce94b {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:27:18 user nova-compute[72351]: WARNING nova.compute.manager [req-1c223bab-da2d-405e-a969-cc4ff9461766 req-8b69ca81-58a3-491c-8a1c-2e1fb7fe75ee service nova] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Received unexpected event network-vif-plugged-be4ec483-8532-45f7-bb9f-0b50952ce94b for instance with vm_state resized and task_state resize_reverting.
Aug 09 02:27:18 user nova-compute[72351]: DEBUG nova.compute.manager [req-1c223bab-da2d-405e-a969-cc4ff9461766 req-8b69ca81-58a3-491c-8a1c-2e1fb7fe75ee service nova] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Received event network-vif-plugged-be4ec483-8532-45f7-bb9f-0b50952ce94b {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:27:18 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-1c223bab-da2d-405e-a969-cc4ff9461766 req-8b69ca81-58a3-491c-8a1c-2e1fb7fe75ee service nova] Acquiring lock "75bbbead-8628-448b-9484-19bf4ab922ff-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:27:18 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-1c223bab-da2d-405e-a969-cc4ff9461766 req-8b69ca81-58a3-491c-8a1c-2e1fb7fe75ee service nova] Lock "75bbbead-8628-448b-9484-19bf4ab922ff-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:27:18 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-1c223bab-da2d-405e-a969-cc4ff9461766 req-8b69ca81-58a3-491c-8a1c-2e1fb7fe75ee service nova] Lock "75bbbead-8628-448b-9484-19bf4ab922ff-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:27:18 user nova-compute[72351]: DEBUG nova.compute.manager [req-1c223bab-da2d-405e-a969-cc4ff9461766 req-8b69ca81-58a3-491c-8a1c-2e1fb7fe75ee service nova] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] No waiting events found dispatching network-vif-plugged-be4ec483-8532-45f7-bb9f-0b50952ce94b {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:27:18 user nova-compute[72351]: WARNING nova.compute.manager [req-1c223bab-da2d-405e-a969-cc4ff9461766 req-8b69ca81-58a3-491c-8a1c-2e1fb7fe75ee service nova] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Received unexpected event network-vif-plugged-be4ec483-8532-45f7-bb9f-0b50952ce94b for instance with vm_state resized and task_state resize_reverting.
Aug 09 02:27:19 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Acquiring lock "1b90316d-6638-4603-bb09-38bf0ed795e3" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:27:19 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Lock "1b90316d-6638-4603-bb09-38bf0ed795e3" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:27:19 user nova-compute[72351]: DEBUG nova.compute.manager [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] Starting instance... {{(pid=72351) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2406}}
Aug 09 02:27:19 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:27:19 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:27:19 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72351) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Aug 09 02:27:19 user nova-compute[72351]: INFO nova.compute.claims [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] Claim successful on node user
Aug 09 02:27:20 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548040.2019925, 75bbbead-8628-448b-9484-19bf4ab922ff => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:27:20 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] VM Resumed (Lifecycle Event)
Aug 09 02:27:20 user nova-compute[72351]: DEBUG nova.compute.manager [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:27:20 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Instance running successfully.
Aug 09 02:27:20 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] finish_revert_migration finished successfully. {{(pid=72351) finish_revert_migration /opt/stack/nova/nova/virt/libvirt/driver.py:11863}}
Aug 09 02:27:20 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:27:20 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:27:20 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: resized, current task_state: resize_reverting, current DB power_state: 1, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:27:20 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:27:20 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] During sync_power_state the instance has a pending task (resize_reverting). Skip.
Aug 09 02:27:20 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548040.20825, 75bbbead-8628-448b-9484-19bf4ab922ff => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:27:20 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] VM Started (Lifecycle Event)
Aug 09 02:27:20 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:27:20 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Synchronizing instance power state after lifecycle event "Started"; current vm_state: resized, current task_state: resize_reverting, current DB power_state: 1, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:27:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.492s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:27:20 user nova-compute[72351]: DEBUG nova.compute.manager [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] Start building networks asynchronously for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2803}}
Aug 09 02:27:20 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] During sync_power_state the instance has a pending task (resize_reverting). Skip.
Aug 09 02:27:20 user nova-compute[72351]: DEBUG nova.compute.manager [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] Allocating IP information in the background. {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1956}}
Aug 09 02:27:20 user nova-compute[72351]: DEBUG nova.network.neutron [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] allocate_for_instance() {{(pid=72351) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Aug 09 02:27:20 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Aug 09 02:27:20 user nova-compute[72351]: DEBUG nova.compute.manager [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] Start building block device mappings for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2838}}
Aug 09 02:27:20 user nova-compute[72351]: INFO nova.compute.manager [None req-210eb2cb-07d7-4643-840d-84d6bd96712c tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Updating instance to original state: 'active'
Aug 09 02:27:20 user nova-compute[72351]: DEBUG nova.policy [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'ebc62bdcafdd4131be9abb7fe4ed2422', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eebc3a715ae04d48918b6c68b135935b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72351) authorize /opt/stack/nova/nova/policy.py:203}}
Aug 09 02:27:20 user nova-compute[72351]: DEBUG nova.compute.manager [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] Start spawning the instance on the hypervisor. {{(pid=72351) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2612}}
Aug 09 02:27:20 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] Creating instance directory {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4717}}
Aug 09 02:27:20 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] Creating image(s)
Aug 09 02:27:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Acquiring lock "/opt/stack/data/nova/instances/1b90316d-6638-4603-bb09-38bf0ed795e3/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:27:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Lock "/opt/stack/data/nova/instances/1b90316d-6638-4603-bb09-38bf0ed795e3/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:27:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Lock "/opt/stack/data/nova/instances/1b90316d-6638-4603-bb09-38bf0ed795e3/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.005s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:27:20 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:20 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.149s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Acquiring lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:27:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:27:20 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:20 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.149s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:20 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/1b90316d-6638-4603-bb09-38bf0ed795e3/disk 1073741824 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:20 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/1b90316d-6638-4603-bb09-38bf0ed795e3/disk 1073741824" returned: 0 in 0.054s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.212s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:27:20 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.164s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:21 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Checking if we can resize image /opt/stack/data/nova/instances/1b90316d-6638-4603-bb09-38bf0ed795e3/disk. size=1073741824 {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Aug 09 02:27:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1b90316d-6638-4603-bb09-38bf0ed795e3/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:21 user nova-compute[72351]: DEBUG nova.network.neutron [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] Successfully created port: fc1e01cd-012e-4aca-992a-6c6e6709f4a1 {{(pid=72351) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Aug 09 02:27:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1b90316d-6638-4603-bb09-38bf0ed795e3/disk --force-share --output=json" returned: 0 in 0.189s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:21 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Cannot resize image /opt/stack/data/nova/instances/1b90316d-6638-4603-bb09-38bf0ed795e3/disk to a smaller size. {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Aug 09 02:27:21 user nova-compute[72351]: DEBUG nova.objects.instance [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Lazy-loading 'migration_context' on Instance uuid 1b90316d-6638-4603-bb09-38bf0ed795e3 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:27:21 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] Created local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4851}}
Aug 09 02:27:21 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] Ensure instance console log exists: /opt/stack/data/nova/instances/1b90316d-6638-4603-bb09-38bf0ed795e3/console.log {{(pid=72351) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4603}}
Aug 09 02:27:21 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:27:21 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:27:21 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:27:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:22 user nova-compute[72351]: DEBUG nova.network.neutron [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] Successfully updated port: fc1e01cd-012e-4aca-992a-6c6e6709f4a1 {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:27:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Acquiring lock "refresh_cache-1b90316d-6638-4603-bb09-38bf0ed795e3" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:27:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Acquired lock "refresh_cache-1b90316d-6638-4603-bb09-38bf0ed795e3" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:27:22 user nova-compute[72351]: DEBUG nova.network.neutron [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:27:22 user nova-compute[72351]: DEBUG nova.compute.manager [req-e9eb43af-1cd7-4b66-8ae0-f94e1987d1be req-8b220ddf-a6fb-433a-976f-075d5e54e1c2 service nova] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] Received event network-changed-fc1e01cd-012e-4aca-992a-6c6e6709f4a1 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:27:22 user nova-compute[72351]: DEBUG nova.compute.manager [req-e9eb43af-1cd7-4b66-8ae0-f94e1987d1be req-8b220ddf-a6fb-433a-976f-075d5e54e1c2 service nova] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] Refreshing instance network info cache due to event network-changed-fc1e01cd-012e-4aca-992a-6c6e6709f4a1. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:27:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-e9eb43af-1cd7-4b66-8ae0-f94e1987d1be req-8b220ddf-a6fb-433a-976f-075d5e54e1c2 service nova] Acquiring lock "refresh_cache-1b90316d-6638-4603-bb09-38bf0ed795e3" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:27:22 user nova-compute[72351]: DEBUG nova.network.neutron [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:27:22 user nova-compute[72351]: DEBUG nova.network.neutron [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] Updating instance_info_cache with network_info: [{"id": "fc1e01cd-012e-4aca-992a-6c6e6709f4a1", "address": "fa:16:3e:69:d6:e1", "network": {"id": "9785f84c-cf45-464e-9127-332ce46abd11", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-87568186-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "eebc3a715ae04d48918b6c68b135935b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfc1e01cd-01", "ovs_interfaceid": "fc1e01cd-012e-4aca-992a-6c6e6709f4a1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:27:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Releasing lock "refresh_cache-1b90316d-6638-4603-bb09-38bf0ed795e3" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:27:22 user nova-compute[72351]: DEBUG nova.compute.manager [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] Instance network_info: |[{"id": "fc1e01cd-012e-4aca-992a-6c6e6709f4a1", "address": "fa:16:3e:69:d6:e1", "network": {"id": "9785f84c-cf45-464e-9127-332ce46abd11", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-87568186-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "eebc3a715ae04d48918b6c68b135935b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfc1e01cd-01", "ovs_interfaceid": "fc1e01cd-012e-4aca-992a-6c6e6709f4a1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1971}}
Aug 09 02:27:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-e9eb43af-1cd7-4b66-8ae0-f94e1987d1be req-8b220ddf-a6fb-433a-976f-075d5e54e1c2 service nova] Acquired lock "refresh_cache-1b90316d-6638-4603-bb09-38bf0ed795e3" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:27:22 user nova-compute[72351]: DEBUG nova.network.neutron [req-e9eb43af-1cd7-4b66-8ae0-f94e1987d1be req-8b220ddf-a6fb-433a-976f-075d5e54e1c2 service nova] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] Refreshing network info cache for port fc1e01cd-012e-4aca-992a-6c6e6709f4a1 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:27:22 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] Start _get_guest_xml network_info=[{"id": "fc1e01cd-012e-4aca-992a-6c6e6709f4a1", "address": "fa:16:3e:69:d6:e1", "network": {"id": "9785f84c-cf45-464e-9127-332ce46abd11", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-87568186-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "eebc3a715ae04d48918b6c68b135935b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfc1e01cd-01", "ovs_interfaceid": "fc1e01cd-012e-4aca-992a-6c6e6709f4a1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'size': 0, 'device_type': 'disk', 'encryption_secret_uuid': None, 'guest_format': None, 'encryption_options': None, 'disk_bus': 'virtio', 'encrypted': False, 'encryption_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'image_id': '18ccda7f-31d3-4978-ab9d-4ed2f74eff94'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:27:22 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:27:22 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:27:22 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:27:22 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:27:22 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:27:22 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:27:22 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:27:22 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:27:22 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:27:22 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:27:22 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:27:22 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:27:22 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:27:22 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:27:22 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:27:22 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:27:22 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:27:22 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:27:22 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:27:19Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServersNegativeTestJSON-server-132169414',display_name='tempest-ServersNegativeTestJSON-server-132169414',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serversnegativetestjson-server-132169414',id=29,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='eebc3a715ae04d48918b6c68b135935b',ramdisk_id='',reservation_id='r-1y0yt363',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServersNegativeTestJSON-1773070288',owner_user_name='tempest-ServersNegativeTestJSON-1773070288-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:27:20Z,user_data=None,user_id='ebc62bdcafdd4131be9abb7fe4ed2422',uuid=1b90316d-6638-4603-bb09-38bf0ed795e3,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "fc1e01cd-012e-4aca-992a-6c6e6709f4a1", "address": "fa:16:3e:69:d6:e1", "network": {"id": "9785f84c-cf45-464e-9127-332ce46abd11", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-87568186-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "eebc3a715ae04d48918b6c68b135935b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfc1e01cd-01", "ovs_interfaceid": "fc1e01cd-012e-4aca-992a-6c6e6709f4a1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:27:22 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Converting VIF {"id": "fc1e01cd-012e-4aca-992a-6c6e6709f4a1", "address": "fa:16:3e:69:d6:e1", "network": {"id": "9785f84c-cf45-464e-9127-332ce46abd11", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-87568186-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "eebc3a715ae04d48918b6c68b135935b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfc1e01cd-01", "ovs_interfaceid": "fc1e01cd-012e-4aca-992a-6c6e6709f4a1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:27:22 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:69:d6:e1,bridge_name='br-int',has_traffic_filtering=True,id=fc1e01cd-012e-4aca-992a-6c6e6709f4a1,network=Network(9785f84c-cf45-464e-9127-332ce46abd11),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfc1e01cd-01') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:27:22 user nova-compute[72351]: DEBUG nova.objects.instance [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Lazy-loading 'pci_devices' on Instance uuid 1b90316d-6638-4603-bb09-38bf0ed795e3 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:27:22 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:27:22 user nova-compute[72351]:   <uuid>1b90316d-6638-4603-bb09-38bf0ed795e3</uuid>
Aug 09 02:27:22 user nova-compute[72351]:   <name>instance-0000001d</name>
Aug 09 02:27:22 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:27:22 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:27:22 user nova-compute[72351]:   <metadata>
Aug 09 02:27:22 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:27:22 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:27:22 user nova-compute[72351]:       <nova:name>tempest-ServersNegativeTestJSON-server-132169414</nova:name>
Aug 09 02:27:22 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:27:22</nova:creationTime>
Aug 09 02:27:22 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:27:22 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:27:22 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:27:22 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:27:22 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:27:22 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:27:22 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:27:22 user nova-compute[72351]:       <nova:owner>
Aug 09 02:27:22 user nova-compute[72351]:         <nova:user uuid="ebc62bdcafdd4131be9abb7fe4ed2422">tempest-ServersNegativeTestJSON-1773070288-project-member</nova:user>
Aug 09 02:27:22 user nova-compute[72351]:         <nova:project uuid="eebc3a715ae04d48918b6c68b135935b">tempest-ServersNegativeTestJSON-1773070288</nova:project>
Aug 09 02:27:22 user nova-compute[72351]:       </nova:owner>
Aug 09 02:27:22 user nova-compute[72351]:       <nova:root type="image" uuid="18ccda7f-31d3-4978-ab9d-4ed2f74eff94"/>
Aug 09 02:27:22 user nova-compute[72351]:       <nova:ports>
Aug 09 02:27:22 user nova-compute[72351]:         <nova:port uuid="fc1e01cd-012e-4aca-992a-6c6e6709f4a1">
Aug 09 02:27:22 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.4" ipVersion="4"/>
Aug 09 02:27:22 user nova-compute[72351]:         </nova:port>
Aug 09 02:27:22 user nova-compute[72351]:       </nova:ports>
Aug 09 02:27:22 user nova-compute[72351]:     </nova:instance>
Aug 09 02:27:22 user nova-compute[72351]:   </metadata>
Aug 09 02:27:22 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:27:22 user nova-compute[72351]:     <system>
Aug 09 02:27:22 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:27:22 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:27:22 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:27:22 user nova-compute[72351]:       <entry name="serial">1b90316d-6638-4603-bb09-38bf0ed795e3</entry>
Aug 09 02:27:22 user nova-compute[72351]:       <entry name="uuid">1b90316d-6638-4603-bb09-38bf0ed795e3</entry>
Aug 09 02:27:22 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:27:22 user nova-compute[72351]:     </system>
Aug 09 02:27:22 user nova-compute[72351]:   </sysinfo>
Aug 09 02:27:22 user nova-compute[72351]:   <os>
Aug 09 02:27:22 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:27:22 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:27:22 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:27:22 user nova-compute[72351]:   </os>
Aug 09 02:27:22 user nova-compute[72351]:   <features>
Aug 09 02:27:22 user nova-compute[72351]:     <acpi/>
Aug 09 02:27:22 user nova-compute[72351]:     <apic/>
Aug 09 02:27:22 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:27:22 user nova-compute[72351]:   </features>
Aug 09 02:27:22 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:27:22 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:27:22 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:27:22 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:27:22 user nova-compute[72351]:   </clock>
Aug 09 02:27:22 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:27:22 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:27:22 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:27:22 user nova-compute[72351]:   </cpu>
Aug 09 02:27:22 user nova-compute[72351]:   <devices>
Aug 09 02:27:22 user nova-compute[72351]:     <disk type="file" device="disk">
Aug 09 02:27:22 user nova-compute[72351]:       <driver name="qemu" type="qcow2" cache="none"/>
Aug 09 02:27:22 user nova-compute[72351]:       <source file="/opt/stack/data/nova/instances/1b90316d-6638-4603-bb09-38bf0ed795e3/disk"/>
Aug 09 02:27:22 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:27:22 user nova-compute[72351]:     </disk>
Aug 09 02:27:22 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:27:22 user nova-compute[72351]:       <mac address="fa:16:3e:69:d6:e1"/>
Aug 09 02:27:22 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:27:22 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:27:22 user nova-compute[72351]:       <target dev="tapfc1e01cd-01"/>
Aug 09 02:27:22 user nova-compute[72351]:     </interface>
Aug 09 02:27:22 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:27:22 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/1b90316d-6638-4603-bb09-38bf0ed795e3/console.log" append="off"/>
Aug 09 02:27:22 user nova-compute[72351]:     </serial>
Aug 09 02:27:22 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:27:22 user nova-compute[72351]:     <video>
Aug 09 02:27:22 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:27:22 user nova-compute[72351]:     </video>
Aug 09 02:27:22 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:27:22 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:27:22 user nova-compute[72351]:     </rng>
Aug 09 02:27:22 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:27:22 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:27:22 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:27:22 user nova-compute[72351]:     </memballoon>
Aug 09 02:27:22 user nova-compute[72351]:   </devices>
Aug 09 02:27:22 user nova-compute[72351]: </domain>
Aug 09 02:27:22 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:27:22 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:27:19Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServersNegativeTestJSON-server-132169414',display_name='tempest-ServersNegativeTestJSON-server-132169414',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serversnegativetestjson-server-132169414',id=29,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='eebc3a715ae04d48918b6c68b135935b',ramdisk_id='',reservation_id='r-1y0yt363',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServersNegativeTestJSON-1773070288',owner_user_name='tempest-ServersNegativeTestJSON-1773070288-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:27:20Z,user_data=None,user_id='ebc62bdcafdd4131be9abb7fe4ed2422',uuid=1b90316d-6638-4603-bb09-38bf0ed795e3,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "fc1e01cd-012e-4aca-992a-6c6e6709f4a1", "address": "fa:16:3e:69:d6:e1", "network": {"id": "9785f84c-cf45-464e-9127-332ce46abd11", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-87568186-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "eebc3a715ae04d48918b6c68b135935b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfc1e01cd-01", "ovs_interfaceid": "fc1e01cd-012e-4aca-992a-6c6e6709f4a1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:27:22 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Converting VIF {"id": "fc1e01cd-012e-4aca-992a-6c6e6709f4a1", "address": "fa:16:3e:69:d6:e1", "network": {"id": "9785f84c-cf45-464e-9127-332ce46abd11", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-87568186-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "eebc3a715ae04d48918b6c68b135935b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfc1e01cd-01", "ovs_interfaceid": "fc1e01cd-012e-4aca-992a-6c6e6709f4a1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:27:22 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:69:d6:e1,bridge_name='br-int',has_traffic_filtering=True,id=fc1e01cd-012e-4aca-992a-6c6e6709f4a1,network=Network(9785f84c-cf45-464e-9127-332ce46abd11),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfc1e01cd-01') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:27:22 user nova-compute[72351]: DEBUG os_vif [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:69:d6:e1,bridge_name='br-int',has_traffic_filtering=True,id=fc1e01cd-012e-4aca-992a-6c6e6709f4a1,network=Network(9785f84c-cf45-464e-9127-332ce46abd11),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfc1e01cd-01') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:27:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:27:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:27:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapfc1e01cd-01, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:27:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapfc1e01cd-01, col_values=(('external_ids', {'iface-id': 'fc1e01cd-012e-4aca-992a-6c6e6709f4a1', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:69:d6:e1', 'vm-uuid': '1b90316d-6638-4603-bb09-38bf0ed795e3'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:27:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:27:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:22 user nova-compute[72351]: INFO os_vif [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:69:d6:e1,bridge_name='br-int',has_traffic_filtering=True,id=fc1e01cd-012e-4aca-992a-6c6e6709f4a1,network=Network(9785f84c-cf45-464e-9127-332ce46abd11),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfc1e01cd-01')
Aug 09 02:27:22 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:27:22 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] No VIF found with MAC fa:16:3e:69:d6:e1, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:27:23 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:23 user nova-compute[72351]: DEBUG nova.network.neutron [req-e9eb43af-1cd7-4b66-8ae0-f94e1987d1be req-8b220ddf-a6fb-433a-976f-075d5e54e1c2 service nova] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] Updated VIF entry in instance network info cache for port fc1e01cd-012e-4aca-992a-6c6e6709f4a1. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:27:23 user nova-compute[72351]: DEBUG nova.network.neutron [req-e9eb43af-1cd7-4b66-8ae0-f94e1987d1be req-8b220ddf-a6fb-433a-976f-075d5e54e1c2 service nova] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] Updating instance_info_cache with network_info: [{"id": "fc1e01cd-012e-4aca-992a-6c6e6709f4a1", "address": "fa:16:3e:69:d6:e1", "network": {"id": "9785f84c-cf45-464e-9127-332ce46abd11", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-87568186-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "eebc3a715ae04d48918b6c68b135935b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfc1e01cd-01", "ovs_interfaceid": "fc1e01cd-012e-4aca-992a-6c6e6709f4a1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:27:23 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-e9eb43af-1cd7-4b66-8ae0-f94e1987d1be req-8b220ddf-a6fb-433a-976f-075d5e54e1c2 service nova] Releasing lock "refresh_cache-1b90316d-6638-4603-bb09-38bf0ed795e3" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:27:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:24 user nova-compute[72351]: DEBUG nova.compute.manager [req-10c63ed5-021f-43a9-ad0c-df0a1e62cead req-294ec6af-833b-4ade-963c-2c8c76b1704b service nova] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] Received event network-vif-plugged-fc1e01cd-012e-4aca-992a-6c6e6709f4a1 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:27:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-10c63ed5-021f-43a9-ad0c-df0a1e62cead req-294ec6af-833b-4ade-963c-2c8c76b1704b service nova] Acquiring lock "1b90316d-6638-4603-bb09-38bf0ed795e3-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:27:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-10c63ed5-021f-43a9-ad0c-df0a1e62cead req-294ec6af-833b-4ade-963c-2c8c76b1704b service nova] Lock "1b90316d-6638-4603-bb09-38bf0ed795e3-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:27:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-10c63ed5-021f-43a9-ad0c-df0a1e62cead req-294ec6af-833b-4ade-963c-2c8c76b1704b service nova] Lock "1b90316d-6638-4603-bb09-38bf0ed795e3-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:27:24 user nova-compute[72351]: DEBUG nova.compute.manager [req-10c63ed5-021f-43a9-ad0c-df0a1e62cead req-294ec6af-833b-4ade-963c-2c8c76b1704b service nova] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] No waiting events found dispatching network-vif-plugged-fc1e01cd-012e-4aca-992a-6c6e6709f4a1 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:27:24 user nova-compute[72351]: WARNING nova.compute.manager [req-10c63ed5-021f-43a9-ad0c-df0a1e62cead req-294ec6af-833b-4ade-963c-2c8c76b1704b service nova] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] Received unexpected event network-vif-plugged-fc1e01cd-012e-4aca-992a-6c6e6709f4a1 for instance with vm_state building and task_state spawning.
Aug 09 02:27:26 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548046.29656, 1b90316d-6638-4603-bb09-38bf0ed795e3 => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:27:26 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] VM Resumed (Lifecycle Event)
Aug 09 02:27:26 user nova-compute[72351]: DEBUG nova.compute.manager [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:27:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] Guest created on hypervisor {{(pid=72351) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4411}}
Aug 09 02:27:26 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] Instance spawned successfully.
Aug 09 02:27:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:894}}
Aug 09 02:27:26 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:27:26 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:27:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] Found default for hw_cdrom_bus of ide {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:27:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] Found default for hw_disk_bus of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:27:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] Found default for hw_input_bus of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:27:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] Found default for hw_pointer_model of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:27:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] Found default for hw_video_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:27:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] Found default for hw_vif_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:27:26 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:27:26 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548046.2997591, 1b90316d-6638-4603-bb09-38bf0ed795e3 => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:27:26 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] VM Started (Lifecycle Event)
Aug 09 02:27:26 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:27:26 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:27:26 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:27:26 user nova-compute[72351]: INFO nova.compute.manager [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] Took 5.88 seconds to spawn the instance on the hypervisor.
Aug 09 02:27:26 user nova-compute[72351]: DEBUG nova.compute.manager [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:27:26 user nova-compute[72351]: INFO nova.compute.manager [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] Took 6.71 seconds to build instance.
Aug 09 02:27:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-aa4da420-e2c7-48b2-9fac-3b5669cdee48 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Lock "1b90316d-6638-4603-bb09-38bf0ed795e3" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 6.821s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:27:26 user nova-compute[72351]: DEBUG nova.compute.manager [req-75d3a28a-94aa-4136-9b64-206aa8f4e187 req-05a4e11c-06fb-4d60-a4d8-cf5295cb22d9 service nova] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] Received event network-vif-plugged-fc1e01cd-012e-4aca-992a-6c6e6709f4a1 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:27:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-75d3a28a-94aa-4136-9b64-206aa8f4e187 req-05a4e11c-06fb-4d60-a4d8-cf5295cb22d9 service nova] Acquiring lock "1b90316d-6638-4603-bb09-38bf0ed795e3-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:27:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-75d3a28a-94aa-4136-9b64-206aa8f4e187 req-05a4e11c-06fb-4d60-a4d8-cf5295cb22d9 service nova] Lock "1b90316d-6638-4603-bb09-38bf0ed795e3-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:27:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-75d3a28a-94aa-4136-9b64-206aa8f4e187 req-05a4e11c-06fb-4d60-a4d8-cf5295cb22d9 service nova] Lock "1b90316d-6638-4603-bb09-38bf0ed795e3-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:27:26 user nova-compute[72351]: DEBUG nova.compute.manager [req-75d3a28a-94aa-4136-9b64-206aa8f4e187 req-05a4e11c-06fb-4d60-a4d8-cf5295cb22d9 service nova] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] No waiting events found dispatching network-vif-plugged-fc1e01cd-012e-4aca-992a-6c6e6709f4a1 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:27:26 user nova-compute[72351]: WARNING nova.compute.manager [req-75d3a28a-94aa-4136-9b64-206aa8f4e187 req-05a4e11c-06fb-4d60-a4d8-cf5295cb22d9 service nova] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] Received unexpected event network-vif-plugged-fc1e01cd-012e-4aca-992a-6c6e6709f4a1 for instance with vm_state active and task_state None.
Aug 09 02:27:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:28 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:31 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:32 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:33 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:34 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:37 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Acquiring lock "a03fbded-23ad-440b-97e7-bea07425e873" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:27:37 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lock "a03fbded-23ad-440b-97e7-bea07425e873" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:27:37 user nova-compute[72351]: DEBUG nova.compute.manager [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Starting instance... {{(pid=72351) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2406}}
Aug 09 02:27:37 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:37 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:27:37 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:27:37 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72351) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Aug 09 02:27:37 user nova-compute[72351]: INFO nova.compute.claims [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Claim successful on node user
Aug 09 02:27:38 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:38 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:27:38 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:27:38 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.555s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:27:38 user nova-compute[72351]: DEBUG nova.compute.manager [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Start building networks asynchronously for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2803}}
Aug 09 02:27:38 user nova-compute[72351]: DEBUG nova.compute.manager [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Allocating IP information in the background. {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1956}}
Aug 09 02:27:38 user nova-compute[72351]: DEBUG nova.network.neutron [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: a03fbded-23ad-440b-97e7-bea07425e873] allocate_for_instance() {{(pid=72351) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Aug 09 02:27:38 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Aug 09 02:27:38 user nova-compute[72351]: DEBUG nova.compute.manager [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Start building block device mappings for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2838}}
Aug 09 02:27:38 user nova-compute[72351]: DEBUG nova.policy [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'a103ebb9b63f4a259163ec25a33a0dcc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc12f9037cd54bb6a2fdff3071132dd4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72351) authorize /opt/stack/nova/nova/policy.py:203}}
Aug 09 02:27:38 user nova-compute[72351]: DEBUG nova.compute.manager [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Start spawning the instance on the hypervisor. {{(pid=72351) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2612}}
Aug 09 02:27:38 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Creating instance directory {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4717}}
Aug 09 02:27:38 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Creating image(s)
Aug 09 02:27:38 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Acquiring lock "/opt/stack/data/nova/instances/a03fbded-23ad-440b-97e7-bea07425e873/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:27:38 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lock "/opt/stack/data/nova/instances/a03fbded-23ad-440b-97e7-bea07425e873/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:27:38 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lock "/opt/stack/data/nova/instances/a03fbded-23ad-440b-97e7-bea07425e873/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:27:38 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:38 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.145s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:38 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Acquiring lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:27:38 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:27:38 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:38 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.158s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:38 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/a03fbded-23ad-440b-97e7-bea07425e873/disk 1073741824 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/a03fbded-23ad-440b-97e7-bea07425e873/disk 1073741824" returned: 0 in 0.237s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:39 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.401s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:27:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.164s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:39 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Checking if we can resize image /opt/stack/data/nova/instances/a03fbded-23ad-440b-97e7-bea07425e873/disk. size=1073741824 {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Aug 09 02:27:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a03fbded-23ad-440b-97e7-bea07425e873/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a03fbded-23ad-440b-97e7-bea07425e873/disk --force-share --output=json" returned: 0 in 0.151s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:39 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Cannot resize image /opt/stack/data/nova/instances/a03fbded-23ad-440b-97e7-bea07425e873/disk to a smaller size. {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Aug 09 02:27:39 user nova-compute[72351]: DEBUG nova.objects.instance [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lazy-loading 'migration_context' on Instance uuid a03fbded-23ad-440b-97e7-bea07425e873 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:27:39 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Created local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4851}}
Aug 09 02:27:39 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Ensure instance console log exists: /opt/stack/data/nova/instances/a03fbded-23ad-440b-97e7-bea07425e873/console.log {{(pid=72351) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4603}}
Aug 09 02:27:39 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:27:39 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:27:39 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:27:39 user nova-compute[72351]: DEBUG nova.network.neutron [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Successfully created port: e58740da-bc24-4309-8e35-48fc70bece21 {{(pid=72351) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Aug 09 02:27:40 user nova-compute[72351]: DEBUG nova.network.neutron [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Successfully updated port: e58740da-bc24-4309-8e35-48fc70bece21 {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:27:40 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Acquiring lock "refresh_cache-a03fbded-23ad-440b-97e7-bea07425e873" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:27:40 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Acquired lock "refresh_cache-a03fbded-23ad-440b-97e7-bea07425e873" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:27:40 user nova-compute[72351]: DEBUG nova.network.neutron [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:27:40 user nova-compute[72351]: DEBUG nova.network.neutron [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:27:40 user nova-compute[72351]: DEBUG nova.compute.manager [req-f6a258a4-e816-4233-9821-f718b83b30eb req-b06c9dec-3446-4c29-8eb8-cacd16505c90 service nova] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Received event network-changed-e58740da-bc24-4309-8e35-48fc70bece21 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:27:40 user nova-compute[72351]: DEBUG nova.compute.manager [req-f6a258a4-e816-4233-9821-f718b83b30eb req-b06c9dec-3446-4c29-8eb8-cacd16505c90 service nova] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Refreshing instance network info cache due to event network-changed-e58740da-bc24-4309-8e35-48fc70bece21. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:27:40 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-f6a258a4-e816-4233-9821-f718b83b30eb req-b06c9dec-3446-4c29-8eb8-cacd16505c90 service nova] Acquiring lock "refresh_cache-a03fbded-23ad-440b-97e7-bea07425e873" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:27:40 user nova-compute[72351]: DEBUG nova.network.neutron [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Updating instance_info_cache with network_info: [{"id": "e58740da-bc24-4309-8e35-48fc70bece21", "address": "fa:16:3e:f3:4e:5e", "network": {"id": "49bfc70e-93c7-4607-9507-4711bda1a686", "bridge": "br-int", "label": "tempest-TestStampPattern-1656229455-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fc12f9037cd54bb6a2fdff3071132dd4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape58740da-bc", "ovs_interfaceid": "e58740da-bc24-4309-8e35-48fc70bece21", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:27:41 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Releasing lock "refresh_cache-a03fbded-23ad-440b-97e7-bea07425e873" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:27:41 user nova-compute[72351]: DEBUG nova.compute.manager [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Instance network_info: |[{"id": "e58740da-bc24-4309-8e35-48fc70bece21", "address": "fa:16:3e:f3:4e:5e", "network": {"id": "49bfc70e-93c7-4607-9507-4711bda1a686", "bridge": "br-int", "label": "tempest-TestStampPattern-1656229455-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fc12f9037cd54bb6a2fdff3071132dd4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape58740da-bc", "ovs_interfaceid": "e58740da-bc24-4309-8e35-48fc70bece21", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1971}}
Aug 09 02:27:41 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-f6a258a4-e816-4233-9821-f718b83b30eb req-b06c9dec-3446-4c29-8eb8-cacd16505c90 service nova] Acquired lock "refresh_cache-a03fbded-23ad-440b-97e7-bea07425e873" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:27:41 user nova-compute[72351]: DEBUG nova.network.neutron [req-f6a258a4-e816-4233-9821-f718b83b30eb req-b06c9dec-3446-4c29-8eb8-cacd16505c90 service nova] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Refreshing network info cache for port e58740da-bc24-4309-8e35-48fc70bece21 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:27:41 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Start _get_guest_xml network_info=[{"id": "e58740da-bc24-4309-8e35-48fc70bece21", "address": "fa:16:3e:f3:4e:5e", "network": {"id": "49bfc70e-93c7-4607-9507-4711bda1a686", "bridge": "br-int", "label": "tempest-TestStampPattern-1656229455-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fc12f9037cd54bb6a2fdff3071132dd4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape58740da-bc", "ovs_interfaceid": "e58740da-bc24-4309-8e35-48fc70bece21", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'size': 0, 'device_type': 'disk', 'encryption_secret_uuid': None, 'guest_format': None, 'encryption_options': None, 'disk_bus': 'virtio', 'encrypted': False, 'encryption_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'image_id': '18ccda7f-31d3-4978-ab9d-4ed2f74eff94'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:27:41 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:27:41 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:27:41 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:27:41 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:27:41 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:27:41 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:27:41 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:27:41 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:27:41 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:27:41 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:27:41 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:27:41 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:27:41 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:27:41 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:27:41 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:27:41 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:27:41 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:27:41 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:27:41 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:27:37Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestStampPattern-server-356325522',display_name='tempest-TestStampPattern-server-356325522',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-teststamppattern-server-356325522',id=30,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBPzyrlB54Gi/8YMv0Cfs/esdmr1xNOdiloZGKR/XYFcg8Nos/w3Zm8ZOseVT8bKKFqeq3DP2KQSbuWg07AAvODzdTY0X+5ZdNLlP+cvLLuP4ef0UvshCJkUtWTANxgqmUw==',key_name='tempest-TestStampPattern-2061411239',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='fc12f9037cd54bb6a2fdff3071132dd4',ramdisk_id='',reservation_id='r-9r7jqe2t',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestStampPattern-1244733964',owner_user_name='tempest-TestStampPattern-1244733964-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:27:39Z,user_data=None,user_id='a103ebb9b63f4a259163ec25a33a0dcc',uuid=a03fbded-23ad-440b-97e7-bea07425e873,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "e58740da-bc24-4309-8e35-48fc70bece21", "address": "fa:16:3e:f3:4e:5e", "network": {"id": "49bfc70e-93c7-4607-9507-4711bda1a686", "bridge": "br-int", "label": "tempest-TestStampPattern-1656229455-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fc12f9037cd54bb6a2fdff3071132dd4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape58740da-bc", "ovs_interfaceid": "e58740da-bc24-4309-8e35-48fc70bece21", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:27:41 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Converting VIF {"id": "e58740da-bc24-4309-8e35-48fc70bece21", "address": "fa:16:3e:f3:4e:5e", "network": {"id": "49bfc70e-93c7-4607-9507-4711bda1a686", "bridge": "br-int", "label": "tempest-TestStampPattern-1656229455-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fc12f9037cd54bb6a2fdff3071132dd4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape58740da-bc", "ovs_interfaceid": "e58740da-bc24-4309-8e35-48fc70bece21", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:27:41 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:f3:4e:5e,bridge_name='br-int',has_traffic_filtering=True,id=e58740da-bc24-4309-8e35-48fc70bece21,network=Network(49bfc70e-93c7-4607-9507-4711bda1a686),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape58740da-bc') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:27:41 user nova-compute[72351]: DEBUG nova.objects.instance [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lazy-loading 'pci_devices' on Instance uuid a03fbded-23ad-440b-97e7-bea07425e873 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:27:41 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: a03fbded-23ad-440b-97e7-bea07425e873] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:27:41 user nova-compute[72351]:   <uuid>a03fbded-23ad-440b-97e7-bea07425e873</uuid>
Aug 09 02:27:41 user nova-compute[72351]:   <name>instance-0000001e</name>
Aug 09 02:27:41 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:27:41 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:27:41 user nova-compute[72351]:   <metadata>
Aug 09 02:27:41 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:27:41 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:27:41 user nova-compute[72351]:       <nova:name>tempest-TestStampPattern-server-356325522</nova:name>
Aug 09 02:27:41 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:27:41</nova:creationTime>
Aug 09 02:27:41 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:27:41 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:27:41 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:27:41 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:27:41 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:27:41 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:27:41 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:27:41 user nova-compute[72351]:       <nova:owner>
Aug 09 02:27:41 user nova-compute[72351]:         <nova:user uuid="a103ebb9b63f4a259163ec25a33a0dcc">tempest-TestStampPattern-1244733964-project-member</nova:user>
Aug 09 02:27:41 user nova-compute[72351]:         <nova:project uuid="fc12f9037cd54bb6a2fdff3071132dd4">tempest-TestStampPattern-1244733964</nova:project>
Aug 09 02:27:41 user nova-compute[72351]:       </nova:owner>
Aug 09 02:27:41 user nova-compute[72351]:       <nova:root type="image" uuid="18ccda7f-31d3-4978-ab9d-4ed2f74eff94"/>
Aug 09 02:27:41 user nova-compute[72351]:       <nova:ports>
Aug 09 02:27:41 user nova-compute[72351]:         <nova:port uuid="e58740da-bc24-4309-8e35-48fc70bece21">
Aug 09 02:27:41 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.12" ipVersion="4"/>
Aug 09 02:27:41 user nova-compute[72351]:         </nova:port>
Aug 09 02:27:41 user nova-compute[72351]:       </nova:ports>
Aug 09 02:27:41 user nova-compute[72351]:     </nova:instance>
Aug 09 02:27:41 user nova-compute[72351]:   </metadata>
Aug 09 02:27:41 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:27:41 user nova-compute[72351]:     <system>
Aug 09 02:27:41 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:27:41 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:27:41 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:27:41 user nova-compute[72351]:       <entry name="serial">a03fbded-23ad-440b-97e7-bea07425e873</entry>
Aug 09 02:27:41 user nova-compute[72351]:       <entry name="uuid">a03fbded-23ad-440b-97e7-bea07425e873</entry>
Aug 09 02:27:41 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:27:41 user nova-compute[72351]:     </system>
Aug 09 02:27:41 user nova-compute[72351]:   </sysinfo>
Aug 09 02:27:41 user nova-compute[72351]:   <os>
Aug 09 02:27:41 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:27:41 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:27:41 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:27:41 user nova-compute[72351]:   </os>
Aug 09 02:27:41 user nova-compute[72351]:   <features>
Aug 09 02:27:41 user nova-compute[72351]:     <acpi/>
Aug 09 02:27:41 user nova-compute[72351]:     <apic/>
Aug 09 02:27:41 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:27:41 user nova-compute[72351]:   </features>
Aug 09 02:27:41 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:27:41 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:27:41 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:27:41 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:27:41 user nova-compute[72351]:   </clock>
Aug 09 02:27:41 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:27:41 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:27:41 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:27:41 user nova-compute[72351]:   </cpu>
Aug 09 02:27:41 user nova-compute[72351]:   <devices>
Aug 09 02:27:41 user nova-compute[72351]:     <disk type="file" device="disk">
Aug 09 02:27:41 user nova-compute[72351]:       <driver name="qemu" type="qcow2" cache="none"/>
Aug 09 02:27:41 user nova-compute[72351]:       <source file="/opt/stack/data/nova/instances/a03fbded-23ad-440b-97e7-bea07425e873/disk"/>
Aug 09 02:27:41 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:27:41 user nova-compute[72351]:     </disk>
Aug 09 02:27:41 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:27:41 user nova-compute[72351]:       <mac address="fa:16:3e:f3:4e:5e"/>
Aug 09 02:27:41 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:27:41 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:27:41 user nova-compute[72351]:       <target dev="tape58740da-bc"/>
Aug 09 02:27:41 user nova-compute[72351]:     </interface>
Aug 09 02:27:41 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:27:41 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/a03fbded-23ad-440b-97e7-bea07425e873/console.log" append="off"/>
Aug 09 02:27:41 user nova-compute[72351]:     </serial>
Aug 09 02:27:41 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:27:41 user nova-compute[72351]:     <video>
Aug 09 02:27:41 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:27:41 user nova-compute[72351]:     </video>
Aug 09 02:27:41 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:27:41 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:27:41 user nova-compute[72351]:     </rng>
Aug 09 02:27:41 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:27:41 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:27:41 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:27:41 user nova-compute[72351]:     </memballoon>
Aug 09 02:27:41 user nova-compute[72351]:   </devices>
Aug 09 02:27:41 user nova-compute[72351]: </domain>
Aug 09 02:27:41 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:27:41 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:27:37Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestStampPattern-server-356325522',display_name='tempest-TestStampPattern-server-356325522',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-teststamppattern-server-356325522',id=30,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBPzyrlB54Gi/8YMv0Cfs/esdmr1xNOdiloZGKR/XYFcg8Nos/w3Zm8ZOseVT8bKKFqeq3DP2KQSbuWg07AAvODzdTY0X+5ZdNLlP+cvLLuP4ef0UvshCJkUtWTANxgqmUw==',key_name='tempest-TestStampPattern-2061411239',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='fc12f9037cd54bb6a2fdff3071132dd4',ramdisk_id='',reservation_id='r-9r7jqe2t',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestStampPattern-1244733964',owner_user_name='tempest-TestStampPattern-1244733964-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:27:39Z,user_data=None,user_id='a103ebb9b63f4a259163ec25a33a0dcc',uuid=a03fbded-23ad-440b-97e7-bea07425e873,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "e58740da-bc24-4309-8e35-48fc70bece21", "address": "fa:16:3e:f3:4e:5e", "network": {"id": "49bfc70e-93c7-4607-9507-4711bda1a686", "bridge": "br-int", "label": "tempest-TestStampPattern-1656229455-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fc12f9037cd54bb6a2fdff3071132dd4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape58740da-bc", "ovs_interfaceid": "e58740da-bc24-4309-8e35-48fc70bece21", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:27:41 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Converting VIF {"id": "e58740da-bc24-4309-8e35-48fc70bece21", "address": "fa:16:3e:f3:4e:5e", "network": {"id": "49bfc70e-93c7-4607-9507-4711bda1a686", "bridge": "br-int", "label": "tempest-TestStampPattern-1656229455-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fc12f9037cd54bb6a2fdff3071132dd4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape58740da-bc", "ovs_interfaceid": "e58740da-bc24-4309-8e35-48fc70bece21", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:27:41 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:f3:4e:5e,bridge_name='br-int',has_traffic_filtering=True,id=e58740da-bc24-4309-8e35-48fc70bece21,network=Network(49bfc70e-93c7-4607-9507-4711bda1a686),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape58740da-bc') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:27:41 user nova-compute[72351]: DEBUG os_vif [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:f3:4e:5e,bridge_name='br-int',has_traffic_filtering=True,id=e58740da-bc24-4309-8e35-48fc70bece21,network=Network(49bfc70e-93c7-4607-9507-4711bda1a686),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape58740da-bc') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:27:41 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:41 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:27:41 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:27:41 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:41 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tape58740da-bc, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:27:41 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tape58740da-bc, col_values=(('external_ids', {'iface-id': 'e58740da-bc24-4309-8e35-48fc70bece21', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:f3:4e:5e', 'vm-uuid': 'a03fbded-23ad-440b-97e7-bea07425e873'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:27:41 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:27:41 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:41 user nova-compute[72351]: INFO os_vif [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:f3:4e:5e,bridge_name='br-int',has_traffic_filtering=True,id=e58740da-bc24-4309-8e35-48fc70bece21,network=Network(49bfc70e-93c7-4607-9507-4711bda1a686),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape58740da-bc')
Aug 09 02:27:41 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:27:41 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] No VIF found with MAC fa:16:3e:f3:4e:5e, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:27:41 user nova-compute[72351]: DEBUG nova.network.neutron [req-f6a258a4-e816-4233-9821-f718b83b30eb req-b06c9dec-3446-4c29-8eb8-cacd16505c90 service nova] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Updated VIF entry in instance network info cache for port e58740da-bc24-4309-8e35-48fc70bece21. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:27:41 user nova-compute[72351]: DEBUG nova.network.neutron [req-f6a258a4-e816-4233-9821-f718b83b30eb req-b06c9dec-3446-4c29-8eb8-cacd16505c90 service nova] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Updating instance_info_cache with network_info: [{"id": "e58740da-bc24-4309-8e35-48fc70bece21", "address": "fa:16:3e:f3:4e:5e", "network": {"id": "49bfc70e-93c7-4607-9507-4711bda1a686", "bridge": "br-int", "label": "tempest-TestStampPattern-1656229455-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fc12f9037cd54bb6a2fdff3071132dd4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape58740da-bc", "ovs_interfaceid": "e58740da-bc24-4309-8e35-48fc70bece21", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:27:41 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-f6a258a4-e816-4233-9821-f718b83b30eb req-b06c9dec-3446-4c29-8eb8-cacd16505c90 service nova] Releasing lock "refresh_cache-a03fbded-23ad-440b-97e7-bea07425e873" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:27:42 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:42 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:42 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:42 user nova-compute[72351]: DEBUG nova.compute.manager [req-32b1aef3-7dc9-4b90-a21d-a1222eb81e86 req-6bfdfb96-f89b-40b2-9396-2233fa256d21 service nova] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Received event network-vif-plugged-e58740da-bc24-4309-8e35-48fc70bece21 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:27:42 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-32b1aef3-7dc9-4b90-a21d-a1222eb81e86 req-6bfdfb96-f89b-40b2-9396-2233fa256d21 service nova] Acquiring lock "a03fbded-23ad-440b-97e7-bea07425e873-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:27:42 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-32b1aef3-7dc9-4b90-a21d-a1222eb81e86 req-6bfdfb96-f89b-40b2-9396-2233fa256d21 service nova] Lock "a03fbded-23ad-440b-97e7-bea07425e873-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:27:42 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-32b1aef3-7dc9-4b90-a21d-a1222eb81e86 req-6bfdfb96-f89b-40b2-9396-2233fa256d21 service nova] Lock "a03fbded-23ad-440b-97e7-bea07425e873-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:27:42 user nova-compute[72351]: DEBUG nova.compute.manager [req-32b1aef3-7dc9-4b90-a21d-a1222eb81e86 req-6bfdfb96-f89b-40b2-9396-2233fa256d21 service nova] [instance: a03fbded-23ad-440b-97e7-bea07425e873] No waiting events found dispatching network-vif-plugged-e58740da-bc24-4309-8e35-48fc70bece21 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:27:42 user nova-compute[72351]: WARNING nova.compute.manager [req-32b1aef3-7dc9-4b90-a21d-a1222eb81e86 req-6bfdfb96-f89b-40b2-9396-2233fa256d21 service nova] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Received unexpected event network-vif-plugged-e58740da-bc24-4309-8e35-48fc70bece21 for instance with vm_state building and task_state spawning.
Aug 09 02:27:43 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:43 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:43 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:43 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:44 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548064.5971026, a03fbded-23ad-440b-97e7-bea07425e873 => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:27:44 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: a03fbded-23ad-440b-97e7-bea07425e873] VM Resumed (Lifecycle Event)
Aug 09 02:27:44 user nova-compute[72351]: DEBUG nova.compute.manager [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:27:44 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Guest created on hypervisor {{(pid=72351) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4411}}
Aug 09 02:27:44 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Instance spawned successfully.
Aug 09 02:27:44 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:894}}
Aug 09 02:27:44 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:27:44 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:27:44 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Found default for hw_cdrom_bus of ide {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:27:44 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Found default for hw_disk_bus of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:27:44 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Found default for hw_input_bus of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:27:44 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Found default for hw_pointer_model of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:27:44 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Found default for hw_video_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:27:44 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Found default for hw_vif_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:27:44 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: a03fbded-23ad-440b-97e7-bea07425e873] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:27:44 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548064.6006665, a03fbded-23ad-440b-97e7-bea07425e873 => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:27:44 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: a03fbded-23ad-440b-97e7-bea07425e873] VM Started (Lifecycle Event)
Aug 09 02:27:44 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:27:44 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:27:44 user nova-compute[72351]: INFO nova.compute.manager [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Took 6.10 seconds to spawn the instance on the hypervisor.
Aug 09 02:27:44 user nova-compute[72351]: DEBUG nova.compute.manager [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:27:44 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: a03fbded-23ad-440b-97e7-bea07425e873] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:27:44 user nova-compute[72351]: INFO nova.compute.manager [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Took 7.06 seconds to build instance.
Aug 09 02:27:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b157928f-6547-4d78-bada-3a85ffcc9804 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lock "a03fbded-23ad-440b-97e7-bea07425e873" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 7.202s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:27:44 user nova-compute[72351]: DEBUG nova.compute.manager [req-c8fcbe5b-17af-41c7-91cb-3cb035698611 req-da2731e3-4f66-4fd9-a1fd-71862c3b263d service nova] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Received event network-vif-plugged-e58740da-bc24-4309-8e35-48fc70bece21 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:27:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c8fcbe5b-17af-41c7-91cb-3cb035698611 req-da2731e3-4f66-4fd9-a1fd-71862c3b263d service nova] Acquiring lock "a03fbded-23ad-440b-97e7-bea07425e873-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:27:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c8fcbe5b-17af-41c7-91cb-3cb035698611 req-da2731e3-4f66-4fd9-a1fd-71862c3b263d service nova] Lock "a03fbded-23ad-440b-97e7-bea07425e873-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:27:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c8fcbe5b-17af-41c7-91cb-3cb035698611 req-da2731e3-4f66-4fd9-a1fd-71862c3b263d service nova] Lock "a03fbded-23ad-440b-97e7-bea07425e873-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:27:44 user nova-compute[72351]: DEBUG nova.compute.manager [req-c8fcbe5b-17af-41c7-91cb-3cb035698611 req-da2731e3-4f66-4fd9-a1fd-71862c3b263d service nova] [instance: a03fbded-23ad-440b-97e7-bea07425e873] No waiting events found dispatching network-vif-plugged-e58740da-bc24-4309-8e35-48fc70bece21 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:27:44 user nova-compute[72351]: WARNING nova.compute.manager [req-c8fcbe5b-17af-41c7-91cb-3cb035698611 req-da2731e3-4f66-4fd9-a1fd-71862c3b263d service nova] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Received unexpected event network-vif-plugged-e58740da-bc24-4309-8e35-48fc70bece21 for instance with vm_state active and task_state None.
Aug 09 02:27:46 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:46 user nova-compute[72351]: DEBUG nova.compute.manager [req-e40504d5-62d3-4279-bf3f-e19136790ff6 req-2b625c93-60d7-49b3-b521-76811d45b0e1 service nova] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Received event network-changed-fb1b25f9-3a24-43df-805c-9fffe7cd74f7 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:27:46 user nova-compute[72351]: DEBUG nova.compute.manager [req-e40504d5-62d3-4279-bf3f-e19136790ff6 req-2b625c93-60d7-49b3-b521-76811d45b0e1 service nova] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Refreshing instance network info cache due to event network-changed-fb1b25f9-3a24-43df-805c-9fffe7cd74f7. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:27:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-e40504d5-62d3-4279-bf3f-e19136790ff6 req-2b625c93-60d7-49b3-b521-76811d45b0e1 service nova] Acquiring lock "refresh_cache-16c1e558-baa1-4c2d-b13b-50585f74d587" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:27:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-e40504d5-62d3-4279-bf3f-e19136790ff6 req-2b625c93-60d7-49b3-b521-76811d45b0e1 service nova] Acquired lock "refresh_cache-16c1e558-baa1-4c2d-b13b-50585f74d587" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:27:46 user nova-compute[72351]: DEBUG nova.network.neutron [req-e40504d5-62d3-4279-bf3f-e19136790ff6 req-2b625c93-60d7-49b3-b521-76811d45b0e1 service nova] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Refreshing network info cache for port fb1b25f9-3a24-43df-805c-9fffe7cd74f7 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:27:47 user nova-compute[72351]: DEBUG nova.network.neutron [req-e40504d5-62d3-4279-bf3f-e19136790ff6 req-2b625c93-60d7-49b3-b521-76811d45b0e1 service nova] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Updated VIF entry in instance network info cache for port fb1b25f9-3a24-43df-805c-9fffe7cd74f7. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:27:47 user nova-compute[72351]: DEBUG nova.network.neutron [req-e40504d5-62d3-4279-bf3f-e19136790ff6 req-2b625c93-60d7-49b3-b521-76811d45b0e1 service nova] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Updating instance_info_cache with network_info: [{"id": "fb1b25f9-3a24-43df-805c-9fffe7cd74f7", "address": "fa:16:3e:66:3c:d2", "network": {"id": "39ddd1d4-5757-425b-92e8-a6a5d8ff29db", "bridge": "br-int", "label": "tempest-TestServerVolumeAttachmentScenario-2122827433-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.175", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "de4a7655f2604c8a9c7964cfd298be37", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfb1b25f9-3a", "ovs_interfaceid": "fb1b25f9-3a24-43df-805c-9fffe7cd74f7", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:27:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-e40504d5-62d3-4279-bf3f-e19136790ff6 req-2b625c93-60d7-49b3-b521-76811d45b0e1 service nova] Releasing lock "refresh_cache-16c1e558-baa1-4c2d-b13b-50585f74d587" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:27:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ea63871b-9e69-44a7-af47-ee0b02817848 tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Acquiring lock "16c1e558-baa1-4c2d-b13b-50585f74d587" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:27:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ea63871b-9e69-44a7-af47-ee0b02817848 tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Lock "16c1e558-baa1-4c2d-b13b-50585f74d587" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:27:47 user nova-compute[72351]: DEBUG nova.objects.instance [None req-ea63871b-9e69-44a7-af47-ee0b02817848 tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Lazy-loading 'flavor' on Instance uuid 16c1e558-baa1-4c2d-b13b-50585f74d587 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:27:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ea63871b-9e69-44a7-af47-ee0b02817848 tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Lock "16c1e558-baa1-4c2d-b13b-50585f74d587" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.099s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:27:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ea63871b-9e69-44a7-af47-ee0b02817848 tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Acquiring lock "16c1e558-baa1-4c2d-b13b-50585f74d587" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:27:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ea63871b-9e69-44a7-af47-ee0b02817848 tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Lock "16c1e558-baa1-4c2d-b13b-50585f74d587" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:27:47 user nova-compute[72351]: INFO nova.compute.manager [None req-ea63871b-9e69-44a7-af47-ee0b02817848 tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Attaching volume a52af854-9d8e-4ec7-935a-b7b7b65d3950 to /dev/vdb
Aug 09 02:27:47 user nova-compute[72351]: DEBUG os_brick.utils [None req-ea63871b-9e69-44a7-af47-ee0b02817848 tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:27:47 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:47 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:47 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[cc65aeb4-87e1-4ca6-bf3e-e4e832ebd905]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:a52269602ff4\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:47 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:48 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:48 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[faf0fe8b-1522-42f1-a602-6b88ae2466b4]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:48 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:48 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:48 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[85364b81-966d-4cd4-aa50-a108304e435d]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:48 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[8b630768-ac31-4352-84c3-ad1578dd151b]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:48 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-ea63871b-9e69-44a7-af47-ee0b02817848 tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Running cmd (subprocess): nvme version {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:48 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-ea63871b-9e69-44a7-af47-ee0b02817848 tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] 'nvme version' failed. Not Retrying. {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:27:48 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:48 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.nvmeof [None req-ea63871b-9e69-44a7-af47-ee0b02817848 tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] nvme not present on system {{(pid=72351) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Aug 09 02:27:48 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:48 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:27:48 user nova-compute[72351]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Aug 09 02:27:48 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[688ddc48-cdcc-4d24-81b0-d76ea0755a3d]: (4, '') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:48 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-ea63871b-9e69-44a7-af47-ee0b02817848 tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72351) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Aug 09 02:27:48 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-ea63871b-9e69-44a7-af47-ee0b02817848 tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Aug 09 02:27:48 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-ea63871b-9e69-44a7-af47-ee0b02817848 tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] LIGHTOS: no hostnqn found. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Aug 09 02:27:48 user nova-compute[72351]: DEBUG os_brick.utils [None req-ea63871b-9e69-44a7-af47-ee0b02817848 tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] <== get_connector_properties: return (140ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:a52269602ff4', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:27:48 user nova-compute[72351]: DEBUG nova.virt.block_device [None req-ea63871b-9e69-44a7-af47-ee0b02817848 tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Updating existing volume attachment record: 996db8c0-5fcc-428e-ad84-f4a5b95d2439 {{(pid=72351) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Aug 09 02:27:51 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquiring lock "2a3bcdea-be36-4a5b-93cb-53396efdfa8e" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:27:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "2a3bcdea-be36-4a5b-93cb-53396efdfa8e" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:27:52 user nova-compute[72351]: DEBUG nova.compute.manager [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Starting instance... {{(pid=72351) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2406}}
Aug 09 02:27:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:27:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:27:52 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72351) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Aug 09 02:27:52 user nova-compute[72351]: INFO nova.compute.claims [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Claim successful on node user
Aug 09 02:27:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:53 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:27:53 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:27:53 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.566s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:27:53 user nova-compute[72351]: DEBUG nova.compute.manager [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Start building networks asynchronously for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2803}}
Aug 09 02:27:53 user nova-compute[72351]: DEBUG nova.compute.manager [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Allocating IP information in the background. {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1956}}
Aug 09 02:27:53 user nova-compute[72351]: DEBUG nova.network.neutron [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] allocate_for_instance() {{(pid=72351) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Aug 09 02:27:53 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Aug 09 02:27:53 user nova-compute[72351]: DEBUG nova.compute.manager [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Start building block device mappings for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2838}}
Aug 09 02:27:53 user nova-compute[72351]: INFO nova.virt.block_device [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Booting with volume 4113a563-77a5-4b68-9198-9d0434737448 at /dev/vda
Aug 09 02:27:53 user nova-compute[72351]: DEBUG nova.policy [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '07d8d5a634274df19c2db769d2d27b29', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4c6bbfd8d904ba1ac69ba5da4564513', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72351) authorize /opt/stack/nova/nova/policy.py:203}}
Aug 09 02:27:53 user nova-compute[72351]: DEBUG os_brick.utils [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:27:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.016s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:53 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[c94798ed-1e37-4dae-b869-e121f56afaa7]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:a52269602ff4\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:53 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[1754d446-1932-4d19-953d-53cfcc21011f]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:53 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ccf16640-89b6-4c61-b5d2-b8c1a3c3fb89]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:53 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[111fa893-e7fd-4952-83e5-6d3007fc81c0]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Running cmd (subprocess): nvme version {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] 'nvme version' failed. Not Retrying. {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:27:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.nvmeof [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] nvme not present on system {{(pid=72351) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Aug 09 02:27:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:27:53 user nova-compute[72351]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Aug 09 02:27:53 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[228f612a-5826-46f6-95b5-a8992a001ab6]: (4, '') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72351) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Aug 09 02:27:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Aug 09 02:27:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] LIGHTOS: no hostnqn found. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Aug 09 02:27:53 user nova-compute[72351]: DEBUG os_brick.utils [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] <== get_connector_properties: return (145ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:a52269602ff4', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:27:53 user nova-compute[72351]: DEBUG nova.virt.block_device [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Updating existing volume attachment record: 58219aa6-8298-444b-95bb-5b4112d23ae8 {{(pid=72351) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Aug 09 02:27:54 user nova-compute[72351]: DEBUG nova.network.neutron [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Successfully created port: 4d36d764-e86b-40ee-81ed-a9f3adfadc48 {{(pid=72351) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Aug 09 02:27:55 user nova-compute[72351]: DEBUG nova.network.neutron [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Successfully updated port: 4d36d764-e86b-40ee-81ed-a9f3adfadc48 {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:27:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquiring lock "refresh_cache-2a3bcdea-be36-4a5b-93cb-53396efdfa8e" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:27:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquired lock "refresh_cache-2a3bcdea-be36-4a5b-93cb-53396efdfa8e" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:27:55 user nova-compute[72351]: DEBUG nova.network.neutron [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:27:55 user nova-compute[72351]: DEBUG nova.compute.manager [req-a2a447c1-1bea-440e-a4f1-4650e3c8a038 req-4c7981bb-0590-4b0c-a5c9-73ff32703b3e service nova] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Received event network-changed-4d36d764-e86b-40ee-81ed-a9f3adfadc48 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:27:55 user nova-compute[72351]: DEBUG nova.compute.manager [req-a2a447c1-1bea-440e-a4f1-4650e3c8a038 req-4c7981bb-0590-4b0c-a5c9-73ff32703b3e service nova] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Refreshing instance network info cache due to event network-changed-4d36d764-e86b-40ee-81ed-a9f3adfadc48. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:27:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a2a447c1-1bea-440e-a4f1-4650e3c8a038 req-4c7981bb-0590-4b0c-a5c9-73ff32703b3e service nova] Acquiring lock "refresh_cache-2a3bcdea-be36-4a5b-93cb-53396efdfa8e" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:27:55 user nova-compute[72351]: DEBUG nova.network.neutron [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:27:56 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:56 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._run_pending_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:27:56 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Cleaning up deleted instances {{(pid=72351) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11129}}
Aug 09 02:27:56 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] There are 0 instances to clean {{(pid=72351) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11138}}
Aug 09 02:27:56 user nova-compute[72351]: DEBUG nova.network.neutron [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Updating instance_info_cache with network_info: [{"id": "4d36d764-e86b-40ee-81ed-a9f3adfadc48", "address": "fa:16:3e:28:03:d9", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4d36d764-e8", "ovs_interfaceid": "4d36d764-e86b-40ee-81ed-a9f3adfadc48", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:27:56 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Releasing lock "refresh_cache-2a3bcdea-be36-4a5b-93cb-53396efdfa8e" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:27:56 user nova-compute[72351]: DEBUG nova.compute.manager [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Instance network_info: |[{"id": "4d36d764-e86b-40ee-81ed-a9f3adfadc48", "address": "fa:16:3e:28:03:d9", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4d36d764-e8", "ovs_interfaceid": "4d36d764-e86b-40ee-81ed-a9f3adfadc48", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1971}}
Aug 09 02:27:56 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a2a447c1-1bea-440e-a4f1-4650e3c8a038 req-4c7981bb-0590-4b0c-a5c9-73ff32703b3e service nova] Acquired lock "refresh_cache-2a3bcdea-be36-4a5b-93cb-53396efdfa8e" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:27:56 user nova-compute[72351]: DEBUG nova.network.neutron [req-a2a447c1-1bea-440e-a4f1-4650e3c8a038 req-4c7981bb-0590-4b0c-a5c9-73ff32703b3e service nova] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Refreshing network info cache for port 4d36d764-e86b-40ee-81ed-a9f3adfadc48 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:27:56 user nova-compute[72351]: DEBUG nova.network.neutron [req-a2a447c1-1bea-440e-a4f1-4650e3c8a038 req-4c7981bb-0590-4b0c-a5c9-73ff32703b3e service nova] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Updated VIF entry in instance network info cache for port 4d36d764-e86b-40ee-81ed-a9f3adfadc48. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:27:56 user nova-compute[72351]: DEBUG nova.network.neutron [req-a2a447c1-1bea-440e-a4f1-4650e3c8a038 req-4c7981bb-0590-4b0c-a5c9-73ff32703b3e service nova] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Updating instance_info_cache with network_info: [{"id": "4d36d764-e86b-40ee-81ed-a9f3adfadc48", "address": "fa:16:3e:28:03:d9", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4d36d764-e8", "ovs_interfaceid": "4d36d764-e86b-40ee-81ed-a9f3adfadc48", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:27:56 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a2a447c1-1bea-440e-a4f1-4650e3c8a038 req-4c7981bb-0590-4b0c-a5c9-73ff32703b3e service nova] Releasing lock "refresh_cache-2a3bcdea-be36-4a5b-93cb-53396efdfa8e" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:27:57 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-ea63871b-9e69-44a7-af47-ee0b02817848 tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 02:27:57 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-ea63871b-9e69-44a7-af47-ee0b02817848 tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'kyoaqdZV', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:27:57 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-ea63871b-9e69-44a7-af47-ee0b02817848 tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:27:57 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-ea63871b-9e69-44a7-af47-ee0b02817848 tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:27:57 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-ea63871b-9e69-44a7-af47-ee0b02817848 tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:27:57 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-ea63871b-9e69-44a7-af47-ee0b02817848 tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:27:57 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-ea63871b-9e69-44a7-af47-ee0b02817848 tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:27:57 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-ea63871b-9e69-44a7-af47-ee0b02817848 tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:27:57 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-ea63871b-9e69-44a7-af47-ee0b02817848 tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 02:27:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950 -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950 -p 172.16.0.220:3260" returned: 21 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:57 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[77605f8e-180a-4e8e-9ed7-c3024de5debe]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:57 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-ea63871b-9e69-44a7-af47-ee0b02817848 tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Aug 09 02:27:57 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:27:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950 -p 172.16.0.220:3260 --interface default --op new {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:57 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[06693f7e-4411-4005-a811-f1b9fc15e89a]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950] added\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:57 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-ea63871b-9e69-44a7-af47-ee0b02817848 tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950] added
Aug 09 02:27:57 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:27:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.027s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:57 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[0f4b83e3-c798-48da-981e-1870705d9377]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:57 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-ea63871b-9e69-44a7-af47-ee0b02817848 tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:27:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:57 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[0572dda5-6950-42df-87a4-52224f1183d3]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:57 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-ea63871b-9e69-44a7-af47-ee0b02817848 tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:27:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v kyoaqdZV {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v kyoaqdZV" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:57 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ea62c2f1-00eb-4268-8dc9-02f0d96457a7]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:57 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-ea63871b-9e69-44a7-af47-ee0b02817848 tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'kyoaqdZV'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:27:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:57 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[1f515cd7-be68-405a-9dc0-865cf3d3719e]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:57 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-ea63871b-9e69-44a7-af47-ee0b02817848 tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:27:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:57 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[3a1c9cfd-f1fa-4fe7-8966-1c5c5520d564]: (4, ('tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)\ntcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)\ntcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:57 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-ea63871b-9e69-44a7-af47-ee0b02817848 tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)
Aug 09 02:27:57 user nova-compute[72351]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)
Aug 09 02:27:57 user nova-compute[72351]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:27:57 user nova-compute[72351]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:27:57 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:27:57 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-ea63871b-9e69-44a7-af47-ee0b02817848 tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] iscsi session list stdout=tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)
Aug 09 02:27:57 user nova-compute[72351]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)
Aug 09 02:27:57 user nova-compute[72351]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:27:57 user nova-compute[72351]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:27:57 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:27:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950 -p 172.16.0.220:3260 --login {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950 -p 172.16.0.220:3260 --login" returned: 0 in 0.076s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:57 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[2efc3e9f-cb98-4247-a1ae-e9228f87b03f]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:57 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-ea63871b-9e69-44a7-af47-ee0b02817848 tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950, portal: 172.16.0.220,3260]
Aug 09 02:27:57 user nova-compute[72351]: Login to [iface: default, target: iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950, portal: 172.16.0.220,3260] successful.
Aug 09 02:27:57 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:27:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.041s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:57 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[f601218f-aac1-42f3-b49d-234a37d33267]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:57 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-ea63871b-9e69-44a7-af47-ee0b02817848 tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:27:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.028s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:57 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[44556d1a-485b-4747-8946-87959d64f453]: (4, ('tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)\ntcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)\ntcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)\ntcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:57 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-ea63871b-9e69-44a7-af47-ee0b02817848 tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)
Aug 09 02:27:57 user nova-compute[72351]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)
Aug 09 02:27:57 user nova-compute[72351]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:27:57 user nova-compute[72351]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:27:57 user nova-compute[72351]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950 (non-flash)
Aug 09 02:27:57 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:27:57 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-ea63871b-9e69-44a7-af47-ee0b02817848 tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] iscsi session list stdout=tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)
Aug 09 02:27:57 user nova-compute[72351]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)
Aug 09 02:27:57 user nova-compute[72351]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:27:57 user nova-compute[72351]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:27:57 user nova-compute[72351]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950 (non-flash)
Aug 09 02:27:57 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:27:57 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-ea63871b-9e69-44a7-af47-ee0b02817848 tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.334s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:27:57 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-ea63871b-9e69-44a7-af47-ee0b02817848 tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 02:27:57 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-ea63871b-9e69-44a7-af47-ee0b02817848 tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] HCTL ('34', '-', '-', 0) found on session 29 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 02:27:57 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-ea63871b-9e69-44a7-af47-ee0b02817848 tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Scanning host 34 c: -, t: -, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 02:27:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:27:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:27:57 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d92e40fe-1e92-44c4-b520-1f6be0207688]: (4, ('- - 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:27:57 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-ea63871b-9e69-44a7-af47-ee0b02817848 tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Searching for a device in session 29 and hctl ['34', '*', '*', 0] yield: sdc {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:27:57 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-ea63871b-9e69-44a7-af47-ee0b02817848 tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Connected to sdc using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'kyoaqdZV', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 02:27:58 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:27:58 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-ea63871b-9e69-44a7-af47-ee0b02817848 tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.371s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:27:58 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-ea63871b-9e69-44a7-af47-ee0b02817848 tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] <== connect_volume: return (1373ms) {'type': 'block', 'scsi_wwn': '23235383661616436', 'path': '/dev/sdc'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:27:58 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-ea63871b-9e69-44a7-af47-ee0b02817848 tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23235383661616436', 'path': '/dev/sdc'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 02:27:58 user nova-compute[72351]: DEBUG nova.objects.instance [None req-ea63871b-9e69-44a7-af47-ee0b02817848 tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Lazy-loading 'flavor' on Instance uuid 16c1e558-baa1-4c2d-b13b-50585f74d587 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:27:58 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-ea63871b-9e69-44a7-af47-ee0b02817848 tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] attach device xml: <disk type="block" device="disk">
Aug 09 02:27:58 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:27:58 user nova-compute[72351]:   <source dev="/dev/sdc"/>
Aug 09 02:27:58 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:27:58 user nova-compute[72351]:   <serial>a52af854-9d8e-4ec7-935a-b7b7b65d3950</serial>
Aug 09 02:27:58 user nova-compute[72351]: </disk>
Aug 09 02:27:58 user nova-compute[72351]:  {{(pid=72351) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Aug 09 02:27:59 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:27:59 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-ea63871b-9e69-44a7-af47-ee0b02817848 tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:27:59 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-ea63871b-9e69-44a7-af47-ee0b02817848 tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:27:59 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-ea63871b-9e69-44a7-af47-ee0b02817848 tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] No VIF found with MAC fa:16:3e:66:3c:d2, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:27:59 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ea63871b-9e69-44a7-af47-ee0b02817848 tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Lock "16c1e558-baa1-4c2d-b13b-50585f74d587" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 11.532s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:27:59 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:27:59 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72351) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10461}}
Aug 09 02:27:59 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:27:59 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:27:59 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:27:59 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:27:59 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Auditing locally available compute resources for user (node: user) {{(pid=72351) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Aug 09 02:28:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json" returned: 0 in 0.144s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json" returned: 0 in 0.173s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:00 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdf (vdb) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:28:00 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdg (vda) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:28:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/47afce95-e5f7-4703-a99f-7ef8b28c4bd2/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/47afce95-e5f7-4703-a99f-7ef8b28c4bd2/disk --force-share --output=json" returned: 0 in 0.140s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/47afce95-e5f7-4703-a99f-7ef8b28c4bd2/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/47afce95-e5f7-4703-a99f-7ef8b28c4bd2/disk --force-share --output=json" returned: 0 in 0.165s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.rescue --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.rescue --force-share --output=json" returned: 0 in 0.133s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.rescue --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.rescue --force-share --output=json" returned: 0 in 0.133s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:01 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:28:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json" returned: 0 in 0.197s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f457e6ea-8d06-46c6-be2e-f650e28a3b69/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f457e6ea-8d06-46c6-be2e-f650e28a3b69/disk --force-share --output=json" returned: 0 in 0.129s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f457e6ea-8d06-46c6-be2e-f650e28a3b69/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f457e6ea-8d06-46c6-be2e-f650e28a3b69/disk --force-share --output=json" returned: 0 in 0.144s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk.rescue --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk.rescue --force-share --output=json" returned: 0 in 0.152s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk.rescue --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk.rescue --force-share --output=json" returned: 0 in 0.186s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json" returned: 0 in 0.155s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/16c1e558-baa1-4c2d-b13b-50585f74d587/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-da51dea8-da0b-4092-9e24-e5062241f50c tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Acquiring lock "16c1e558-baa1-4c2d-b13b-50585f74d587" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:28:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-da51dea8-da0b-4092-9e24-e5062241f50c tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Lock "16c1e558-baa1-4c2d-b13b-50585f74d587" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:28:02 user nova-compute[72351]: DEBUG nova.objects.instance [None req-da51dea8-da0b-4092-9e24-e5062241f50c tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Lazy-loading 'flavor' on Instance uuid 16c1e558-baa1-4c2d-b13b-50585f74d587 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:28:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/16c1e558-baa1-4c2d-b13b-50585f74d587/disk --force-share --output=json" returned: 0 in 0.170s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/16c1e558-baa1-4c2d-b13b-50585f74d587/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-da51dea8-da0b-4092-9e24-e5062241f50c tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Lock "16c1e558-baa1-4c2d-b13b-50585f74d587" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.096s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:28:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/16c1e558-baa1-4c2d-b13b-50585f74d587/disk --force-share --output=json" returned: 0 in 0.153s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:02 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdc (vdb) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:28:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-da51dea8-da0b-4092-9e24-e5062241f50c tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Acquiring lock "16c1e558-baa1-4c2d-b13b-50585f74d587" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:28:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-da51dea8-da0b-4092-9e24-e5062241f50c tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Lock "16c1e558-baa1-4c2d-b13b-50585f74d587" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:28:02 user nova-compute[72351]: INFO nova.compute.manager [None req-da51dea8-da0b-4092-9e24-e5062241f50c tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Attaching volume 336fe492-aeb4-4440-9802-e98e8408a431 to /dev/vdc
Aug 09 02:28:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json" returned: 0 in 0.150s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:02 user nova-compute[72351]: DEBUG os_brick.utils [None req-da51dea8-da0b-4092-9e24-e5062241f50c tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:28:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.013s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:02 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[1d117b41-9d0c-4b16-b8f2-4be73210b219]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:a52269602ff4\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:28:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:02 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[07d8f79e-530b-45bb-a6a8-d71222f56b14]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:28:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:02 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[16244454-e490-4d05-9aad-11071df1b255]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:28:02 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[4dc97e52-5e11-47e2-9cda-4edb0c62260f]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:28:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-da51dea8-da0b-4092-9e24-e5062241f50c tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Running cmd (subprocess): nvme version {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-da51dea8-da0b-4092-9e24-e5062241f50c tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] 'nvme version' failed. Not Retrying. {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:28:03 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.nvmeof [None req-da51dea8-da0b-4092-9e24-e5062241f50c tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] nvme not present on system {{(pid=72351) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Aug 09 02:28:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json" returned: 0 in 0.155s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1b90316d-6638-4603-bb09-38bf0ed795e3/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:28:03 user nova-compute[72351]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Aug 09 02:28:03 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a3ca2b6f-e500-43d2-9dec-9e99297a9336]: (4, '') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:28:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:28:03 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-da51dea8-da0b-4092-9e24-e5062241f50c tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72351) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Aug 09 02:28:03 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-da51dea8-da0b-4092-9e24-e5062241f50c tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Aug 09 02:28:03 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-da51dea8-da0b-4092-9e24-e5062241f50c tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] LIGHTOS: no hostnqn found. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Aug 09 02:28:03 user nova-compute[72351]: DEBUG os_brick.utils [None req-da51dea8-da0b-4092-9e24-e5062241f50c tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] <== get_connector_properties: return (172ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:a52269602ff4', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:28:03 user nova-compute[72351]: DEBUG nova.virt.block_device [None req-da51dea8-da0b-4092-9e24-e5062241f50c tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Updating existing volume attachment record: 0c9aa577-602f-4c54-a580-f57488b88142 {{(pid=72351) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Aug 09 02:28:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1b90316d-6638-4603-bb09-38bf0ed795e3/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1b90316d-6638-4603-bb09-38bf0ed795e3/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1b90316d-6638-4603-bb09-38bf0ed795e3/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff/disk --force-share --output=json" returned: 0 in 0.144s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:03 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdb (vdb) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:28:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a03fbded-23ad-440b-97e7-bea07425e873/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a03fbded-23ad-440b-97e7-bea07425e873/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a03fbded-23ad-440b-97e7-bea07425e873/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a03fbded-23ad-440b-97e7-bea07425e873/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/831c864a-75d9-42b3-b952-749f4201e40e/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/831c864a-75d9-42b3-b952-749f4201e40e/disk --force-share --output=json" returned: 0 in 0.140s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/831c864a-75d9-42b3-b952-749f4201e40e/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/831c864a-75d9-42b3-b952-749f4201e40e/disk --force-share --output=json" returned: 0 in 0.133s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:04 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sde (vdb) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:28:04 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:28:04 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:28:04 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Hypervisor/Node resource view: name=user free_ram=6819MB free_disk=25.293556213378906GB free_vcpus=0 pci_devices=[{"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72351) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Aug 09 02:28:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:28:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:28:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a992a852-b4b4-4efe-b63f-241df5631faf tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Acquiring lock "f457e6ea-8d06-46c6-be2e-f650e28a3b69" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:28:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a992a852-b4b4-4efe-b63f-241df5631faf tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Lock "f457e6ea-8d06-46c6-be2e-f650e28a3b69" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:28:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a992a852-b4b4-4efe-b63f-241df5631faf tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Acquiring lock "f457e6ea-8d06-46c6-be2e-f650e28a3b69-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:28:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a992a852-b4b4-4efe-b63f-241df5631faf tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Lock "f457e6ea-8d06-46c6-be2e-f650e28a3b69-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:28:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a992a852-b4b4-4efe-b63f-241df5631faf tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Lock "f457e6ea-8d06-46c6-be2e-f650e28a3b69-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:28:04 user nova-compute[72351]: INFO nova.compute.manager [None req-a992a852-b4b4-4efe-b63f-241df5631faf tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Terminating instance
Aug 09 02:28:04 user nova-compute[72351]: DEBUG nova.compute.manager [None req-a992a852-b4b4-4efe-b63f-241df5631faf tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Start destroying the instance on the hypervisor. {{(pid=72351) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3124}}
Aug 09 02:28:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:28:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:28:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:28:04 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:28:04 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance eb86cb49-2018-42c2-bd97-c662b829775d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:28:04 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:28:04 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance a7f3d076-864b-4b68-9207-214ce2c21444 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:28:04 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 831c864a-75d9-42b3-b952-749f4201e40e actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:28:04 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance f457e6ea-8d06-46c6-be2e-f650e28a3b69 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:28:04 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 75bbbead-8628-448b-9484-19bf4ab922ff actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:28:04 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 47afce95-e5f7-4703-a99f-7ef8b28c4bd2 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:28:04 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 16c1e558-baa1-4c2d-b13b-50585f74d587 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:28:04 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 957c1510-1320-4492-9052-a58ff540319d actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:28:04 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 1b90316d-6638-4603-bb09-38bf0ed795e3 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:28:04 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance a03fbded-23ad-440b-97e7-bea07425e873 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:28:04 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 2a3bcdea-be36-4a5b-93cb-53396efdfa8e actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:28:04 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Total usable vcpus: 12, total allocated vcpus: 13 {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Aug 09 02:28:04 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Final resource view: name=user phys_ram=16011MB used_ram=7168MB phys_disk=40GB used_disk=11GB total_vcpus=12 used_vcpus=13 pci_stats=[] {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Aug 09 02:28:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:28:05 user nova-compute[72351]: DEBUG nova.compute.manager [req-b55a5736-f94a-400c-b253-dd718071555a req-445ae7ad-b191-434f-ae55-8ce31c931854 service nova] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Received event network-vif-unplugged-dc244cf9-b0f6-4906-b9f3-53634b1847e3 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:28:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-b55a5736-f94a-400c-b253-dd718071555a req-445ae7ad-b191-434f-ae55-8ce31c931854 service nova] Acquiring lock "f457e6ea-8d06-46c6-be2e-f650e28a3b69-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:28:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-b55a5736-f94a-400c-b253-dd718071555a req-445ae7ad-b191-434f-ae55-8ce31c931854 service nova] Lock "f457e6ea-8d06-46c6-be2e-f650e28a3b69-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:28:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-b55a5736-f94a-400c-b253-dd718071555a req-445ae7ad-b191-434f-ae55-8ce31c931854 service nova] Lock "f457e6ea-8d06-46c6-be2e-f650e28a3b69-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:28:05 user nova-compute[72351]: DEBUG nova.compute.manager [req-b55a5736-f94a-400c-b253-dd718071555a req-445ae7ad-b191-434f-ae55-8ce31c931854 service nova] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] No waiting events found dispatching network-vif-unplugged-dc244cf9-b0f6-4906-b9f3-53634b1847e3 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:28:05 user nova-compute[72351]: DEBUG nova.compute.manager [req-b55a5736-f94a-400c-b253-dd718071555a req-445ae7ad-b191-434f-ae55-8ce31c931854 service nova] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Received event network-vif-unplugged-dc244cf9-b0f6-4906-b9f3-53634b1847e3 for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:28:05 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:28:05 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:28:05 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Compute_service record updated for user:user {{(pid=72351) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Aug 09 02:28:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.681s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:28:05 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Instance destroyed successfully.
Aug 09 02:28:05 user nova-compute[72351]: DEBUG nova.objects.instance [None req-a992a852-b4b4-4efe-b63f-241df5631faf tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Lazy-loading 'resources' on Instance uuid f457e6ea-8d06-46c6-be2e-f650e28a3b69 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:28:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-a992a852-b4b4-4efe-b63f-241df5631faf tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-08-09T02:24:58Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-443615500',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-443615500',id=25,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBK8nrHk7zghGHoCUEI3vExNw2oXJc2YwDe7QrMxgC2DNUP/PkVzpLwy9DWnyTLCCR4ddGxe1t7xIbXts5zbiafpc3PmU6by4kyzLopbWZ1BYcJyQcmguVSz4dz8HMbCR9g==',key_name='tempest-keypair-1705746290',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:25:05Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='bde0634670914ae8bfbbc6b31c09d6dd',ramdisk_id='',reservation_id='r-5xxup5ze',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedAttachmentsTest-480515778',owner_user_name='tempest-TaggedAttachmentsTest-480515778-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:25:06Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='8159c9e78af04e01a1faf9b3cfc3591e',uuid=f457e6ea-8d06-46c6-be2e-f650e28a3b69,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "dc244cf9-b0f6-4906-b9f3-53634b1847e3", "address": "fa:16:3e:0e:77:88", "network": {"id": "3fdd9499-7552-4ecc-a897-ed4c58be9519", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-969934765-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.20", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bde0634670914ae8bfbbc6b31c09d6dd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdc244cf9-b0", "ovs_interfaceid": "dc244cf9-b0f6-4906-b9f3-53634b1847e3", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:28:05 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-a992a852-b4b4-4efe-b63f-241df5631faf tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Converting VIF {"id": "dc244cf9-b0f6-4906-b9f3-53634b1847e3", "address": "fa:16:3e:0e:77:88", "network": {"id": "3fdd9499-7552-4ecc-a897-ed4c58be9519", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-969934765-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.20", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bde0634670914ae8bfbbc6b31c09d6dd", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdc244cf9-b0", "ovs_interfaceid": "dc244cf9-b0f6-4906-b9f3-53634b1847e3", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:28:05 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-a992a852-b4b4-4efe-b63f-241df5631faf tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:0e:77:88,bridge_name='br-int',has_traffic_filtering=True,id=dc244cf9-b0f6-4906-b9f3-53634b1847e3,network=Network(3fdd9499-7552-4ecc-a897-ed4c58be9519),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdc244cf9-b0') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:28:05 user nova-compute[72351]: DEBUG os_vif [None req-a992a852-b4b4-4efe-b63f-241df5631faf tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:0e:77:88,bridge_name='br-int',has_traffic_filtering=True,id=dc244cf9-b0f6-4906-b9f3-53634b1847e3,network=Network(3fdd9499-7552-4ecc-a897-ed4c58be9519),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdc244cf9-b0') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:28:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:28:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapdc244cf9-b0, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:28:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:28:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:28:05 user nova-compute[72351]: INFO os_vif [None req-a992a852-b4b4-4efe-b63f-241df5631faf tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:0e:77:88,bridge_name='br-int',has_traffic_filtering=True,id=dc244cf9-b0f6-4906-b9f3-53634b1847e3,network=Network(3fdd9499-7552-4ecc-a897-ed4c58be9519),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdc244cf9-b0')
Aug 09 02:28:05 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-a992a852-b4b4-4efe-b63f-241df5631faf tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Deleting instance files /opt/stack/data/nova/instances/f457e6ea-8d06-46c6-be2e-f650e28a3b69_del
Aug 09 02:28:05 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-a992a852-b4b4-4efe-b63f-241df5631faf tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Deletion of /opt/stack/data/nova/instances/f457e6ea-8d06-46c6-be2e-f650e28a3b69_del complete
Aug 09 02:28:05 user nova-compute[72351]: INFO nova.compute.manager [None req-a992a852-b4b4-4efe-b63f-241df5631faf tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Took 0.89 seconds to destroy the instance on the hypervisor.
Aug 09 02:28:05 user nova-compute[72351]: DEBUG oslo.service.loopingcall [None req-a992a852-b4b4-4efe-b63f-241df5631faf tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72351) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Aug 09 02:28:05 user nova-compute[72351]: DEBUG nova.compute.manager [-] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Deallocating network for instance {{(pid=72351) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2263}}
Aug 09 02:28:05 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] deallocate_for_instance() {{(pid=72351) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1795}}
Aug 09 02:28:05 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:28:05 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:28:05 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Starting heal instance info cache {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9842}}
Aug 09 02:28:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "refresh_cache-a7f3d076-864b-4b68-9207-214ce2c21444" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:28:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquired lock "refresh_cache-a7f3d076-864b-4b68-9207-214ce2c21444" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:28:05 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Forcefully refreshing network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1996}}
Aug 09 02:28:06 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Updating instance_info_cache with network_info: [{"id": "64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f", "address": "fa:16:3e:ed:9c:88", "network": {"id": "9a452414-d61f-4045-a6cc-9be93c892c4c", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "321111f935dd4e8aab4a083206ea9a7e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap64bd80e2-6c", "ovs_interfaceid": "64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:28:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Releasing lock "refresh_cache-a7f3d076-864b-4b68-9207-214ce2c21444" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:28:06 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Updated the network info_cache for instance {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9913}}
Aug 09 02:28:06 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:28:06 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:28:06 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:28:06 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._cleanup_incomplete_migrations {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:28:06 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Cleaning up deleted instances with incomplete migration  {{(pid=72351) _cleanup_incomplete_migrations /opt/stack/nova/nova/compute/manager.py:11167}}
Aug 09 02:28:06 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:28:06 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Took 0.99 seconds to deallocate network for instance.
Aug 09 02:28:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a992a852-b4b4-4efe-b63f-241df5631faf tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:28:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a992a852-b4b4-4efe-b63f-241df5631faf tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:28:06 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:28:07 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-a992a852-b4b4-4efe-b63f-241df5631faf tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:28:07 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-a992a852-b4b4-4efe-b63f-241df5631faf tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:28:07 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a992a852-b4b4-4efe-b63f-241df5631faf tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.448s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:28:07 user nova-compute[72351]: DEBUG nova.compute.manager [req-e91d209e-b3bb-4e30-be2a-05fe1ea8d0ee req-eccbb4ea-6f2f-423c-83d0-c97d2e69681c service nova] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Received event network-vif-plugged-dc244cf9-b0f6-4906-b9f3-53634b1847e3 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:28:07 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-e91d209e-b3bb-4e30-be2a-05fe1ea8d0ee req-eccbb4ea-6f2f-423c-83d0-c97d2e69681c service nova] Acquiring lock "f457e6ea-8d06-46c6-be2e-f650e28a3b69-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:28:07 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-e91d209e-b3bb-4e30-be2a-05fe1ea8d0ee req-eccbb4ea-6f2f-423c-83d0-c97d2e69681c service nova] Lock "f457e6ea-8d06-46c6-be2e-f650e28a3b69-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:28:07 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-e91d209e-b3bb-4e30-be2a-05fe1ea8d0ee req-eccbb4ea-6f2f-423c-83d0-c97d2e69681c service nova] Lock "f457e6ea-8d06-46c6-be2e-f650e28a3b69-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:28:07 user nova-compute[72351]: DEBUG nova.compute.manager [req-e91d209e-b3bb-4e30-be2a-05fe1ea8d0ee req-eccbb4ea-6f2f-423c-83d0-c97d2e69681c service nova] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] No waiting events found dispatching network-vif-plugged-dc244cf9-b0f6-4906-b9f3-53634b1847e3 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:28:07 user nova-compute[72351]: WARNING nova.compute.manager [req-e91d209e-b3bb-4e30-be2a-05fe1ea8d0ee req-eccbb4ea-6f2f-423c-83d0-c97d2e69681c service nova] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Received unexpected event network-vif-plugged-dc244cf9-b0f6-4906-b9f3-53634b1847e3 for instance with vm_state deleted and task_state None.
Aug 09 02:28:07 user nova-compute[72351]: DEBUG nova.compute.manager [req-e91d209e-b3bb-4e30-be2a-05fe1ea8d0ee req-eccbb4ea-6f2f-423c-83d0-c97d2e69681c service nova] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Received event network-vif-deleted-dc244cf9-b0f6-4906-b9f3-53634b1847e3 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:28:07 user nova-compute[72351]: INFO nova.scheduler.client.report [None req-a992a852-b4b4-4efe-b63f-241df5631faf tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Deleted allocations for instance f457e6ea-8d06-46c6-be2e-f650e28a3b69
Aug 09 02:28:07 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a992a852-b4b4-4efe-b63f-241df5631faf tempest-TaggedAttachmentsTest-480515778 tempest-TaggedAttachmentsTest-480515778-project-member] Lock "f457e6ea-8d06-46c6-be2e-f650e28a3b69" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.718s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:28:07 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._cleanup_expired_console_auth_tokens {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:28:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:28:10 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:28:12 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-da51dea8-da0b-4092-9e24-e5062241f50c tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 02:28:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-da51dea8-da0b-4092-9e24-e5062241f50c tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:336fe492-aeb4-4440-9802-e98e8408a431'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'SdDZPfLw', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:28:12 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-da51dea8-da0b-4092-9e24-e5062241f50c tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:28:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-da51dea8-da0b-4092-9e24-e5062241f50c tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:28:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-da51dea8-da0b-4092-9e24-e5062241f50c tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:28:12 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-da51dea8-da0b-4092-9e24-e5062241f50c tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:28:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-da51dea8-da0b-4092-9e24-e5062241f50c tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:336fe492-aeb4-4440-9802-e98e8408a431" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:28:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-da51dea8-da0b-4092-9e24-e5062241f50c tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:336fe492-aeb4-4440-9802-e98e8408a431" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:28:12 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-da51dea8-da0b-4092-9e24-e5062241f50c tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 02:28:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:336fe492-aeb4-4440-9802-e98e8408a431 -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:336fe492-aeb4-4440-9802-e98e8408a431 -p 172.16.0.220:3260" returned: 21 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:12 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[edd88b0f-18f7-4db7-bec3-bc41cfbd2975]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:28:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-da51dea8-da0b-4092-9e24-e5062241f50c tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Aug 09 02:28:12 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:28:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:336fe492-aeb4-4440-9802-e98e8408a431 -p 172.16.0.220:3260 --interface default --op new {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:336fe492-aeb4-4440-9802-e98e8408a431 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:12 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[04051d67-fe62-46a7-a488-ff28b00728a9]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:336fe492-aeb4-4440-9802-e98e8408a431] added\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:28:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-da51dea8-da0b-4092-9e24-e5062241f50c tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:336fe492-aeb4-4440-9802-e98e8408a431] added
Aug 09 02:28:12 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:28:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:336fe492-aeb4-4440-9802-e98e8408a431 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:336fe492-aeb4-4440-9802-e98e8408a431 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:12 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[c74a1217-6a89-43ad-ba11-a0e173043492]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:28:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-da51dea8-da0b-4092-9e24-e5062241f50c tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:28:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:336fe492-aeb4-4440-9802-e98e8408a431 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:336fe492-aeb4-4440-9802-e98e8408a431 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:12 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[48c9cad2-3646-4a40-97f8-d2e8871ea99c]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:28:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-da51dea8-da0b-4092-9e24-e5062241f50c tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:28:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:336fe492-aeb4-4440-9802-e98e8408a431 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v SdDZPfLw {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:336fe492-aeb4-4440-9802-e98e8408a431 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v SdDZPfLw" returned: 0 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:12 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[47f5001d-6c61-4a82-a7f0-ee1c2de06e83]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:28:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-da51dea8-da0b-4092-9e24-e5062241f50c tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'SdDZPfLw'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:28:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:336fe492-aeb4-4440-9802-e98e8408a431 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:336fe492-aeb4-4440-9802-e98e8408a431 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:12 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ababc3f4-c371-45ed-b1e7-e59feae46ee5]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:28:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-da51dea8-da0b-4092-9e24-e5062241f50c tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:28:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:12 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e8393337-2aa9-4df2-baaf-865363dd6212]: (4, ('tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)\ntcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)\ntcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)\ntcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:28:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-da51dea8-da0b-4092-9e24-e5062241f50c tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)
Aug 09 02:28:12 user nova-compute[72351]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)
Aug 09 02:28:12 user nova-compute[72351]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:28:12 user nova-compute[72351]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:28:12 user nova-compute[72351]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950 (non-flash)
Aug 09 02:28:12 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:28:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-da51dea8-da0b-4092-9e24-e5062241f50c tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] iscsi session list stdout=tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)
Aug 09 02:28:12 user nova-compute[72351]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)
Aug 09 02:28:12 user nova-compute[72351]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:28:12 user nova-compute[72351]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:28:12 user nova-compute[72351]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950 (non-flash)
Aug 09 02:28:12 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:28:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:336fe492-aeb4-4440-9802-e98e8408a431 -p 172.16.0.220:3260 --login {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:336fe492-aeb4-4440-9802-e98e8408a431 -p 172.16.0.220:3260 --login" returned: 0 in 0.078s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:12 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[91dbee32-ac0e-4473-89f7-f3b31ac072f2]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:336fe492-aeb4-4440-9802-e98e8408a431, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:336fe492-aeb4-4440-9802-e98e8408a431, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:28:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-da51dea8-da0b-4092-9e24-e5062241f50c tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:336fe492-aeb4-4440-9802-e98e8408a431, portal: 172.16.0.220,3260]
Aug 09 02:28:12 user nova-compute[72351]: Login to [iface: default, target: iqn.2010-10.org.openstack:336fe492-aeb4-4440-9802-e98e8408a431, portal: 172.16.0.220,3260] successful.
Aug 09 02:28:12 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:28:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:336fe492-aeb4-4440-9802-e98e8408a431 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:336fe492-aeb4-4440-9802-e98e8408a431 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.024s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:12 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[2a774c7f-2de4-4e9c-ac77-69e26e6c5628]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:28:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-da51dea8-da0b-4092-9e24-e5062241f50c tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:28:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.030s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:12 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d09c34d3-0a49-48b4-851e-4da3ae16d27e]: (4, ('tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)\ntcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)\ntcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)\ntcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950 (non-flash)\ntcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:336fe492-aeb4-4440-9802-e98e8408a431 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:28:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-da51dea8-da0b-4092-9e24-e5062241f50c tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)
Aug 09 02:28:12 user nova-compute[72351]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)
Aug 09 02:28:12 user nova-compute[72351]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:28:12 user nova-compute[72351]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:28:12 user nova-compute[72351]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950 (non-flash)
Aug 09 02:28:12 user nova-compute[72351]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:336fe492-aeb4-4440-9802-e98e8408a431 (non-flash)
Aug 09 02:28:12 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:28:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-da51dea8-da0b-4092-9e24-e5062241f50c tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] iscsi session list stdout=tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)
Aug 09 02:28:12 user nova-compute[72351]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)
Aug 09 02:28:12 user nova-compute[72351]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:28:12 user nova-compute[72351]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:28:12 user nova-compute[72351]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950 (non-flash)
Aug 09 02:28:12 user nova-compute[72351]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:336fe492-aeb4-4440-9802-e98e8408a431 (non-flash)
Aug 09 02:28:12 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:28:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-da51dea8-da0b-4092-9e24-e5062241f50c tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:336fe492-aeb4-4440-9802-e98e8408a431" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.314s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:28:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-da51dea8-da0b-4092-9e24-e5062241f50c tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 02:28:12 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-da51dea8-da0b-4092-9e24-e5062241f50c tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] HCTL ('35', '-', '-', 0) found on session 30 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 02:28:12 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-da51dea8-da0b-4092-9e24-e5062241f50c tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Scanning host 35 c: -, t: -, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 02:28:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host35/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host35/scan" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:12 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[5d410a1c-c6a8-475a-bde0-89afdbe23c9f]: (4, ('- - 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:28:12 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-da51dea8-da0b-4092-9e24-e5062241f50c tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Searching for a device in session 30 and hctl ['35', '*', '*', 0] yield: sdd {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:28:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-da51dea8-da0b-4092-9e24-e5062241f50c tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Connected to sdd using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'SdDZPfLw', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 02:28:12 user nova-compute[72351]: INFO nova.compute.manager [None req-1e61dfe0-d7d1-41cd-bdb2-9a175ad5ed6e tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Unrescuing
Aug 09 02:28:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1e61dfe0-d7d1-41cd-bdb2-9a175ad5ed6e tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Acquiring lock "refresh_cache-e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:28:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1e61dfe0-d7d1-41cd-bdb2-9a175ad5ed6e tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Acquired lock "refresh_cache-e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:28:12 user nova-compute[72351]: DEBUG nova.network.neutron [None req-1e61dfe0-d7d1-41cd-bdb2-9a175ad5ed6e tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:28:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:28:13 user nova-compute[72351]: DEBUG nova.network.neutron [None req-1e61dfe0-d7d1-41cd-bdb2-9a175ad5ed6e tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Updating instance_info_cache with network_info: [{"id": "9481e457-9c23-4c45-b0bd-1faf63f8b7b8", "address": "fa:16:3e:69:03:30", "network": {"id": "49e8751c-e119-430f-9110-3dc6069b1589", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-899275135-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3aea1e66805146f7a8cd1d97e64a24af", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9481e457-9c", "ovs_interfaceid": "9481e457-9c23-4c45-b0bd-1faf63f8b7b8", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:28:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1e61dfe0-d7d1-41cd-bdb2-9a175ad5ed6e tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Releasing lock "refresh_cache-e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:28:13 user nova-compute[72351]: DEBUG nova.objects.instance [None req-1e61dfe0-d7d1-41cd-bdb2-9a175ad5ed6e tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lazy-loading 'flavor' on Instance uuid e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:28:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-da51dea8-da0b-4092-9e24-e5062241f50c tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.348s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:28:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-da51dea8-da0b-4092-9e24-e5062241f50c tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] <== connect_volume: return (1350ms) {'type': 'block', 'scsi_wwn': '23064393439626662', 'path': '/dev/sdd'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:28:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-da51dea8-da0b-4092-9e24-e5062241f50c tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23064393439626662', 'path': '/dev/sdd'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 02:28:13 user nova-compute[72351]: DEBUG nova.objects.instance [None req-da51dea8-da0b-4092-9e24-e5062241f50c tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Lazy-loading 'flavor' on Instance uuid 16c1e558-baa1-4c2d-b13b-50585f74d587 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:28:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-da51dea8-da0b-4092-9e24-e5062241f50c tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] attach device xml: <disk type="block" device="disk">
Aug 09 02:28:13 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:28:13 user nova-compute[72351]:   <source dev="/dev/sdd"/>
Aug 09 02:28:13 user nova-compute[72351]:   <target dev="vdc" bus="virtio"/>
Aug 09 02:28:13 user nova-compute[72351]:   <serial>336fe492-aeb4-4440-9802-e98e8408a431</serial>
Aug 09 02:28:13 user nova-compute[72351]: </disk>
Aug 09 02:28:13 user nova-compute[72351]:  {{(pid=72351) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Aug 09 02:28:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:28:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:28:13 user nova-compute[72351]: DEBUG nova.compute.manager [req-a0a5bd37-8dd7-488e-94b0-48f4a0ffe00d req-60aa2004-2637-4c85-8c18-b4954d4f117d service nova] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Received event network-vif-unplugged-9481e457-9c23-4c45-b0bd-1faf63f8b7b8 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:28:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a0a5bd37-8dd7-488e-94b0-48f4a0ffe00d req-60aa2004-2637-4c85-8c18-b4954d4f117d service nova] Acquiring lock "e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:28:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a0a5bd37-8dd7-488e-94b0-48f4a0ffe00d req-60aa2004-2637-4c85-8c18-b4954d4f117d service nova] Lock "e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:28:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a0a5bd37-8dd7-488e-94b0-48f4a0ffe00d req-60aa2004-2637-4c85-8c18-b4954d4f117d service nova] Lock "e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:28:13 user nova-compute[72351]: DEBUG nova.compute.manager [req-a0a5bd37-8dd7-488e-94b0-48f4a0ffe00d req-60aa2004-2637-4c85-8c18-b4954d4f117d service nova] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] No waiting events found dispatching network-vif-unplugged-9481e457-9c23-4c45-b0bd-1faf63f8b7b8 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:28:13 user nova-compute[72351]: WARNING nova.compute.manager [req-a0a5bd37-8dd7-488e-94b0-48f4a0ffe00d req-60aa2004-2637-4c85-8c18-b4954d4f117d service nova] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Received unexpected event network-vif-unplugged-9481e457-9c23-4c45-b0bd-1faf63f8b7b8 for instance with vm_state rescued and task_state unrescuing.
Aug 09 02:28:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:28:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:28:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-da51dea8-da0b-4092-9e24-e5062241f50c tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:28:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-da51dea8-da0b-4092-9e24-e5062241f50c tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:28:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-da51dea8-da0b-4092-9e24-e5062241f50c tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] No BDM found with device name vdc, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:28:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-da51dea8-da0b-4092-9e24-e5062241f50c tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] No VIF found with MAC fa:16:3e:66:3c:d2, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:28:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-da51dea8-da0b-4092-9e24-e5062241f50c tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Lock "16c1e558-baa1-4c2d-b13b-50585f74d587" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 11.264s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:28:14 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Instance destroyed successfully.
Aug 09 02:28:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a68687d3-459a-40d9-8044-74ce7508f50e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Acquiring lock "16c1e558-baa1-4c2d-b13b-50585f74d587" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:28:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a68687d3-459a-40d9-8044-74ce7508f50e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Lock "16c1e558-baa1-4c2d-b13b-50585f74d587" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:28:14 user nova-compute[72351]: INFO nova.compute.manager [None req-a68687d3-459a-40d9-8044-74ce7508f50e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Detaching volume 336fe492-aeb4-4440-9802-e98e8408a431
Aug 09 02:28:14 user nova-compute[72351]: INFO nova.virt.block_device [None req-a68687d3-459a-40d9-8044-74ce7508f50e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Attempting to driver detach volume 336fe492-aeb4-4440-9802-e98e8408a431 from mountpoint /dev/vdc
Aug 09 02:28:14 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-a68687d3-459a-40d9-8044-74ce7508f50e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Attempting to detach device vdc from instance 16c1e558-baa1-4c2d-b13b-50585f74d587 from the persistent domain config. {{(pid=72351) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2477}}
Aug 09 02:28:14 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-a68687d3-459a-40d9-8044-74ce7508f50e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 02:28:14 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:28:14 user nova-compute[72351]:   <source dev="/dev/sdd"/>
Aug 09 02:28:14 user nova-compute[72351]:   <target dev="vdc" bus="virtio"/>
Aug 09 02:28:14 user nova-compute[72351]:   <serial>336fe492-aeb4-4440-9802-e98e8408a431</serial>
Aug 09 02:28:14 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x08" function="0x0"/>
Aug 09 02:28:14 user nova-compute[72351]: </disk>
Aug 09 02:28:14 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 02:28:14 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-a68687d3-459a-40d9-8044-74ce7508f50e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Successfully detached device vdc from instance 16c1e558-baa1-4c2d-b13b-50585f74d587 from the persistent domain config.
Aug 09 02:28:14 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-a68687d3-459a-40d9-8044-74ce7508f50e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] (1/8): Attempting to detach device vdc with device alias virtio-disk2 from instance 16c1e558-baa1-4c2d-b13b-50585f74d587 from the live domain config. {{(pid=72351) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2513}}
Aug 09 02:28:14 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-a68687d3-459a-40d9-8044-74ce7508f50e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 02:28:14 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:28:14 user nova-compute[72351]:   <source dev="/dev/sdd"/>
Aug 09 02:28:14 user nova-compute[72351]:   <target dev="vdc" bus="virtio"/>
Aug 09 02:28:14 user nova-compute[72351]:   <serial>336fe492-aeb4-4440-9802-e98e8408a431</serial>
Aug 09 02:28:14 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x08" function="0x0"/>
Aug 09 02:28:14 user nova-compute[72351]: </disk>
Aug 09 02:28:14 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG nova.compute.manager [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Start spawning the instance on the hypervisor. {{(pid=72351) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2612}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Creating instance directory {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4717}}
Aug 09 02:28:15 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Creating image(s)
Aug 09 02:28:15 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Did not create local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4853}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Ensure instance console log exists: /opt/stack/data/nova/instances/2a3bcdea-be36-4a5b-93cb-53396efdfa8e/console.log {{(pid=72351) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4603}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Start _get_guest_xml network_info=[{"id": "4d36d764-e86b-40ee-81ed-a9f3adfadc48", "address": "fa:16:3e:28:03:d9", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4d36d764-e8", "ovs_interfaceid": "4d36d764-e86b-40ee-81ed-a9f3adfadc48", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'device_type': 'disk', 'guest_format': None, 'disk_bus': 'virtio', 'mount_device': '/dev/vda', 'boot_index': 0, 'attachment_id': '58219aa6-8298-444b-95bb-5b4112d23ae8', 'delete_on_termination': False, 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqns': ['iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'CdzJhfwW', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '2a3bcdea-be36-4a5b-93cb-53396efdfa8e', 'attached_at': '', 'detached_at': '', 'volume_id': '4113a563-77a5-4b68-9198-9d0434737448', 'serial': '4113a563-77a5-4b68-9198-9d0434737448'}, 'volume_type': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:28:15 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:28:15 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:28:15 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'CdzJhfwW', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:28:15 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:28:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:28:15 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:28:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:28:15 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 02:28:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 -p 172.16.0.220:3260" returned: 21 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[76da7c33-7356-4bac-add2-64a60f569c27]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Aug 09 02:28:15 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 -p 172.16.0.220:3260 --interface default --op new {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[b207001e-eefe-4bca-a11f-d523beae7b70]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448] added\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448] added
Aug 09 02:28:15 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[720c67f5-222e-4c6f-a3f3-569f17ddc03a]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[183cf502-7031-4eb3-961d-00b63be89663]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v CdzJhfwW {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v CdzJhfwW" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[0882b4eb-669e-41bd-bd96-52ec4c90e5fb]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'CdzJhfwW'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Received event <DeviceRemovedEvent: 1691548095.5284588, 16c1e558-baa1-4c2d-b13b-50585f74d587 => virtio-disk2> from libvirt while the driver is waiting for it; dispatched. {{(pid=72351) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2360}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[0edfb0eb-60e1-4b31-b9f1-30af569f3208]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-a68687d3-459a-40d9-8044-74ce7508f50e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Start waiting for the detach event from libvirt for device vdc with device alias virtio-disk2 for instance 16c1e558-baa1-4c2d-b13b-50585f74d587 {{(pid=72351) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2589}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:15 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-a68687d3-459a-40d9-8044-74ce7508f50e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Successfully detached device vdc from instance 16c1e558-baa1-4c2d-b13b-50585f74d587 from the live domain config.
Aug 09 02:28:15 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-a68687d3-459a-40d9-8044-74ce7508f50e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-a68687d3-459a-40d9-8044-74ce7508f50e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:336fe492-aeb4-4440-9802-e98e8408a431'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'SdDZPfLw', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdd'}, None), 'kwargs': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:28:15 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-a68687d3-459a-40d9-8044-74ce7508f50e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:28:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-a68687d3-459a-40d9-8044-74ce7508f50e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.028s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[3411667f-1b46-49ca-ab05-4a3fb689e676]: (4, ('tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)\ntcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)\ntcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)\ntcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950 (non-flash)\ntcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:336fe492-aeb4-4440-9802-e98e8408a431 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)
Aug 09 02:28:15 user nova-compute[72351]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)
Aug 09 02:28:15 user nova-compute[72351]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:28:15 user nova-compute[72351]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:28:15 user nova-compute[72351]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950 (non-flash)
Aug 09 02:28:15 user nova-compute[72351]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:336fe492-aeb4-4440-9802-e98e8408a431 (non-flash)
Aug 09 02:28:15 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsi session list stdout=tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)
Aug 09 02:28:15 user nova-compute[72351]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)
Aug 09 02:28:15 user nova-compute[72351]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:28:15 user nova-compute[72351]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:28:15 user nova-compute[72351]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950 (non-flash)
Aug 09 02:28:15 user nova-compute[72351]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:336fe492-aeb4-4440-9802-e98e8408a431 (non-flash)
Aug 09 02:28:15 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 -p 172.16.0.220:3260 --login {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 -p 172.16.0.220:3260 --login" returned: 0 in 0.078s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[1d550da5-153e-40b3-91b6-3020a34926ee]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448, portal: 172.16.0.220,3260]
Aug 09 02:28:15 user nova-compute[72351]: Login to [iface: default, target: iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448, portal: 172.16.0.220,3260] successful.
Aug 09 02:28:15 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[af93e097-bb99-4245-87cb-21602f058ddc]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.027s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[7f0ee21c-a7b7-456d-bfa3-77c9d6029b6b]: (4, ('tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)\ntcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)\ntcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)\ntcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950 (non-flash)\ntcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:336fe492-aeb4-4440-9802-e98e8408a431 (non-flash)\ntcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)
Aug 09 02:28:15 user nova-compute[72351]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)
Aug 09 02:28:15 user nova-compute[72351]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:28:15 user nova-compute[72351]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:28:15 user nova-compute[72351]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950 (non-flash)
Aug 09 02:28:15 user nova-compute[72351]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:336fe492-aeb4-4440-9802-e98e8408a431 (non-flash)
Aug 09 02:28:15 user nova-compute[72351]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:28:15 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsi session list stdout=tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)
Aug 09 02:28:15 user nova-compute[72351]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)
Aug 09 02:28:15 user nova-compute[72351]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:28:15 user nova-compute[72351]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:28:15 user nova-compute[72351]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950 (non-flash)
Aug 09 02:28:15 user nova-compute[72351]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:336fe492-aeb4-4440-9802-e98e8408a431 (non-flash)
Aug 09 02:28:15 user nova-compute[72351]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:28:15 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.317s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] HCTL ('39', '-', '-', 0) found on session 31 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Scanning host 39 c: -, t: -, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host39/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host39/scan" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[5898f4d6-3866-4ca4-9e09-efa8a1c2b557]: (4, ('- - 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Searching for a device in session 31 and hctl ['39', '*', '*', 0] yield: sdh {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Connected to sdh using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'CdzJhfwW', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG nova.compute.manager [req-729b2541-fa0f-49f6-88d0-df8b4e501140 req-69ac7054-0df9-4601-98cc-2d21fd105b54 service nova] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Received event network-vif-plugged-9481e457-9c23-4c45-b0bd-1faf63f8b7b8 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-729b2541-fa0f-49f6-88d0-df8b4e501140 req-69ac7054-0df9-4601-98cc-2d21fd105b54 service nova] Acquiring lock "e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-729b2541-fa0f-49f6-88d0-df8b4e501140 req-69ac7054-0df9-4601-98cc-2d21fd105b54 service nova] Lock "e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-729b2541-fa0f-49f6-88d0-df8b4e501140 req-69ac7054-0df9-4601-98cc-2d21fd105b54 service nova] Lock "e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG nova.compute.manager [req-729b2541-fa0f-49f6-88d0-df8b4e501140 req-69ac7054-0df9-4601-98cc-2d21fd105b54 service nova] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] No waiting events found dispatching network-vif-plugged-9481e457-9c23-4c45-b0bd-1faf63f8b7b8 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:28:15 user nova-compute[72351]: WARNING nova.compute.manager [req-729b2541-fa0f-49f6-88d0-df8b4e501140 req-69ac7054-0df9-4601-98cc-2d21fd105b54 service nova] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Received unexpected event network-vif-plugged-9481e457-9c23-4c45-b0bd-1faf63f8b7b8 for instance with vm_state rescued and task_state unrescuing.
Aug 09 02:28:15 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:28:15 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:28:16 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:28:16 user nova-compute[72351]: INFO nova.compute.manager [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 957c1510-1320-4492-9052-a58ff540319d] Rescuing
Aug 09 02:28:16 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Acquiring lock "refresh_cache-957c1510-1320-4492-9052-a58ff540319d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:28:16 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Acquired lock "refresh_cache-957c1510-1320-4492-9052-a58ff540319d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:28:16 user nova-compute[72351]: DEBUG nova.network.neutron [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 957c1510-1320-4492-9052-a58ff540319d] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:28:16 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.356s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:28:16 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] <== connect_volume: return (1356ms) {'type': 'block', 'scsi_wwn': '23931306338316130', 'path': '/dev/sdh'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:28:16 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23931306338316130', 'path': '/dev/sdh'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 02:28:16 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-a68687d3-459a-40d9-8044-74ce7508f50e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 1.200s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:28:16 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-a68687d3-459a-40d9-8044-74ce7508f50e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:336fe492-aeb4-4440-9802-e98e8408a431', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 02:28:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:16 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:27:52Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-server-510987620',display_name='tempest-TestVolumeBootPattern-server-510987620',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testvolumebootpattern-server-510987620',id=31,image_ref='',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBM+1oHdu/+hmVTCW73FChAt+AvG2bL++YbEuGYMvrUaUYAyZQrt9Vwv7j34dhFmoVuCI0Rk18KgJ1X07X8pPcTOnOW0XFALfY2BbRHpJcmwARDVbYh8uPoNVxqSRoIR2gA==',key_name='tempest-TestVolumeBootPattern-805107831',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='a4c6bbfd8d904ba1ac69ba5da4564513',ramdisk_id='',reservation_id='r-65paxd61',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestVolumeBootPattern-350743460',owner_user_name='tempest-TestVolumeBootPattern-350743460-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:27:54Z,user_data=None,user_id='07d8d5a634274df19c2db769d2d27b29',uuid=2a3bcdea-be36-4a5b-93cb-53396efdfa8e,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "4d36d764-e86b-40ee-81ed-a9f3adfadc48", "address": "fa:16:3e:28:03:d9", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4d36d764-e8", "ovs_interfaceid": "4d36d764-e86b-40ee-81ed-a9f3adfadc48", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:28:16 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Converting VIF {"id": "4d36d764-e86b-40ee-81ed-a9f3adfadc48", "address": "fa:16:3e:28:03:d9", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4d36d764-e8", "ovs_interfaceid": "4d36d764-e86b-40ee-81ed-a9f3adfadc48", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:28:16 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:28:03:d9,bridge_name='br-int',has_traffic_filtering=True,id=4d36d764-e86b-40ee-81ed-a9f3adfadc48,network=Network(5ef428cf-e408-490c-8f60-70934f760f4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap4d36d764-e8') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:28:16 user nova-compute[72351]: DEBUG nova.objects.instance [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lazy-loading 'pci_devices' on Instance uuid 2a3bcdea-be36-4a5b-93cb-53396efdfa8e {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:28:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.024s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:16 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[7b1e1ac2-e69f-4fc5-9feb-d2c9f6a743bf]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:336fe492-aeb4-4440-9802-e98e8408a431\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:28:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:16 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:28:16 user nova-compute[72351]:   <uuid>2a3bcdea-be36-4a5b-93cb-53396efdfa8e</uuid>
Aug 09 02:28:16 user nova-compute[72351]:   <name>instance-0000001f</name>
Aug 09 02:28:16 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:28:16 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:28:16 user nova-compute[72351]:   <metadata>
Aug 09 02:28:16 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:28:16 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:28:16 user nova-compute[72351]:       <nova:name>tempest-TestVolumeBootPattern-server-510987620</nova:name>
Aug 09 02:28:16 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:28:15</nova:creationTime>
Aug 09 02:28:16 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:28:16 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:28:16 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:28:16 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:28:16 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:28:16 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:28:16 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:28:16 user nova-compute[72351]:       <nova:owner>
Aug 09 02:28:16 user nova-compute[72351]:         <nova:user uuid="07d8d5a634274df19c2db769d2d27b29">tempest-TestVolumeBootPattern-350743460-project-member</nova:user>
Aug 09 02:28:16 user nova-compute[72351]:         <nova:project uuid="a4c6bbfd8d904ba1ac69ba5da4564513">tempest-TestVolumeBootPattern-350743460</nova:project>
Aug 09 02:28:16 user nova-compute[72351]:       </nova:owner>
Aug 09 02:28:16 user nova-compute[72351]:       <nova:ports>
Aug 09 02:28:16 user nova-compute[72351]:         <nova:port uuid="4d36d764-e86b-40ee-81ed-a9f3adfadc48">
Aug 09 02:28:16 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.3" ipVersion="4"/>
Aug 09 02:28:16 user nova-compute[72351]:         </nova:port>
Aug 09 02:28:16 user nova-compute[72351]:       </nova:ports>
Aug 09 02:28:16 user nova-compute[72351]:     </nova:instance>
Aug 09 02:28:16 user nova-compute[72351]:   </metadata>
Aug 09 02:28:16 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:28:16 user nova-compute[72351]:     <system>
Aug 09 02:28:16 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:28:16 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:28:16 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:28:16 user nova-compute[72351]:       <entry name="serial">2a3bcdea-be36-4a5b-93cb-53396efdfa8e</entry>
Aug 09 02:28:16 user nova-compute[72351]:       <entry name="uuid">2a3bcdea-be36-4a5b-93cb-53396efdfa8e</entry>
Aug 09 02:28:16 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:28:16 user nova-compute[72351]:     </system>
Aug 09 02:28:16 user nova-compute[72351]:   </sysinfo>
Aug 09 02:28:16 user nova-compute[72351]:   <os>
Aug 09 02:28:16 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:28:16 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:28:16 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:28:16 user nova-compute[72351]:   </os>
Aug 09 02:28:16 user nova-compute[72351]:   <features>
Aug 09 02:28:16 user nova-compute[72351]:     <acpi/>
Aug 09 02:28:16 user nova-compute[72351]:     <apic/>
Aug 09 02:28:16 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:28:16 user nova-compute[72351]:   </features>
Aug 09 02:28:16 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:28:16 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:28:16 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:28:16 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:28:16 user nova-compute[72351]:   </clock>
Aug 09 02:28:16 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:28:16 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:28:16 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:28:16 user nova-compute[72351]:   </cpu>
Aug 09 02:28:16 user nova-compute[72351]:   <devices>
Aug 09 02:28:16 user nova-compute[72351]:     <disk type="block" device="disk">
Aug 09 02:28:16 user nova-compute[72351]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:28:16 user nova-compute[72351]:       <source dev="/dev/sdh"/>
Aug 09 02:28:16 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:28:16 user nova-compute[72351]:       <serial>4113a563-77a5-4b68-9198-9d0434737448</serial>
Aug 09 02:28:16 user nova-compute[72351]:     </disk>
Aug 09 02:28:16 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:28:16 user nova-compute[72351]:       <mac address="fa:16:3e:28:03:d9"/>
Aug 09 02:28:16 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:28:16 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:28:16 user nova-compute[72351]:       <target dev="tap4d36d764-e8"/>
Aug 09 02:28:16 user nova-compute[72351]:     </interface>
Aug 09 02:28:16 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:28:16 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/2a3bcdea-be36-4a5b-93cb-53396efdfa8e/console.log" append="off"/>
Aug 09 02:28:16 user nova-compute[72351]:     </serial>
Aug 09 02:28:16 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:28:16 user nova-compute[72351]:     <video>
Aug 09 02:28:16 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:28:16 user nova-compute[72351]:     </video>
Aug 09 02:28:16 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:28:16 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:28:16 user nova-compute[72351]:     </rng>
Aug 09 02:28:16 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:28:16 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:28:16 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:28:16 user nova-compute[72351]:     </memballoon>
Aug 09 02:28:16 user nova-compute[72351]:   </devices>
Aug 09 02:28:16 user nova-compute[72351]: </domain>
Aug 09 02:28:16 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:28:16 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:27:52Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-server-510987620',display_name='tempest-TestVolumeBootPattern-server-510987620',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testvolumebootpattern-server-510987620',id=31,image_ref='',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBM+1oHdu/+hmVTCW73FChAt+AvG2bL++YbEuGYMvrUaUYAyZQrt9Vwv7j34dhFmoVuCI0Rk18KgJ1X07X8pPcTOnOW0XFALfY2BbRHpJcmwARDVbYh8uPoNVxqSRoIR2gA==',key_name='tempest-TestVolumeBootPattern-805107831',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='a4c6bbfd8d904ba1ac69ba5da4564513',ramdisk_id='',reservation_id='r-65paxd61',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestVolumeBootPattern-350743460',owner_user_name='tempest-TestVolumeBootPattern-350743460-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:27:54Z,user_data=None,user_id='07d8d5a634274df19c2db769d2d27b29',uuid=2a3bcdea-be36-4a5b-93cb-53396efdfa8e,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "4d36d764-e86b-40ee-81ed-a9f3adfadc48", "address": "fa:16:3e:28:03:d9", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4d36d764-e8", "ovs_interfaceid": "4d36d764-e86b-40ee-81ed-a9f3adfadc48", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:28:16 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Converting VIF {"id": "4d36d764-e86b-40ee-81ed-a9f3adfadc48", "address": "fa:16:3e:28:03:d9", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4d36d764-e8", "ovs_interfaceid": "4d36d764-e86b-40ee-81ed-a9f3adfadc48", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:28:16 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:28:03:d9,bridge_name='br-int',has_traffic_filtering=True,id=4d36d764-e86b-40ee-81ed-a9f3adfadc48,network=Network(5ef428cf-e408-490c-8f60-70934f760f4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap4d36d764-e8') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:28:16 user nova-compute[72351]: DEBUG os_vif [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:28:03:d9,bridge_name='br-int',has_traffic_filtering=True,id=4d36d764-e86b-40ee-81ed-a9f3adfadc48,network=Network(5ef428cf-e408-490c-8f60-70934f760f4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap4d36d764-e8') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:28:16 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:28:16 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:28:16 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:28:16 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:28:16 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap4d36d764-e8, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:28:16 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap4d36d764-e8, col_values=(('external_ids', {'iface-id': '4d36d764-e86b-40ee-81ed-a9f3adfadc48', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:28:03:d9', 'vm-uuid': '2a3bcdea-be36-4a5b-93cb-53396efdfa8e'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:28:16 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:28:16 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:28:16 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:28:16 user nova-compute[72351]: INFO os_vif [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:28:03:d9,bridge_name='br-int',has_traffic_filtering=True,id=4d36d764-e86b-40ee-81ed-a9f3adfadc48,network=Network(5ef428cf-e408-490c-8f60-70934f760f4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap4d36d764-e8')
Aug 09 02:28:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.042s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:16 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a14fae31-20b5-42ac-8f22-642383a45764]: (4, ('tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)\ntcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)\ntcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)\ntcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950 (non-flash)\ntcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:336fe492-aeb4-4440-9802-e98e8408a431 (non-flash)\ntcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:28:16 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-a68687d3-459a-40d9-8044-74ce7508f50e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)
Aug 09 02:28:16 user nova-compute[72351]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)
Aug 09 02:28:16 user nova-compute[72351]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:28:16 user nova-compute[72351]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:28:16 user nova-compute[72351]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950 (non-flash)
Aug 09 02:28:16 user nova-compute[72351]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:336fe492-aeb4-4440-9802-e98e8408a431 (non-flash)
Aug 09 02:28:16 user nova-compute[72351]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:28:16 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:28:16 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-a68687d3-459a-40d9-8044-74ce7508f50e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] iscsi session list stdout=tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)
Aug 09 02:28:16 user nova-compute[72351]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)
Aug 09 02:28:16 user nova-compute[72351]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:28:16 user nova-compute[72351]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:28:16 user nova-compute[72351]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950 (non-flash)
Aug 09 02:28:16 user nova-compute[72351]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:336fe492-aeb4-4440-9802-e98e8408a431 (non-flash)
Aug 09 02:28:16 user nova-compute[72351]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:28:16 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:28:16 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-a68687d3-459a-40d9-8044-74ce7508f50e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fcfdc423c70>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:336fe492-aeb4-4440-9802-e98e8408a431'): ({'sdd'}, set())}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 02:28:16 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-a68687d3-459a-40d9-8044-74ce7508f50e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Removing single pathed devices sdd {{(pid=72351) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Aug 09 02:28:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:16 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[f8e1c548-8bc1-4780-be59-e65e4ce31a46]: (4, ('path checker states:\nup                  6\npending             1\n\npaths: 0\nbusy: False\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:28:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdd {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:16 user nova-compute[72351]: DEBUG nova.network.neutron [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 957c1510-1320-4492-9052-a58ff540319d] Updating instance_info_cache with network_info: [{"id": "e679cfd9-123f-4b88-8fd2-397e4c8616cc", "address": "fa:16:3e:7f:d6:1b", "network": {"id": "9a452414-d61f-4045-a6cc-9be93c892c4c", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "321111f935dd4e8aab4a083206ea9a7e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape679cfd9-12", "ovs_interfaceid": "e679cfd9-123f-4b88-8fd2-397e4c8616cc", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:28:16 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Releasing lock "refresh_cache-957c1510-1320-4492-9052-a58ff540319d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:28:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdd" returned: 0 in 0.025s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:16 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[6fec0909-1a26-4311-bc76-38bc4b4c3874]: (4, ('ok\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:28:16 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-a68687d3-459a-40d9-8044-74ce7508f50e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Flushing IO for device /dev/sdd {{(pid=72351) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Aug 09 02:28:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdd {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:16 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:28:16 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] No VIF found with MAC fa:16:3e:28:03:d9, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:28:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdd" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:16 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[cba28cb3-ce16-4d82-8e9e-55cd17ecf892]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:28:16 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-a68687d3-459a-40d9-8044-74ce7508f50e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Remove SCSI device /dev/sdd with /sys/block/sdd/device/delete {{(pid=72351) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Aug 09 02:28:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdd/device/delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdd/device/delete" returned: 0 in 0.046s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:16 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a3b23c7d-ced0-4144-81f1-d2c4117ff555]: (4, ('1', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:28:16 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-a68687d3-459a-40d9-8044-74ce7508f50e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Checking to see if SCSI volumes sdd have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Aug 09 02:28:16 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-a68687d3-459a-40d9-8044-74ce7508f50e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] SCSI volumes sdd have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Aug 09 02:28:16 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-a68687d3-459a-40d9-8044-74ce7508f50e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:336fe492-aeb4-4440-9802-e98e8408a431')] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 02:28:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:336fe492-aeb4-4440-9802-e98e8408a431 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:336fe492-aeb4-4440-9802-e98e8408a431 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:16 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d5ed14e2-1fcd-4d17-b4ba-a420d72d82c3]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:28:16 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-a68687d3-459a-40d9-8044-74ce7508f50e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:28:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:336fe492-aeb4-4440-9802-e98e8408a431 -p 172.16.0.220:3260 --logout {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:17 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:28:17 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:28:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:336fe492-aeb4-4440-9802-e98e8408a431 -p 172.16.0.220:3260 --logout" returned: 0 in 0.096s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:17 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[87e44ae8-3821-4faa-bba9-ff2d622bdd29]: (4, ('Logging out of session [sid: 30, target: iqn.2010-10.org.openstack:336fe492-aeb4-4440-9802-e98e8408a431, portal: 172.16.0.220,3260]\nLogout of [sid: 30, target: iqn.2010-10.org.openstack:336fe492-aeb4-4440-9802-e98e8408a431, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:28:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-a68687d3-459a-40d9-8044-74ce7508f50e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 30, target: iqn.2010-10.org.openstack:336fe492-aeb4-4440-9802-e98e8408a431, portal: 172.16.0.220,3260]
Aug 09 02:28:17 user nova-compute[72351]: Logout of [sid: 30, target: iqn.2010-10.org.openstack:336fe492-aeb4-4440-9802-e98e8408a431, portal: 172.16.0.220,3260] successful.
Aug 09 02:28:17 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:28:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:336fe492-aeb4-4440-9802-e98e8408a431 -p 172.16.0.220:3260 --op delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:336fe492-aeb4-4440-9802-e98e8408a431 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.038s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:17 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a2be549c-2776-46bf-a36b-0fa1874a3e45]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:28:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-a68687d3-459a-40d9-8044-74ce7508f50e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:28:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-a68687d3-459a-40d9-8044-74ce7508f50e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.391s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:28:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-a68687d3-459a-40d9-8044-74ce7508f50e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] <== disconnect_volume: return (1591ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:28:17 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-a68687d3-459a-40d9-8044-74ce7508f50e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 02:28:17 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:28:17 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:28:17 user nova-compute[72351]: DEBUG nova.compute.manager [req-7e7a7056-d783-41dd-ae2f-7d3db49123e7 req-95e2578b-aca1-4204-83e9-25c1fb847451 service nova] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Received event network-vif-plugged-9481e457-9c23-4c45-b0bd-1faf63f8b7b8 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:28:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-7e7a7056-d783-41dd-ae2f-7d3db49123e7 req-95e2578b-aca1-4204-83e9-25c1fb847451 service nova] Acquiring lock "e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:28:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-7e7a7056-d783-41dd-ae2f-7d3db49123e7 req-95e2578b-aca1-4204-83e9-25c1fb847451 service nova] Lock "e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:28:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-7e7a7056-d783-41dd-ae2f-7d3db49123e7 req-95e2578b-aca1-4204-83e9-25c1fb847451 service nova] Lock "e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:28:17 user nova-compute[72351]: DEBUG nova.compute.manager [req-7e7a7056-d783-41dd-ae2f-7d3db49123e7 req-95e2578b-aca1-4204-83e9-25c1fb847451 service nova] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] No waiting events found dispatching network-vif-plugged-9481e457-9c23-4c45-b0bd-1faf63f8b7b8 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:28:17 user nova-compute[72351]: WARNING nova.compute.manager [req-7e7a7056-d783-41dd-ae2f-7d3db49123e7 req-95e2578b-aca1-4204-83e9-25c1fb847451 service nova] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Received unexpected event network-vif-plugged-9481e457-9c23-4c45-b0bd-1faf63f8b7b8 for instance with vm_state rescued and task_state unrescuing.
Aug 09 02:28:17 user nova-compute[72351]: DEBUG nova.compute.manager [req-7e7a7056-d783-41dd-ae2f-7d3db49123e7 req-95e2578b-aca1-4204-83e9-25c1fb847451 service nova] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Received event network-vif-plugged-9481e457-9c23-4c45-b0bd-1faf63f8b7b8 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:28:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-7e7a7056-d783-41dd-ae2f-7d3db49123e7 req-95e2578b-aca1-4204-83e9-25c1fb847451 service nova] Acquiring lock "e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:28:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-7e7a7056-d783-41dd-ae2f-7d3db49123e7 req-95e2578b-aca1-4204-83e9-25c1fb847451 service nova] Lock "e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:28:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-7e7a7056-d783-41dd-ae2f-7d3db49123e7 req-95e2578b-aca1-4204-83e9-25c1fb847451 service nova] Lock "e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:28:17 user nova-compute[72351]: DEBUG nova.compute.manager [req-7e7a7056-d783-41dd-ae2f-7d3db49123e7 req-95e2578b-aca1-4204-83e9-25c1fb847451 service nova] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] No waiting events found dispatching network-vif-plugged-9481e457-9c23-4c45-b0bd-1faf63f8b7b8 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:28:17 user nova-compute[72351]: WARNING nova.compute.manager [req-7e7a7056-d783-41dd-ae2f-7d3db49123e7 req-95e2578b-aca1-4204-83e9-25c1fb847451 service nova] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Received unexpected event network-vif-plugged-9481e457-9c23-4c45-b0bd-1faf63f8b7b8 for instance with vm_state rescued and task_state unrescuing.
Aug 09 02:28:17 user nova-compute[72351]: DEBUG nova.compute.manager [req-7e7a7056-d783-41dd-ae2f-7d3db49123e7 req-95e2578b-aca1-4204-83e9-25c1fb847451 service nova] [instance: 957c1510-1320-4492-9052-a58ff540319d] Received event network-vif-unplugged-e679cfd9-123f-4b88-8fd2-397e4c8616cc {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:28:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-7e7a7056-d783-41dd-ae2f-7d3db49123e7 req-95e2578b-aca1-4204-83e9-25c1fb847451 service nova] Acquiring lock "957c1510-1320-4492-9052-a58ff540319d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:28:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-7e7a7056-d783-41dd-ae2f-7d3db49123e7 req-95e2578b-aca1-4204-83e9-25c1fb847451 service nova] Lock "957c1510-1320-4492-9052-a58ff540319d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:28:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-7e7a7056-d783-41dd-ae2f-7d3db49123e7 req-95e2578b-aca1-4204-83e9-25c1fb847451 service nova] Lock "957c1510-1320-4492-9052-a58ff540319d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:28:17 user nova-compute[72351]: DEBUG nova.compute.manager [req-7e7a7056-d783-41dd-ae2f-7d3db49123e7 req-95e2578b-aca1-4204-83e9-25c1fb847451 service nova] [instance: 957c1510-1320-4492-9052-a58ff540319d] No waiting events found dispatching network-vif-unplugged-e679cfd9-123f-4b88-8fd2-397e4c8616cc {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:28:17 user nova-compute[72351]: WARNING nova.compute.manager [req-7e7a7056-d783-41dd-ae2f-7d3db49123e7 req-95e2578b-aca1-4204-83e9-25c1fb847451 service nova] [instance: 957c1510-1320-4492-9052-a58ff540319d] Received unexpected event network-vif-unplugged-e679cfd9-123f-4b88-8fd2-397e4c8616cc for instance with vm_state active and task_state rescuing.
Aug 09 02:28:17 user nova-compute[72351]: DEBUG nova.compute.manager [req-7e7a7056-d783-41dd-ae2f-7d3db49123e7 req-95e2578b-aca1-4204-83e9-25c1fb847451 service nova] [instance: 957c1510-1320-4492-9052-a58ff540319d] Received event network-vif-plugged-e679cfd9-123f-4b88-8fd2-397e4c8616cc {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:28:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-7e7a7056-d783-41dd-ae2f-7d3db49123e7 req-95e2578b-aca1-4204-83e9-25c1fb847451 service nova] Acquiring lock "957c1510-1320-4492-9052-a58ff540319d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:28:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-7e7a7056-d783-41dd-ae2f-7d3db49123e7 req-95e2578b-aca1-4204-83e9-25c1fb847451 service nova] Lock "957c1510-1320-4492-9052-a58ff540319d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:28:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-7e7a7056-d783-41dd-ae2f-7d3db49123e7 req-95e2578b-aca1-4204-83e9-25c1fb847451 service nova] Lock "957c1510-1320-4492-9052-a58ff540319d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:28:17 user nova-compute[72351]: DEBUG nova.compute.manager [req-7e7a7056-d783-41dd-ae2f-7d3db49123e7 req-95e2578b-aca1-4204-83e9-25c1fb847451 service nova] [instance: 957c1510-1320-4492-9052-a58ff540319d] No waiting events found dispatching network-vif-plugged-e679cfd9-123f-4b88-8fd2-397e4c8616cc {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:28:17 user nova-compute[72351]: WARNING nova.compute.manager [req-7e7a7056-d783-41dd-ae2f-7d3db49123e7 req-95e2578b-aca1-4204-83e9-25c1fb847451 service nova] [instance: 957c1510-1320-4492-9052-a58ff540319d] Received unexpected event network-vif-plugged-e679cfd9-123f-4b88-8fd2-397e4c8616cc for instance with vm_state active and task_state rescuing.
Aug 09 02:28:18 user nova-compute[72351]: DEBUG nova.objects.instance [None req-a68687d3-459a-40d9-8044-74ce7508f50e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Lazy-loading 'flavor' on Instance uuid 16c1e558-baa1-4c2d-b13b-50585f74d587 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:28:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:28:18 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a68687d3-459a-40d9-8044-74ce7508f50e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Lock "16c1e558-baa1-4c2d-b13b-50585f74d587" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 3.203s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:28:18 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Removed pending event for e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335 due to event {{(pid=72351) _event_emit_delayed /opt/stack/nova/nova/virt/libvirt/host.py:438}}
Aug 09 02:28:18 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548098.1450071, e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335 => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:28:18 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] VM Resumed (Lifecycle Event)
Aug 09 02:28:18 user nova-compute[72351]: DEBUG nova.compute.manager [None req-1e61dfe0-d7d1-41cd-bdb2-9a175ad5ed6e tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:28:18 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:28:18 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: rescued, current task_state: unrescuing, current DB power_state: 1, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:28:18 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] During sync_power_state the instance has a pending task (unrescuing). Skip.
Aug 09 02:28:18 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548098.1461635, e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335 => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:28:18 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] VM Started (Lifecycle Event)
Aug 09 02:28:18 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 957c1510-1320-4492-9052-a58ff540319d] Instance destroyed successfully.
Aug 09 02:28:18 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:28:18 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 957c1510-1320-4492-9052-a58ff540319d] Attempting a stable device rescue
Aug 09 02:28:18 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Synchronizing instance power state after lifecycle event "Started"; current vm_state: active, current task_state: None, current DB power_state: 1, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:28:18 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] rescue generated disk_info: {'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk.rescue': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} {{(pid=72351) rescue /opt/stack/nova/nova/virt/libvirt/driver.py:4308}}
Aug 09 02:28:18 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 957c1510-1320-4492-9052-a58ff540319d] Instance directory exists: not creating {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4713}}
Aug 09 02:28:18 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 957c1510-1320-4492-9052-a58ff540319d] Creating image(s)
Aug 09 02:28:18 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Acquiring lock "/opt/stack/data/nova/instances/957c1510-1320-4492-9052-a58ff540319d/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:28:18 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "/opt/stack/data/nova/instances/957c1510-1320-4492-9052-a58ff540319d/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:28:18 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "/opt/stack/data/nova/instances/957c1510-1320-4492-9052-a58ff540319d/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:28:18 user nova-compute[72351]: DEBUG nova.objects.instance [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lazy-loading 'trusted_certs' on Instance uuid 957c1510-1320-4492-9052-a58ff540319d {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:28:18 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Acquiring lock "ccad13889b377d9d232f2cde88dc197c0daa4f29" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:28:18 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "ccad13889b377d9d232f2cde88dc197c0daa4f29" acquired by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:28:19 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/ccad13889b377d9d232f2cde88dc197c0daa4f29.part --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:19 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:28:19 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/ccad13889b377d9d232f2cde88dc197c0daa4f29.part --force-share --output=json" returned: 0 in 0.158s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:19 user nova-compute[72351]: DEBUG nova.virt.images [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] aaf0f355-970a-42d7-9b24-1ea26fc9062b was qcow2, converting to raw {{(pid=72351) fetch_to_raw /opt/stack/nova/nova/virt/images.py:165}}
Aug 09 02:28:19 user nova-compute[72351]: DEBUG nova.privsep.utils [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=72351) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Aug 09 02:28:19 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Running cmd (subprocess): qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/ccad13889b377d9d232f2cde88dc197c0daa4f29.part /opt/stack/data/nova/instances/_base/ccad13889b377d9d232f2cde88dc197c0daa4f29.converted {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG nova.compute.manager [req-78dce421-1873-4e11-b00b-e90513d5649b req-ff329872-570b-4762-b03a-ef8586fe7bc6 service nova] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Received event network-vif-plugged-4d36d764-e86b-40ee-81ed-a9f3adfadc48 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-78dce421-1873-4e11-b00b-e90513d5649b req-ff329872-570b-4762-b03a-ef8586fe7bc6 service nova] Acquiring lock "2a3bcdea-be36-4a5b-93cb-53396efdfa8e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-78dce421-1873-4e11-b00b-e90513d5649b req-ff329872-570b-4762-b03a-ef8586fe7bc6 service nova] Lock "2a3bcdea-be36-4a5b-93cb-53396efdfa8e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-78dce421-1873-4e11-b00b-e90513d5649b req-ff329872-570b-4762-b03a-ef8586fe7bc6 service nova] Lock "2a3bcdea-be36-4a5b-93cb-53396efdfa8e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG nova.compute.manager [req-78dce421-1873-4e11-b00b-e90513d5649b req-ff329872-570b-4762-b03a-ef8586fe7bc6 service nova] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] No waiting events found dispatching network-vif-plugged-4d36d764-e86b-40ee-81ed-a9f3adfadc48 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:28:20 user nova-compute[72351]: WARNING nova.compute.manager [req-78dce421-1873-4e11-b00b-e90513d5649b req-ff329872-570b-4762-b03a-ef8586fe7bc6 service nova] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Received unexpected event network-vif-plugged-4d36d764-e86b-40ee-81ed-a9f3adfadc48 for instance with vm_state building and task_state spawning.
Aug 09 02:28:20 user nova-compute[72351]: DEBUG nova.compute.manager [req-78dce421-1873-4e11-b00b-e90513d5649b req-ff329872-570b-4762-b03a-ef8586fe7bc6 service nova] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Received event network-vif-plugged-4d36d764-e86b-40ee-81ed-a9f3adfadc48 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-78dce421-1873-4e11-b00b-e90513d5649b req-ff329872-570b-4762-b03a-ef8586fe7bc6 service nova] Acquiring lock "2a3bcdea-be36-4a5b-93cb-53396efdfa8e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-78dce421-1873-4e11-b00b-e90513d5649b req-ff329872-570b-4762-b03a-ef8586fe7bc6 service nova] Lock "2a3bcdea-be36-4a5b-93cb-53396efdfa8e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-78dce421-1873-4e11-b00b-e90513d5649b req-ff329872-570b-4762-b03a-ef8586fe7bc6 service nova] Lock "2a3bcdea-be36-4a5b-93cb-53396efdfa8e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG nova.compute.manager [req-78dce421-1873-4e11-b00b-e90513d5649b req-ff329872-570b-4762-b03a-ef8586fe7bc6 service nova] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] No waiting events found dispatching network-vif-plugged-4d36d764-e86b-40ee-81ed-a9f3adfadc48 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:28:20 user nova-compute[72351]: WARNING nova.compute.manager [req-78dce421-1873-4e11-b00b-e90513d5649b req-ff329872-570b-4762-b03a-ef8586fe7bc6 service nova] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Received unexpected event network-vif-plugged-4d36d764-e86b-40ee-81ed-a9f3adfadc48 for instance with vm_state building and task_state spawning.
Aug 09 02:28:20 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] CMD "qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/ccad13889b377d9d232f2cde88dc197c0daa4f29.part /opt/stack/data/nova/instances/_base/ccad13889b377d9d232f2cde88dc197c0daa4f29.converted" returned: 0 in 0.609s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/ccad13889b377d9d232f2cde88dc197c0daa4f29.converted --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/ccad13889b377d9d232f2cde88dc197c0daa4f29.converted --force-share --output=json" returned: 0 in 0.160s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "ccad13889b377d9d232f2cde88dc197c0daa4f29" "released" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: held 2.148s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Acquiring lock "ccad13889b377d9d232f2cde88dc197c0daa4f29" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "ccad13889b377d9d232f2cde88dc197c0daa4f29" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/ccad13889b377d9d232f2cde88dc197c0daa4f29 --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691548085.5445507, f457e6ea-8d06-46c6-be2e-f650e28a3b69 => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:28:20 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] VM Stopped (Lifecycle Event)
Aug 09 02:28:20 user nova-compute[72351]: DEBUG nova.compute.manager [None req-61de150d-855e-4c42-b354-0ff07ed39daf None None] [instance: f457e6ea-8d06-46c6-be2e-f650e28a3b69] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/ccad13889b377d9d232f2cde88dc197c0daa4f29 --force-share --output=json" returned: 0 in 0.151s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/ccad13889b377d9d232f2cde88dc197c0daa4f29,backing_fmt=raw /opt/stack/data/nova/instances/957c1510-1320-4492-9052-a58ff540319d/disk.rescue {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/ccad13889b377d9d232f2cde88dc197c0daa4f29,backing_fmt=raw /opt/stack/data/nova/instances/957c1510-1320-4492-9052-a58ff540319d/disk.rescue" returned: 0 in 0.052s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "ccad13889b377d9d232f2cde88dc197c0daa4f29" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.208s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG nova.objects.instance [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lazy-loading 'migration_context' on Instance uuid 957c1510-1320-4492-9052-a58ff540319d {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 957c1510-1320-4492-9052-a58ff540319d] Created local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4851}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 957c1510-1320-4492-9052-a58ff540319d] Start _get_guest_xml network_info=[{"id": "e679cfd9-123f-4b88-8fd2-397e4c8616cc", "address": "fa:16:3e:7f:d6:1b", "network": {"id": "9a452414-d61f-4045-a6cc-9be93c892c4c", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [], "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "vif_mac": "fa:16:3e:7f:d6:1b"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "321111f935dd4e8aab4a083206ea9a7e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape679cfd9-12", "ovs_interfaceid": "e679cfd9-123f-4b88-8fd2-397e4c8616cc", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk.rescue': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>) rescue={'image_id': 'aaf0f355-970a-42d7-9b24-1ea26fc9062b', 'kernel_id': '', 'ramdisk_id': ''} block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'device_type': 'disk', 'guest_format': None, 'disk_bus': 'virtio', 'mount_device': '/dev/vda', 'boot_index': 0, 'attachment_id': 'd39213a9-ff03-4a4b-a1f7-78783e8e17c7', 'delete_on_termination': False, 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqns': ['iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'MbqJXivK', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdg'}, 'status': 'reserved', 'instance': '957c1510-1320-4492-9052-a58ff540319d', 'attached_at': '', 'detached_at': '', 'volume_id': '492c8c70-5cca-4ba1-954d-e1a7ebea4235', 'serial': '492c8c70-5cca-4ba1-954d-e1a7ebea4235'}, 'volume_type': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG nova.objects.instance [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lazy-loading 'resources' on Instance uuid 957c1510-1320-4492-9052-a58ff540319d {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG nova.objects.instance [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lazy-loading 'numa_topology' on Instance uuid 957c1510-1320-4492-9052-a58ff540319d {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:28:20 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:28:20 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:28:20 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG nova.objects.instance [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lazy-loading 'vcpu_model' on Instance uuid 957c1510-1320-4492-9052-a58ff540319d {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 957c1510-1320-4492-9052-a58ff540319d] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'MbqJXivK', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path'}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:28:20 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:28:20 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:28:20 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:28:20 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:28:20 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 02:28:20 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 -p 172.16.0.220:3260" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a8537d8c-d731-4fad-b4ca-3c2d03c0bc13]: (4, ('# BEGIN RECORD 2.1.5\nnode.name = iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235\nnode.tpgt = -1\nnode.startup = automatic\nnode.leading_login = No\niface.iscsi_ifacename = default\niface.net_ifacename = <empty>\niface.ipaddress = <empty>\niface.prefix_len = 0\niface.hwaddress = <empty>\niface.transport_name = tcp\niface.initiatorname = <empty>\niface.state = <empty>\niface.vlan_id = 0\niface.vlan_priority = 0\niface.vlan_state = <empty>\niface.iface_num = 0\niface.mtu = 0\niface.port = 0\niface.bootproto = <empty>\niface.subnet_mask = <empty>\niface.gateway = <empty>\niface.dhcp_alt_client_id_state = <empty>\niface.dhcp_alt_client_id = <empty>\niface.dhcp_dns = <empty>\niface.dhcp_learn_iqn = <empty>\niface.dhcp_req_vendor_id_state = <empty>\niface.dhcp_vendor_id_state = <empty>\niface.dhcp_vendor_id = <empty>\niface.dhcp_slp_da = <empty>\niface.fragmentation = <empty>\niface.gratuitous_arp = <empty>\niface.incoming_forwarding = <empty>\niface.tos_state = <empty>\niface.tos = 0\niface.ttl = 0\niface.delayed_ack = <empty>\niface.tcp_nagle = <empty>\niface.tcp_wsf_state = <empty>\niface.tcp_wsf = 0\niface.tcp_timer_scale = 0\niface.tcp_timestamp = <empty>\niface.redirect = <empty>\niface.def_task_mgmt_timeout = 0\niface.header_digest = <empty>\niface.data_digest = <empty>\niface.immediate_data = <empty>\niface.initial_r2t = <empty>\niface.data_seq_inorder = <empty>\niface.data_pdu_inorder = <empty>\niface.erl = 0\niface.max_receive_data_len = 0\niface.first_burst_len = 0\niface.max_outstanding_r2t = 0\niface.max_burst_len = 0\niface.chap_auth = <empty>\niface.bidi_chap = <empty>\niface.strict_login_compliance = <empty>\niface.discovery_auth = <empty>\niface.discovery_logout = <empty>\nnode.discovery_address = <empty>\nnode.discovery_port = 0\nnode.discovery_type = static\nnode.session.initial_cmdsn = 0\nnode.session.initial_login_retry_max = 8\nnode.session.xmit_thread_priority = -20\nnode.session.cmds_max = 128\nnode.session.queue_depth = 32\nnode.session.nr_sessions = 1\nnode.session.auth.authmethod = CHAP\nnode.session.auth.username = MbqJXivK\nnode.session.auth.password = ********\nnode.session.auth.username_in = <empty>\nnode.session.auth.password_in = <empty>\nnode.session.auth.chap_algs = MD5\nnode.session.timeo.replacement_timeout = 120\nnode.session.err_timeo.abort_timeout = 15\nnode.session.err_timeo.lu_reset_timeout = 30\nnode.session.err_timeo.tgt_reset_timeout = 30\nnode.session.err_timeo.host_reset_timeout = 60\nnode.session.iscsi.FastAbort = Yes\nnode.session.iscsi.InitialR2T = No\nnode.session.iscsi.ImmediateData = Yes\nnode.session.iscsi.FirstBurstLength = 262144\nnode.session.iscsi.MaxBurstLength = 16776192\nnode.session.iscsi.DefaultTime2Retain = 0\nnode.session.iscsi.DefaultTime2Wait = 2\nnode.session.iscsi.MaxConnections = 1\nnode.session.iscsi.MaxOutstandingR2T = 1\nnode.session.iscsi.ERL = 0\nnode.session.scan = manual\nnode.session.reopen_max = 0\nnode.conn[0].address = 172.16.0.220\nnode.conn[0].port = 3260\nnode.conn[0].startup = manual\nnode.conn[0].tcp.window_size = 524288\nnode.conn[0].tcp.type_of_service = 0\nnode.conn[0].timeo.logout_timeout = 15\nnode.conn[0].timeo.login_timeout = 15\nnode.conn[0].timeo.auth_timeout = 45\nnode.conn[0].timeo.noop_out_interval = 5\nnode.conn[0].timeo.noop_out_timeout = 5\nnode.conn[0].iscsi.MaxXmitDataSegmentLength = 0\nnode.conn[0].iscsi.MaxRecvDataSegmentLength = 262144\nnode.conn[0].iscsi.HeaderDigest = None\nnode.conn[0].iscsi.DataDigest = None\nnode.conn[0].iscsi.IFMarker = No\nnode.conn[0].iscsi.OFMarker = No\n# END RECORD\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] iscsiadm (): stdout=# BEGIN RECORD 2.1.5
Aug 09 02:28:20 user nova-compute[72351]: node.name = iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235
Aug 09 02:28:20 user nova-compute[72351]: node.tpgt = -1
Aug 09 02:28:20 user nova-compute[72351]: node.startup = automatic
Aug 09 02:28:20 user nova-compute[72351]: node.leading_login = No
Aug 09 02:28:20 user nova-compute[72351]: iface.iscsi_ifacename = default
Aug 09 02:28:20 user nova-compute[72351]: iface.net_ifacename = <empty>
Aug 09 02:28:20 user nova-compute[72351]: iface.ipaddress = <empty>
Aug 09 02:28:20 user nova-compute[72351]: iface.prefix_len = 0
Aug 09 02:28:20 user nova-compute[72351]: iface.hwaddress = <empty>
Aug 09 02:28:20 user nova-compute[72351]: iface.transport_name = tcp
Aug 09 02:28:20 user nova-compute[72351]: iface.initiatorname = <empty>
Aug 09 02:28:20 user nova-compute[72351]: iface.state = <empty>
Aug 09 02:28:20 user nova-compute[72351]: iface.vlan_id = 0
Aug 09 02:28:20 user nova-compute[72351]: iface.vlan_priority = 0
Aug 09 02:28:20 user nova-compute[72351]: iface.vlan_state = <empty>
Aug 09 02:28:20 user nova-compute[72351]: iface.iface_num = 0
Aug 09 02:28:20 user nova-compute[72351]: iface.mtu = 0
Aug 09 02:28:20 user nova-compute[72351]: iface.port = 0
Aug 09 02:28:20 user nova-compute[72351]: iface.bootproto = <empty>
Aug 09 02:28:20 user nova-compute[72351]: iface.subnet_mask = <empty>
Aug 09 02:28:20 user nova-compute[72351]: iface.gateway = <empty>
Aug 09 02:28:20 user nova-compute[72351]: iface.dhcp_alt_client_id_state = <empty>
Aug 09 02:28:20 user nova-compute[72351]: iface.dhcp_alt_client_id = <empty>
Aug 09 02:28:20 user nova-compute[72351]: iface.dhcp_dns = <empty>
Aug 09 02:28:20 user nova-compute[72351]: iface.dhcp_learn_iqn = <empty>
Aug 09 02:28:20 user nova-compute[72351]: iface.dhcp_req_vendor_id_state = <empty>
Aug 09 02:28:20 user nova-compute[72351]: iface.dhcp_vendor_id_state = <empty>
Aug 09 02:28:20 user nova-compute[72351]: iface.dhcp_vendor_id = <empty>
Aug 09 02:28:20 user nova-compute[72351]: iface.dhcp_slp_da = <empty>
Aug 09 02:28:20 user nova-compute[72351]: iface.fragmentation = <empty>
Aug 09 02:28:20 user nova-compute[72351]: iface.gratuitous_arp = <empty>
Aug 09 02:28:20 user nova-compute[72351]: iface.incoming_forwarding = <empty>
Aug 09 02:28:20 user nova-compute[72351]: iface.tos_state = <empty>
Aug 09 02:28:20 user nova-compute[72351]: iface.tos = 0
Aug 09 02:28:20 user nova-compute[72351]: iface.ttl = 0
Aug 09 02:28:20 user nova-compute[72351]: iface.delayed_ack = <empty>
Aug 09 02:28:20 user nova-compute[72351]: iface.tcp_nagle = <empty>
Aug 09 02:28:20 user nova-compute[72351]: iface.tcp_wsf_state = <empty>
Aug 09 02:28:20 user nova-compute[72351]: iface.tcp_wsf = 0
Aug 09 02:28:20 user nova-compute[72351]: iface.tcp_timer_scale = 0
Aug 09 02:28:20 user nova-compute[72351]: iface.tcp_timestamp = <empty>
Aug 09 02:28:20 user nova-compute[72351]: iface.redirect = <empty>
Aug 09 02:28:20 user nova-compute[72351]: iface.def_task_mgmt_timeout = 0
Aug 09 02:28:20 user nova-compute[72351]: iface.header_digest = <empty>
Aug 09 02:28:20 user nova-compute[72351]: iface.data_digest = <empty>
Aug 09 02:28:20 user nova-compute[72351]: iface.immediate_data = <empty>
Aug 09 02:28:20 user nova-compute[72351]: iface.initial_r2t = <empty>
Aug 09 02:28:20 user nova-compute[72351]: iface.data_seq_inorder = <empty>
Aug 09 02:28:20 user nova-compute[72351]: iface.data_pdu_inorder = <empty>
Aug 09 02:28:20 user nova-compute[72351]: iface.erl = 0
Aug 09 02:28:20 user nova-compute[72351]: iface.max_receive_data_len = 0
Aug 09 02:28:20 user nova-compute[72351]: iface.first_burst_len = 0
Aug 09 02:28:20 user nova-compute[72351]: iface.max_outstanding_r2t = 0
Aug 09 02:28:20 user nova-compute[72351]: iface.max_burst_len = 0
Aug 09 02:28:20 user nova-compute[72351]: iface.chap_auth = <empty>
Aug 09 02:28:20 user nova-compute[72351]: iface.bidi_chap = <empty>
Aug 09 02:28:20 user nova-compute[72351]: iface.strict_login_compliance = <empty>
Aug 09 02:28:20 user nova-compute[72351]: iface.discovery_auth = <empty>
Aug 09 02:28:20 user nova-compute[72351]: iface.discovery_logout = <empty>
Aug 09 02:28:20 user nova-compute[72351]: node.discovery_address = <empty>
Aug 09 02:28:20 user nova-compute[72351]: node.discovery_port = 0
Aug 09 02:28:20 user nova-compute[72351]: node.discovery_type = static
Aug 09 02:28:20 user nova-compute[72351]: node.session.initial_cmdsn = 0
Aug 09 02:28:20 user nova-compute[72351]: node.session.initial_login_retry_max = 8
Aug 09 02:28:20 user nova-compute[72351]: node.session.xmit_thread_priority = -20
Aug 09 02:28:20 user nova-compute[72351]: node.session.cmds_max = 128
Aug 09 02:28:20 user nova-compute[72351]: node.session.queue_depth = 32
Aug 09 02:28:20 user nova-compute[72351]: node.session.nr_sessions = 1
Aug 09 02:28:20 user nova-compute[72351]: node.session.auth.authmethod = CHAP
Aug 09 02:28:20 user nova-compute[72351]: node.session.auth.username = MbqJXivK
Aug 09 02:28:20 user nova-compute[72351]: node.session.auth.password = ***
Aug 09 02:28:20 user nova-compute[72351]: node.session.auth.username_in = <empty>
Aug 09 02:28:20 user nova-compute[72351]: node.session.auth.password_in = <empty>
Aug 09 02:28:20 user nova-compute[72351]: node.session.auth.chap_algs = MD5
Aug 09 02:28:20 user nova-compute[72351]: node.session.timeo.replacement_timeout = 120
Aug 09 02:28:20 user nova-compute[72351]: node.session.err_timeo.abort_timeout = 15
Aug 09 02:28:20 user nova-compute[72351]: node.session.err_timeo.lu_reset_timeout = 30
Aug 09 02:28:20 user nova-compute[72351]: node.session.err_timeo.tgt_reset_timeout = 30
Aug 09 02:28:20 user nova-compute[72351]: node.session.err_timeo.host_reset_timeout = 60
Aug 09 02:28:20 user nova-compute[72351]: node.session.iscsi.FastAbort = Yes
Aug 09 02:28:20 user nova-compute[72351]: node.session.iscsi.InitialR2T = No
Aug 09 02:28:20 user nova-compute[72351]: node.session.iscsi.ImmediateData = Yes
Aug 09 02:28:20 user nova-compute[72351]: node.session.iscsi.FirstBurstLength = 262144
Aug 09 02:28:20 user nova-compute[72351]: node.session.iscsi.MaxBurstLength = 16776192
Aug 09 02:28:20 user nova-compute[72351]: node.session.iscsi.DefaultTime2Retain = 0
Aug 09 02:28:20 user nova-compute[72351]: node.session.iscsi.DefaultTime2Wait = 2
Aug 09 02:28:20 user nova-compute[72351]: node.session.iscsi.MaxConnections = 1
Aug 09 02:28:20 user nova-compute[72351]: node.session.iscsi.MaxOutstandingR2T = 1
Aug 09 02:28:20 user nova-compute[72351]: node.session.iscsi.ERL = 0
Aug 09 02:28:20 user nova-compute[72351]: node.session.scan = manual
Aug 09 02:28:20 user nova-compute[72351]: node.session.reopen_max = 0
Aug 09 02:28:20 user nova-compute[72351]: node.conn[0].address = 172.16.0.220
Aug 09 02:28:20 user nova-compute[72351]: node.conn[0].port = 3260
Aug 09 02:28:20 user nova-compute[72351]: node.conn[0].startup = manual
Aug 09 02:28:20 user nova-compute[72351]: node.conn[0].tcp.window_size = 524288
Aug 09 02:28:20 user nova-compute[72351]: node.conn[0].tcp.type_of_service = 0
Aug 09 02:28:20 user nova-compute[72351]: node.conn[0].timeo.logout_timeout = 15
Aug 09 02:28:20 user nova-compute[72351]: node.conn[0].timeo.login_timeout = 15
Aug 09 02:28:20 user nova-compute[72351]: node.conn[0].timeo.auth_timeout = 45
Aug 09 02:28:20 user nova-compute[72351]: node.conn[0].timeo.noop_out_interval = 5
Aug 09 02:28:20 user nova-compute[72351]: node.conn[0].timeo.noop_out_timeout = 5
Aug 09 02:28:20 user nova-compute[72351]: node.conn[0].iscsi.MaxXmitDataSegmentLength = 0
Aug 09 02:28:20 user nova-compute[72351]: node.conn[0].iscsi.MaxRecvDataSegmentLength = 262144
Aug 09 02:28:20 user nova-compute[72351]: node.conn[0].iscsi.HeaderDigest = None
Aug 09 02:28:20 user nova-compute[72351]: node.conn[0].iscsi.DataDigest = None
Aug 09 02:28:20 user nova-compute[72351]: node.conn[0].iscsi.IFMarker = No
Aug 09 02:28:20 user nova-compute[72351]: node.conn[0].iscsi.OFMarker = No
Aug 09 02:28:20 user nova-compute[72351]: # END RECORD
Aug 09 02:28:20 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.027s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[77f4bbc1-09ce-4c06-b119-fb4565b0246f]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[85078a2d-9f68-4dc7-8dd5-7cae35d2d25e]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v MbqJXivK {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v MbqJXivK" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[877e2dac-310b-4268-80e4-57b7a9abc4b7]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'MbqJXivK'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[2b9e365a-95a6-43dc-9544-54239e8d42e4]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:28:20 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.029s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:21 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[9fa2b7af-47a1-47a6-8c49-1a42d57d5a1d]: (4, ('tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)\ntcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)\ntcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)\ntcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950 (non-flash)\ntcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:28:21 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)
Aug 09 02:28:21 user nova-compute[72351]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)
Aug 09 02:28:21 user nova-compute[72351]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:28:21 user nova-compute[72351]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:28:21 user nova-compute[72351]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950 (non-flash)
Aug 09 02:28:21 user nova-compute[72351]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:28:21 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:28:21 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] iscsi session list stdout=tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)
Aug 09 02:28:21 user nova-compute[72351]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)
Aug 09 02:28:21 user nova-compute[72351]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:28:21 user nova-compute[72351]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:28:21 user nova-compute[72351]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950 (non-flash)
Aug 09 02:28:21 user nova-compute[72351]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:28:21 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:28:21 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.160s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:28:21 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 02:28:21 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] HCTL ('38', '0', '0', 0) found on session 26 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 02:28:21 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Scanning host 38 c: 0, t: 0, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 02:28:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host38/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host38/scan" returned: 0 in 0.016s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:21 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[33e7d5f3-d72e-4f22-a821-4693e3e8703c]: (4, ('0 0 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:28:21 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Searching for a device in session 26 and hctl ('38', '0', '0', 0) yield: sdg {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:28:21 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Connected to sdg using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'MbqJXivK', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdg', 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 02:28:21 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 0.187s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:28:21 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] <== connect_volume: return (189ms) {'type': 'block', 'scsi_wwn': '26461343430643335', 'path': '/dev/sdg'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:28:21 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 957c1510-1320-4492-9052-a58ff540319d] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26461343430643335', 'path': '/dev/sdg'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 02:28:21 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:26:24Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-821495844',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-821495844',id=28,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:26:50Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='321111f935dd4e8aab4a083206ea9a7e',ramdisk_id='',reservation_id='r-081hk0tj',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-2108329302',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member'},tags=<?>,task_state='rescuing',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:26:50Z,user_data=None,user_id='73d05c48be5640c28455e8ba8b05ef31',uuid=957c1510-1320-4492-9052-a58ff540319d,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "e679cfd9-123f-4b88-8fd2-397e4c8616cc", "address": "fa:16:3e:7f:d6:1b", "network": {"id": "9a452414-d61f-4045-a6cc-9be93c892c4c", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [], "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "vif_mac": "fa:16:3e:7f:d6:1b"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "321111f935dd4e8aab4a083206ea9a7e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape679cfd9-12", "ovs_interfaceid": "e679cfd9-123f-4b88-8fd2-397e4c8616cc", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:28:21 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Converting VIF {"id": "e679cfd9-123f-4b88-8fd2-397e4c8616cc", "address": "fa:16:3e:7f:d6:1b", "network": {"id": "9a452414-d61f-4045-a6cc-9be93c892c4c", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [], "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "vif_mac": "fa:16:3e:7f:d6:1b"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "321111f935dd4e8aab4a083206ea9a7e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape679cfd9-12", "ovs_interfaceid": "e679cfd9-123f-4b88-8fd2-397e4c8616cc", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:28:21 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:7f:d6:1b,bridge_name='br-int',has_traffic_filtering=True,id=e679cfd9-123f-4b88-8fd2-397e4c8616cc,network=Network(9a452414-d61f-4045-a6cc-9be93c892c4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape679cfd9-12') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:28:21 user nova-compute[72351]: DEBUG nova.objects.instance [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lazy-loading 'pci_devices' on Instance uuid 957c1510-1320-4492-9052-a58ff540319d {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:28:21 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 957c1510-1320-4492-9052-a58ff540319d] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:28:21 user nova-compute[72351]:   <uuid>957c1510-1320-4492-9052-a58ff540319d</uuid>
Aug 09 02:28:21 user nova-compute[72351]:   <name>instance-0000001c</name>
Aug 09 02:28:21 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:28:21 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:28:21 user nova-compute[72351]:   <metadata>
Aug 09 02:28:21 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:28:21 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:28:21 user nova-compute[72351]:       <nova:name>tempest-ServerBootFromVolumeStableRescueTest-server-821495844</nova:name>
Aug 09 02:28:21 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:28:20</nova:creationTime>
Aug 09 02:28:21 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:28:21 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:28:21 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:28:21 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:28:21 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:28:21 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:28:21 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:28:21 user nova-compute[72351]:       <nova:owner>
Aug 09 02:28:21 user nova-compute[72351]:         <nova:user uuid="73d05c48be5640c28455e8ba8b05ef31">tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member</nova:user>
Aug 09 02:28:21 user nova-compute[72351]:         <nova:project uuid="321111f935dd4e8aab4a083206ea9a7e">tempest-ServerBootFromVolumeStableRescueTest-2108329302</nova:project>
Aug 09 02:28:21 user nova-compute[72351]:       </nova:owner>
Aug 09 02:28:21 user nova-compute[72351]:       <nova:root type="image" uuid="18ccda7f-31d3-4978-ab9d-4ed2f74eff94"/>
Aug 09 02:28:21 user nova-compute[72351]:       <nova:ports>
Aug 09 02:28:21 user nova-compute[72351]:         <nova:port uuid="e679cfd9-123f-4b88-8fd2-397e4c8616cc">
Aug 09 02:28:21 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.7" ipVersion="4"/>
Aug 09 02:28:21 user nova-compute[72351]:         </nova:port>
Aug 09 02:28:21 user nova-compute[72351]:       </nova:ports>
Aug 09 02:28:21 user nova-compute[72351]:     </nova:instance>
Aug 09 02:28:21 user nova-compute[72351]:   </metadata>
Aug 09 02:28:21 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:28:21 user nova-compute[72351]:     <system>
Aug 09 02:28:21 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:28:21 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:28:21 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:28:21 user nova-compute[72351]:       <entry name="serial">957c1510-1320-4492-9052-a58ff540319d</entry>
Aug 09 02:28:21 user nova-compute[72351]:       <entry name="uuid">957c1510-1320-4492-9052-a58ff540319d</entry>
Aug 09 02:28:21 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:28:21 user nova-compute[72351]:     </system>
Aug 09 02:28:21 user nova-compute[72351]:   </sysinfo>
Aug 09 02:28:21 user nova-compute[72351]:   <os>
Aug 09 02:28:21 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:28:21 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:28:21 user nova-compute[72351]:   </os>
Aug 09 02:28:21 user nova-compute[72351]:   <features>
Aug 09 02:28:21 user nova-compute[72351]:     <acpi/>
Aug 09 02:28:21 user nova-compute[72351]:     <apic/>
Aug 09 02:28:21 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:28:21 user nova-compute[72351]:   </features>
Aug 09 02:28:21 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:28:21 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:28:21 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:28:21 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:28:21 user nova-compute[72351]:   </clock>
Aug 09 02:28:21 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:28:21 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:28:21 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:28:21 user nova-compute[72351]:   </cpu>
Aug 09 02:28:21 user nova-compute[72351]:   <devices>
Aug 09 02:28:21 user nova-compute[72351]:     <disk type="block" device="disk">
Aug 09 02:28:21 user nova-compute[72351]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:28:21 user nova-compute[72351]:       <source dev="/dev/sdg"/>
Aug 09 02:28:21 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:28:21 user nova-compute[72351]:       <serial>492c8c70-5cca-4ba1-954d-e1a7ebea4235</serial>
Aug 09 02:28:21 user nova-compute[72351]:     </disk>
Aug 09 02:28:21 user nova-compute[72351]:     <disk type="file" device="disk">
Aug 09 02:28:21 user nova-compute[72351]:       <driver name="qemu" type="qcow2" cache="none"/>
Aug 09 02:28:21 user nova-compute[72351]:       <source file="/opt/stack/data/nova/instances/957c1510-1320-4492-9052-a58ff540319d/disk.rescue"/>
Aug 09 02:28:21 user nova-compute[72351]:       <target dev="vdb" bus="virtio"/>
Aug 09 02:28:21 user nova-compute[72351]:       <boot order="1"/>
Aug 09 02:28:21 user nova-compute[72351]:     </disk>
Aug 09 02:28:21 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:28:21 user nova-compute[72351]:       <mac address="fa:16:3e:7f:d6:1b"/>
Aug 09 02:28:21 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:28:21 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:28:21 user nova-compute[72351]:       <target dev="tape679cfd9-12"/>
Aug 09 02:28:21 user nova-compute[72351]:     </interface>
Aug 09 02:28:21 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:28:21 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/957c1510-1320-4492-9052-a58ff540319d/console.log" append="off"/>
Aug 09 02:28:21 user nova-compute[72351]:     </serial>
Aug 09 02:28:21 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:28:21 user nova-compute[72351]:     <video>
Aug 09 02:28:21 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:28:21 user nova-compute[72351]:     </video>
Aug 09 02:28:21 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:28:21 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:28:21 user nova-compute[72351]:     </rng>
Aug 09 02:28:21 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:28:21 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:28:21 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:28:21 user nova-compute[72351]:     </memballoon>
Aug 09 02:28:21 user nova-compute[72351]:   </devices>
Aug 09 02:28:21 user nova-compute[72351]: </domain>
Aug 09 02:28:21 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:28:21 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 957c1510-1320-4492-9052-a58ff540319d] Instance destroyed successfully.
Aug 09 02:28:21 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548101.188127, 2a3bcdea-be36-4a5b-93cb-53396efdfa8e => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:28:21 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] VM Resumed (Lifecycle Event)
Aug 09 02:28:21 user nova-compute[72351]: DEBUG nova.compute.manager [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:28:21 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Guest created on hypervisor {{(pid=72351) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4411}}
Aug 09 02:28:21 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:28:21 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:28:21 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] No VIF found with MAC fa:16:3e:7f:d6:1b, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:28:21 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:28:21 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Instance spawned successfully.
Aug 09 02:28:21 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:894}}
Aug 09 02:28:21 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:28:21 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Found default for hw_cdrom_bus of ide {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:28:21 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Found default for hw_disk_bus of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:28:21 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Found default for hw_input_bus of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:28:21 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Found default for hw_pointer_model of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:28:21 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Found default for hw_video_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:28:21 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Found default for hw_vif_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:28:21 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:28:21 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548101.189788, 2a3bcdea-be36-4a5b-93cb-53396efdfa8e => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:28:21 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] VM Started (Lifecycle Event)
Aug 09 02:28:21 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:28:21 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:28:21 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:28:21 user nova-compute[72351]: INFO nova.compute.manager [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Took 6.01 seconds to spawn the instance on the hypervisor.
Aug 09 02:28:21 user nova-compute[72351]: DEBUG nova.compute.manager [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:28:21 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Acquiring lock "9df16e46-d097-4553-b44c-732ea24a4169" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:28:21 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lock "9df16e46-d097-4553-b44c-732ea24a4169" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:28:21 user nova-compute[72351]: DEBUG nova.compute.manager [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Starting instance... {{(pid=72351) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2406}}
Aug 09 02:28:21 user nova-compute[72351]: INFO nova.compute.manager [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Took 28.66 seconds to build instance.
Aug 09 02:28:21 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e446035a-bfb8-4d55-9ea4-defca0e1fe32 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "2a3bcdea-be36-4a5b-93cb-53396efdfa8e" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 28.807s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:28:21 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:28:21 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:28:21 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72351) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Aug 09 02:28:21 user nova-compute[72351]: INFO nova.compute.claims [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Claim successful on node user
Aug 09 02:28:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:28:21 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:28:21 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:28:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.523s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:28:22 user nova-compute[72351]: DEBUG nova.compute.manager [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Start building networks asynchronously for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2803}}
Aug 09 02:28:22 user nova-compute[72351]: DEBUG nova.compute.manager [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Allocating IP information in the background. {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1956}}
Aug 09 02:28:22 user nova-compute[72351]: DEBUG nova.network.neutron [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] allocate_for_instance() {{(pid=72351) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Aug 09 02:28:22 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Aug 09 02:28:22 user nova-compute[72351]: DEBUG nova.compute.manager [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Start building block device mappings for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2838}}
Aug 09 02:28:22 user nova-compute[72351]: DEBUG nova.policy [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '97267b9f897e4034b71627a698210071', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c8ff52319444079b754f79843c67b2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72351) authorize /opt/stack/nova/nova/policy.py:203}}
Aug 09 02:28:22 user nova-compute[72351]: DEBUG nova.compute.manager [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Start spawning the instance on the hypervisor. {{(pid=72351) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2612}}
Aug 09 02:28:22 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Creating instance directory {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4717}}
Aug 09 02:28:22 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Creating image(s)
Aug 09 02:28:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Acquiring lock "/opt/stack/data/nova/instances/9df16e46-d097-4553-b44c-732ea24a4169/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:28:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lock "/opt/stack/data/nova/instances/9df16e46-d097-4553-b44c-732ea24a4169/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:28:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lock "/opt/stack/data/nova/instances/9df16e46-d097-4553-b44c-732ea24a4169/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.006s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:28:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.162s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Acquiring lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:28:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.004s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:28:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.150s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/9df16e46-d097-4553-b44c-732ea24a4169/disk 1073741824 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/9df16e46-d097-4553-b44c-732ea24a4169/disk 1073741824" returned: 0 in 0.070s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.227s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:28:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:28:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:28:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:28:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.174s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:22 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Checking if we can resize image /opt/stack/data/nova/instances/9df16e46-d097-4553-b44c-732ea24a4169/disk. size=1073741824 {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Aug 09 02:28:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9df16e46-d097-4553-b44c-732ea24a4169/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9df16e46-d097-4553-b44c-732ea24a4169/disk --force-share --output=json" returned: 0 in 0.146s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:22 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Cannot resize image /opt/stack/data/nova/instances/9df16e46-d097-4553-b44c-732ea24a4169/disk to a smaller size. {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Aug 09 02:28:22 user nova-compute[72351]: DEBUG nova.objects.instance [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lazy-loading 'migration_context' on Instance uuid 9df16e46-d097-4553-b44c-732ea24a4169 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:28:23 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Created local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4851}}
Aug 09 02:28:23 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Ensure instance console log exists: /opt/stack/data/nova/instances/9df16e46-d097-4553-b44c-732ea24a4169/console.log {{(pid=72351) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4603}}
Aug 09 02:28:23 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:28:23 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:28:23 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:28:23 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:28:23 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:28:23 user nova-compute[72351]: DEBUG nova.compute.manager [req-6d427b9d-4366-4d35-b2fd-ec40caeca7c5 req-b79bd665-7923-407e-8f19-220f74bb4ae1 service nova] [instance: 957c1510-1320-4492-9052-a58ff540319d] Received event network-vif-plugged-e679cfd9-123f-4b88-8fd2-397e4c8616cc {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:28:23 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-6d427b9d-4366-4d35-b2fd-ec40caeca7c5 req-b79bd665-7923-407e-8f19-220f74bb4ae1 service nova] Acquiring lock "957c1510-1320-4492-9052-a58ff540319d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:28:23 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-6d427b9d-4366-4d35-b2fd-ec40caeca7c5 req-b79bd665-7923-407e-8f19-220f74bb4ae1 service nova] Lock "957c1510-1320-4492-9052-a58ff540319d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:28:23 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-6d427b9d-4366-4d35-b2fd-ec40caeca7c5 req-b79bd665-7923-407e-8f19-220f74bb4ae1 service nova] Lock "957c1510-1320-4492-9052-a58ff540319d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:28:23 user nova-compute[72351]: DEBUG nova.compute.manager [req-6d427b9d-4366-4d35-b2fd-ec40caeca7c5 req-b79bd665-7923-407e-8f19-220f74bb4ae1 service nova] [instance: 957c1510-1320-4492-9052-a58ff540319d] No waiting events found dispatching network-vif-plugged-e679cfd9-123f-4b88-8fd2-397e4c8616cc {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:28:23 user nova-compute[72351]: WARNING nova.compute.manager [req-6d427b9d-4366-4d35-b2fd-ec40caeca7c5 req-b79bd665-7923-407e-8f19-220f74bb4ae1 service nova] [instance: 957c1510-1320-4492-9052-a58ff540319d] Received unexpected event network-vif-plugged-e679cfd9-123f-4b88-8fd2-397e4c8616cc for instance with vm_state active and task_state rescuing.
Aug 09 02:28:23 user nova-compute[72351]: DEBUG nova.network.neutron [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Successfully created port: e8c25a1c-a605-4d7a-bb03-395b3d844571 {{(pid=72351) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Aug 09 02:28:24 user nova-compute[72351]: DEBUG nova.network.neutron [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Successfully updated port: e8c25a1c-a605-4d7a-bb03-395b3d844571 {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:28:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Acquiring lock "refresh_cache-9df16e46-d097-4553-b44c-732ea24a4169" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:28:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Acquired lock "refresh_cache-9df16e46-d097-4553-b44c-732ea24a4169" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:28:24 user nova-compute[72351]: DEBUG nova.network.neutron [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:28:24 user nova-compute[72351]: DEBUG nova.network.neutron [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:28:25 user nova-compute[72351]: DEBUG nova.network.neutron [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Updating instance_info_cache with network_info: [{"id": "e8c25a1c-a605-4d7a-bb03-395b3d844571", "address": "fa:16:3e:69:6a:fa", "network": {"id": "96018a29-3b2e-44de-b2c4-67abc92e3a30", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-924549970-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "1c8ff52319444079b754f79843c67b2b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape8c25a1c-a6", "ovs_interfaceid": "e8c25a1c-a605-4d7a-bb03-395b3d844571", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:28:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Releasing lock "refresh_cache-9df16e46-d097-4553-b44c-732ea24a4169" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:28:25 user nova-compute[72351]: DEBUG nova.compute.manager [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Instance network_info: |[{"id": "e8c25a1c-a605-4d7a-bb03-395b3d844571", "address": "fa:16:3e:69:6a:fa", "network": {"id": "96018a29-3b2e-44de-b2c4-67abc92e3a30", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-924549970-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "1c8ff52319444079b754f79843c67b2b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape8c25a1c-a6", "ovs_interfaceid": "e8c25a1c-a605-4d7a-bb03-395b3d844571", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1971}}
Aug 09 02:28:25 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Start _get_guest_xml network_info=[{"id": "e8c25a1c-a605-4d7a-bb03-395b3d844571", "address": "fa:16:3e:69:6a:fa", "network": {"id": "96018a29-3b2e-44de-b2c4-67abc92e3a30", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-924549970-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "1c8ff52319444079b754f79843c67b2b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape8c25a1c-a6", "ovs_interfaceid": "e8c25a1c-a605-4d7a-bb03-395b3d844571", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'size': 0, 'device_type': 'disk', 'encryption_secret_uuid': None, 'guest_format': None, 'encryption_options': None, 'disk_bus': 'virtio', 'encrypted': False, 'encryption_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'image_id': '18ccda7f-31d3-4978-ab9d-4ed2f74eff94'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:28:25 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:28:25 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:28:25 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:28:25 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:28:25 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:28:25 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:28:25 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:28:25 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:28:25 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:28:25 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:28:25 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:28:25 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:28:25 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:28:25 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:28:25 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:28:25 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:28:25 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:28:25 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:28:25 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:28:20Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesSnapshotTestJSON-instance-331359218',display_name='tempest-VolumesSnapshotTestJSON-instance-331359218',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-volumessnapshottestjson-instance-331359218',id=32,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBK3B9vHv/qSnUSCwj6Rdzu2263gZuLWjCHjmlx8FcOAqP+5T4EY3PruhW7fbCfv9oefdX6GMInUABslwsRJw+NQCIBUj+l8LlSPF2Ts4uLlBIlXGEb16HrWbz9sTVz9uHQ==',key_name='tempest-keypair-1656343598',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='1c8ff52319444079b754f79843c67b2b',ramdisk_id='',reservation_id='r-w9rkeroj',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesSnapshotTestJSON-343261326',owner_user_name='tempest-VolumesSnapshotTestJSON-343261326-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:28:22Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='97267b9f897e4034b71627a698210071',uuid=9df16e46-d097-4553-b44c-732ea24a4169,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "e8c25a1c-a605-4d7a-bb03-395b3d844571", "address": "fa:16:3e:69:6a:fa", "network": {"id": "96018a29-3b2e-44de-b2c4-67abc92e3a30", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-924549970-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "1c8ff52319444079b754f79843c67b2b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape8c25a1c-a6", "ovs_interfaceid": "e8c25a1c-a605-4d7a-bb03-395b3d844571", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:28:25 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Converting VIF {"id": "e8c25a1c-a605-4d7a-bb03-395b3d844571", "address": "fa:16:3e:69:6a:fa", "network": {"id": "96018a29-3b2e-44de-b2c4-67abc92e3a30", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-924549970-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "1c8ff52319444079b754f79843c67b2b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape8c25a1c-a6", "ovs_interfaceid": "e8c25a1c-a605-4d7a-bb03-395b3d844571", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:28:25 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:69:6a:fa,bridge_name='br-int',has_traffic_filtering=True,id=e8c25a1c-a605-4d7a-bb03-395b3d844571,network=Network(96018a29-3b2e-44de-b2c4-67abc92e3a30),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape8c25a1c-a6') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:28:25 user nova-compute[72351]: DEBUG nova.objects.instance [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lazy-loading 'pci_devices' on Instance uuid 9df16e46-d097-4553-b44c-732ea24a4169 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:28:25 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:28:25 user nova-compute[72351]:   <uuid>9df16e46-d097-4553-b44c-732ea24a4169</uuid>
Aug 09 02:28:25 user nova-compute[72351]:   <name>instance-00000020</name>
Aug 09 02:28:25 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:28:25 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:28:25 user nova-compute[72351]:   <metadata>
Aug 09 02:28:25 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:28:25 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:28:25 user nova-compute[72351]:       <nova:name>tempest-VolumesSnapshotTestJSON-instance-331359218</nova:name>
Aug 09 02:28:25 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:28:25</nova:creationTime>
Aug 09 02:28:25 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:28:25 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:28:25 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:28:25 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:28:25 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:28:25 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:28:25 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:28:25 user nova-compute[72351]:       <nova:owner>
Aug 09 02:28:25 user nova-compute[72351]:         <nova:user uuid="97267b9f897e4034b71627a698210071">tempest-VolumesSnapshotTestJSON-343261326-project-member</nova:user>
Aug 09 02:28:25 user nova-compute[72351]:         <nova:project uuid="1c8ff52319444079b754f79843c67b2b">tempest-VolumesSnapshotTestJSON-343261326</nova:project>
Aug 09 02:28:25 user nova-compute[72351]:       </nova:owner>
Aug 09 02:28:25 user nova-compute[72351]:       <nova:root type="image" uuid="18ccda7f-31d3-4978-ab9d-4ed2f74eff94"/>
Aug 09 02:28:25 user nova-compute[72351]:       <nova:ports>
Aug 09 02:28:25 user nova-compute[72351]:         <nova:port uuid="e8c25a1c-a605-4d7a-bb03-395b3d844571">
Aug 09 02:28:25 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.10" ipVersion="4"/>
Aug 09 02:28:25 user nova-compute[72351]:         </nova:port>
Aug 09 02:28:25 user nova-compute[72351]:       </nova:ports>
Aug 09 02:28:25 user nova-compute[72351]:     </nova:instance>
Aug 09 02:28:25 user nova-compute[72351]:   </metadata>
Aug 09 02:28:25 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:28:25 user nova-compute[72351]:     <system>
Aug 09 02:28:25 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:28:25 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:28:25 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:28:25 user nova-compute[72351]:       <entry name="serial">9df16e46-d097-4553-b44c-732ea24a4169</entry>
Aug 09 02:28:25 user nova-compute[72351]:       <entry name="uuid">9df16e46-d097-4553-b44c-732ea24a4169</entry>
Aug 09 02:28:25 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:28:25 user nova-compute[72351]:     </system>
Aug 09 02:28:25 user nova-compute[72351]:   </sysinfo>
Aug 09 02:28:25 user nova-compute[72351]:   <os>
Aug 09 02:28:25 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:28:25 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:28:25 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:28:25 user nova-compute[72351]:   </os>
Aug 09 02:28:25 user nova-compute[72351]:   <features>
Aug 09 02:28:25 user nova-compute[72351]:     <acpi/>
Aug 09 02:28:25 user nova-compute[72351]:     <apic/>
Aug 09 02:28:25 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:28:25 user nova-compute[72351]:   </features>
Aug 09 02:28:25 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:28:25 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:28:25 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:28:25 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:28:25 user nova-compute[72351]:   </clock>
Aug 09 02:28:25 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:28:25 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:28:25 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:28:25 user nova-compute[72351]:   </cpu>
Aug 09 02:28:25 user nova-compute[72351]:   <devices>
Aug 09 02:28:25 user nova-compute[72351]:     <disk type="file" device="disk">
Aug 09 02:28:25 user nova-compute[72351]:       <driver name="qemu" type="qcow2" cache="none"/>
Aug 09 02:28:25 user nova-compute[72351]:       <source file="/opt/stack/data/nova/instances/9df16e46-d097-4553-b44c-732ea24a4169/disk"/>
Aug 09 02:28:25 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:28:25 user nova-compute[72351]:     </disk>
Aug 09 02:28:25 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:28:25 user nova-compute[72351]:       <mac address="fa:16:3e:69:6a:fa"/>
Aug 09 02:28:25 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:28:25 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:28:25 user nova-compute[72351]:       <target dev="tape8c25a1c-a6"/>
Aug 09 02:28:25 user nova-compute[72351]:     </interface>
Aug 09 02:28:25 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:28:25 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/9df16e46-d097-4553-b44c-732ea24a4169/console.log" append="off"/>
Aug 09 02:28:25 user nova-compute[72351]:     </serial>
Aug 09 02:28:25 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:28:25 user nova-compute[72351]:     <video>
Aug 09 02:28:25 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:28:25 user nova-compute[72351]:     </video>
Aug 09 02:28:25 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:28:25 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:28:25 user nova-compute[72351]:     </rng>
Aug 09 02:28:25 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:28:25 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:28:25 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:28:25 user nova-compute[72351]:     </memballoon>
Aug 09 02:28:25 user nova-compute[72351]:   </devices>
Aug 09 02:28:25 user nova-compute[72351]: </domain>
Aug 09 02:28:25 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:28:25 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:28:20Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesSnapshotTestJSON-instance-331359218',display_name='tempest-VolumesSnapshotTestJSON-instance-331359218',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-volumessnapshottestjson-instance-331359218',id=32,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBK3B9vHv/qSnUSCwj6Rdzu2263gZuLWjCHjmlx8FcOAqP+5T4EY3PruhW7fbCfv9oefdX6GMInUABslwsRJw+NQCIBUj+l8LlSPF2Ts4uLlBIlXGEb16HrWbz9sTVz9uHQ==',key_name='tempest-keypair-1656343598',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='1c8ff52319444079b754f79843c67b2b',ramdisk_id='',reservation_id='r-w9rkeroj',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesSnapshotTestJSON-343261326',owner_user_name='tempest-VolumesSnapshotTestJSON-343261326-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:28:22Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='97267b9f897e4034b71627a698210071',uuid=9df16e46-d097-4553-b44c-732ea24a4169,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "e8c25a1c-a605-4d7a-bb03-395b3d844571", "address": "fa:16:3e:69:6a:fa", "network": {"id": "96018a29-3b2e-44de-b2c4-67abc92e3a30", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-924549970-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "1c8ff52319444079b754f79843c67b2b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape8c25a1c-a6", "ovs_interfaceid": "e8c25a1c-a605-4d7a-bb03-395b3d844571", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:28:25 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Converting VIF {"id": "e8c25a1c-a605-4d7a-bb03-395b3d844571", "address": "fa:16:3e:69:6a:fa", "network": {"id": "96018a29-3b2e-44de-b2c4-67abc92e3a30", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-924549970-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "1c8ff52319444079b754f79843c67b2b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape8c25a1c-a6", "ovs_interfaceid": "e8c25a1c-a605-4d7a-bb03-395b3d844571", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:28:25 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:69:6a:fa,bridge_name='br-int',has_traffic_filtering=True,id=e8c25a1c-a605-4d7a-bb03-395b3d844571,network=Network(96018a29-3b2e-44de-b2c4-67abc92e3a30),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape8c25a1c-a6') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:28:25 user nova-compute[72351]: DEBUG os_vif [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:69:6a:fa,bridge_name='br-int',has_traffic_filtering=True,id=e8c25a1c-a605-4d7a-bb03-395b3d844571,network=Network(96018a29-3b2e-44de-b2c4-67abc92e3a30),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape8c25a1c-a6') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:28:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:28:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:28:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:28:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:28:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tape8c25a1c-a6, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:28:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tape8c25a1c-a6, col_values=(('external_ids', {'iface-id': 'e8c25a1c-a605-4d7a-bb03-395b3d844571', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:69:6a:fa', 'vm-uuid': '9df16e46-d097-4553-b44c-732ea24a4169'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:28:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:28:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:28:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:28:25 user nova-compute[72351]: INFO os_vif [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:69:6a:fa,bridge_name='br-int',has_traffic_filtering=True,id=e8c25a1c-a605-4d7a-bb03-395b3d844571,network=Network(96018a29-3b2e-44de-b2c4-67abc92e3a30),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape8c25a1c-a6')
Aug 09 02:28:25 user nova-compute[72351]: DEBUG nova.compute.manager [req-8cd38a77-9dd6-40fc-86c3-39210e07e5a2 req-01a57464-352a-4dac-a8ad-7694391b16ea service nova] [instance: 957c1510-1320-4492-9052-a58ff540319d] Received event network-vif-plugged-e679cfd9-123f-4b88-8fd2-397e4c8616cc {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:28:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-8cd38a77-9dd6-40fc-86c3-39210e07e5a2 req-01a57464-352a-4dac-a8ad-7694391b16ea service nova] Acquiring lock "957c1510-1320-4492-9052-a58ff540319d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:28:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-8cd38a77-9dd6-40fc-86c3-39210e07e5a2 req-01a57464-352a-4dac-a8ad-7694391b16ea service nova] Lock "957c1510-1320-4492-9052-a58ff540319d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:28:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-8cd38a77-9dd6-40fc-86c3-39210e07e5a2 req-01a57464-352a-4dac-a8ad-7694391b16ea service nova] Lock "957c1510-1320-4492-9052-a58ff540319d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:28:25 user nova-compute[72351]: DEBUG nova.compute.manager [req-8cd38a77-9dd6-40fc-86c3-39210e07e5a2 req-01a57464-352a-4dac-a8ad-7694391b16ea service nova] [instance: 957c1510-1320-4492-9052-a58ff540319d] No waiting events found dispatching network-vif-plugged-e679cfd9-123f-4b88-8fd2-397e4c8616cc {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:28:25 user nova-compute[72351]: WARNING nova.compute.manager [req-8cd38a77-9dd6-40fc-86c3-39210e07e5a2 req-01a57464-352a-4dac-a8ad-7694391b16ea service nova] [instance: 957c1510-1320-4492-9052-a58ff540319d] Received unexpected event network-vif-plugged-e679cfd9-123f-4b88-8fd2-397e4c8616cc for instance with vm_state active and task_state rescuing.
Aug 09 02:28:25 user nova-compute[72351]: DEBUG nova.compute.manager [req-8cd38a77-9dd6-40fc-86c3-39210e07e5a2 req-01a57464-352a-4dac-a8ad-7694391b16ea service nova] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Received event network-changed-e8c25a1c-a605-4d7a-bb03-395b3d844571 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:28:25 user nova-compute[72351]: DEBUG nova.compute.manager [req-8cd38a77-9dd6-40fc-86c3-39210e07e5a2 req-01a57464-352a-4dac-a8ad-7694391b16ea service nova] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Refreshing instance network info cache due to event network-changed-e8c25a1c-a605-4d7a-bb03-395b3d844571. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:28:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-8cd38a77-9dd6-40fc-86c3-39210e07e5a2 req-01a57464-352a-4dac-a8ad-7694391b16ea service nova] Acquiring lock "refresh_cache-9df16e46-d097-4553-b44c-732ea24a4169" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:28:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-8cd38a77-9dd6-40fc-86c3-39210e07e5a2 req-01a57464-352a-4dac-a8ad-7694391b16ea service nova] Acquired lock "refresh_cache-9df16e46-d097-4553-b44c-732ea24a4169" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:28:25 user nova-compute[72351]: DEBUG nova.network.neutron [req-8cd38a77-9dd6-40fc-86c3-39210e07e5a2 req-01a57464-352a-4dac-a8ad-7694391b16ea service nova] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Refreshing network info cache for port e8c25a1c-a605-4d7a-bb03-395b3d844571 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:28:25 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:28:25 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] No VIF found with MAC fa:16:3e:69:6a:fa, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:28:25 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Removed pending event for 957c1510-1320-4492-9052-a58ff540319d due to event {{(pid=72351) _event_emit_delayed /opt/stack/nova/nova/virt/libvirt/host.py:438}}
Aug 09 02:28:25 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548105.3339791, 957c1510-1320-4492-9052-a58ff540319d => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:28:25 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 957c1510-1320-4492-9052-a58ff540319d] VM Resumed (Lifecycle Event)
Aug 09 02:28:25 user nova-compute[72351]: DEBUG nova.compute.manager [None req-ba1e8c17-437c-4c55-b219-88582604164e tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 957c1510-1320-4492-9052-a58ff540319d] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:28:25 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 957c1510-1320-4492-9052-a58ff540319d] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:28:25 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 957c1510-1320-4492-9052-a58ff540319d] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: active, current task_state: rescuing, current DB power_state: 1, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:28:25 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 957c1510-1320-4492-9052-a58ff540319d] During sync_power_state the instance has a pending task (rescuing). Skip.
Aug 09 02:28:25 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548105.3356757, 957c1510-1320-4492-9052-a58ff540319d => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:28:25 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 957c1510-1320-4492-9052-a58ff540319d] VM Started (Lifecycle Event)
Aug 09 02:28:25 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 957c1510-1320-4492-9052-a58ff540319d] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:28:25 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 957c1510-1320-4492-9052-a58ff540319d] Synchronizing instance power state after lifecycle event "Started"; current vm_state: active, current task_state: rescuing, current DB power_state: 1, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:28:25 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 957c1510-1320-4492-9052-a58ff540319d] During sync_power_state the instance has a pending task (rescuing). Skip.
Aug 09 02:28:25 user nova-compute[72351]: DEBUG nova.network.neutron [req-8cd38a77-9dd6-40fc-86c3-39210e07e5a2 req-01a57464-352a-4dac-a8ad-7694391b16ea service nova] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Updated VIF entry in instance network info cache for port e8c25a1c-a605-4d7a-bb03-395b3d844571. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:28:25 user nova-compute[72351]: DEBUG nova.network.neutron [req-8cd38a77-9dd6-40fc-86c3-39210e07e5a2 req-01a57464-352a-4dac-a8ad-7694391b16ea service nova] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Updating instance_info_cache with network_info: [{"id": "e8c25a1c-a605-4d7a-bb03-395b3d844571", "address": "fa:16:3e:69:6a:fa", "network": {"id": "96018a29-3b2e-44de-b2c4-67abc92e3a30", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-924549970-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "1c8ff52319444079b754f79843c67b2b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape8c25a1c-a6", "ovs_interfaceid": "e8c25a1c-a605-4d7a-bb03-395b3d844571", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:28:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-8cd38a77-9dd6-40fc-86c3-39210e07e5a2 req-01a57464-352a-4dac-a8ad-7694391b16ea service nova] Releasing lock "refresh_cache-9df16e46-d097-4553-b44c-732ea24a4169" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:28:26 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:28:26 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:28:26 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:28:27 user nova-compute[72351]: DEBUG nova.compute.manager [req-8d5840b7-ac14-4c96-86c7-8ae730155180 req-26ae3b8a-542b-448d-b561-66f9eee5abd3 service nova] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Received event network-vif-plugged-e8c25a1c-a605-4d7a-bb03-395b3d844571 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:28:27 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-8d5840b7-ac14-4c96-86c7-8ae730155180 req-26ae3b8a-542b-448d-b561-66f9eee5abd3 service nova] Acquiring lock "9df16e46-d097-4553-b44c-732ea24a4169-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:28:27 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-8d5840b7-ac14-4c96-86c7-8ae730155180 req-26ae3b8a-542b-448d-b561-66f9eee5abd3 service nova] Lock "9df16e46-d097-4553-b44c-732ea24a4169-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:28:27 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-8d5840b7-ac14-4c96-86c7-8ae730155180 req-26ae3b8a-542b-448d-b561-66f9eee5abd3 service nova] Lock "9df16e46-d097-4553-b44c-732ea24a4169-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:28:27 user nova-compute[72351]: DEBUG nova.compute.manager [req-8d5840b7-ac14-4c96-86c7-8ae730155180 req-26ae3b8a-542b-448d-b561-66f9eee5abd3 service nova] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] No waiting events found dispatching network-vif-plugged-e8c25a1c-a605-4d7a-bb03-395b3d844571 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:28:27 user nova-compute[72351]: WARNING nova.compute.manager [req-8d5840b7-ac14-4c96-86c7-8ae730155180 req-26ae3b8a-542b-448d-b561-66f9eee5abd3 service nova] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Received unexpected event network-vif-plugged-e8c25a1c-a605-4d7a-bb03-395b3d844571 for instance with vm_state building and task_state spawning.
Aug 09 02:28:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:28:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:28:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:28:28 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:28:28 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548108.976501, 9df16e46-d097-4553-b44c-732ea24a4169 => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:28:28 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] VM Resumed (Lifecycle Event)
Aug 09 02:28:28 user nova-compute[72351]: DEBUG nova.compute.manager [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:28:28 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Guest created on hypervisor {{(pid=72351) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4411}}
Aug 09 02:28:28 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Instance spawned successfully.
Aug 09 02:28:28 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:894}}
Aug 09 02:28:29 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Found default for hw_cdrom_bus of ide {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:28:29 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Found default for hw_disk_bus of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:28:29 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Found default for hw_input_bus of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:28:29 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Found default for hw_pointer_model of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:28:29 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Found default for hw_video_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:28:29 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Found default for hw_vif_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:28:29 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:28:29 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:28:29 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:28:29 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548108.978456, 9df16e46-d097-4553-b44c-732ea24a4169 => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:28:29 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] VM Started (Lifecycle Event)
Aug 09 02:28:29 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:28:29 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:28:29 user nova-compute[72351]: INFO nova.compute.manager [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Took 6.84 seconds to spawn the instance on the hypervisor.
Aug 09 02:28:29 user nova-compute[72351]: DEBUG nova.compute.manager [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:28:29 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:28:29 user nova-compute[72351]: INFO nova.compute.manager [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Took 7.71 seconds to build instance.
Aug 09 02:28:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1ea63ce7-87cb-4b7d-92ea-f0e43040cae9 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lock "9df16e46-d097-4553-b44c-732ea24a4169" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 7.849s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:28:29 user nova-compute[72351]: DEBUG nova.compute.manager [req-d89ebf8a-04d8-4ca8-af51-0145c35f7362 req-6acb4eb5-f2d9-4429-82ac-3c2ceaeaab1c service nova] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Received event network-vif-plugged-e8c25a1c-a605-4d7a-bb03-395b3d844571 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:28:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d89ebf8a-04d8-4ca8-af51-0145c35f7362 req-6acb4eb5-f2d9-4429-82ac-3c2ceaeaab1c service nova] Acquiring lock "9df16e46-d097-4553-b44c-732ea24a4169-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:28:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d89ebf8a-04d8-4ca8-af51-0145c35f7362 req-6acb4eb5-f2d9-4429-82ac-3c2ceaeaab1c service nova] Lock "9df16e46-d097-4553-b44c-732ea24a4169-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:28:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d89ebf8a-04d8-4ca8-af51-0145c35f7362 req-6acb4eb5-f2d9-4429-82ac-3c2ceaeaab1c service nova] Lock "9df16e46-d097-4553-b44c-732ea24a4169-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:28:29 user nova-compute[72351]: DEBUG nova.compute.manager [req-d89ebf8a-04d8-4ca8-af51-0145c35f7362 req-6acb4eb5-f2d9-4429-82ac-3c2ceaeaab1c service nova] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] No waiting events found dispatching network-vif-plugged-e8c25a1c-a605-4d7a-bb03-395b3d844571 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:28:29 user nova-compute[72351]: WARNING nova.compute.manager [req-d89ebf8a-04d8-4ca8-af51-0145c35f7362 req-6acb4eb5-f2d9-4429-82ac-3c2ceaeaab1c service nova] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Received unexpected event network-vif-plugged-e8c25a1c-a605-4d7a-bb03-395b3d844571 for instance with vm_state active and task_state None.
Aug 09 02:28:30 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:28:33 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:28:33 user nova-compute[72351]: DEBUG nova.compute.manager [req-47cccd60-2be3-4a86-a449-5cf80e8dbcb1 req-8dccef64-a1cf-4400-a625-ae2315f5ae04 service nova] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Received event network-changed-02c9ff8a-e779-443c-9b3b-567d2e69cea0 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:28:33 user nova-compute[72351]: DEBUG nova.compute.manager [req-47cccd60-2be3-4a86-a449-5cf80e8dbcb1 req-8dccef64-a1cf-4400-a625-ae2315f5ae04 service nova] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Refreshing instance network info cache due to event network-changed-02c9ff8a-e779-443c-9b3b-567d2e69cea0. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:28:33 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-47cccd60-2be3-4a86-a449-5cf80e8dbcb1 req-8dccef64-a1cf-4400-a625-ae2315f5ae04 service nova] Acquiring lock "refresh_cache-831c864a-75d9-42b3-b952-749f4201e40e" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:28:33 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-47cccd60-2be3-4a86-a449-5cf80e8dbcb1 req-8dccef64-a1cf-4400-a625-ae2315f5ae04 service nova] Acquired lock "refresh_cache-831c864a-75d9-42b3-b952-749f4201e40e" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:28:33 user nova-compute[72351]: DEBUG nova.network.neutron [req-47cccd60-2be3-4a86-a449-5cf80e8dbcb1 req-8dccef64-a1cf-4400-a625-ae2315f5ae04 service nova] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Refreshing network info cache for port 02c9ff8a-e779-443c-9b3b-567d2e69cea0 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:28:34 user nova-compute[72351]: DEBUG nova.network.neutron [req-47cccd60-2be3-4a86-a449-5cf80e8dbcb1 req-8dccef64-a1cf-4400-a625-ae2315f5ae04 service nova] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Updated VIF entry in instance network info cache for port 02c9ff8a-e779-443c-9b3b-567d2e69cea0. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:28:34 user nova-compute[72351]: DEBUG nova.network.neutron [req-47cccd60-2be3-4a86-a449-5cf80e8dbcb1 req-8dccef64-a1cf-4400-a625-ae2315f5ae04 service nova] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Updating instance_info_cache with network_info: [{"id": "02c9ff8a-e779-443c-9b3b-567d2e69cea0", "address": "fa:16:3e:32:5d:c2", "network": {"id": "7024d1f6-a400-4482-9364-78ae3b6adc80", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1603258958-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.94", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c2f67da658d94fabbab82ee6a6171c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap02c9ff8a-e7", "ovs_interfaceid": "02c9ff8a-e779-443c-9b3b-567d2e69cea0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:28:34 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-47cccd60-2be3-4a86-a449-5cf80e8dbcb1 req-8dccef64-a1cf-4400-a625-ae2315f5ae04 service nova] Releasing lock "refresh_cache-831c864a-75d9-42b3-b952-749f4201e40e" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:28:34 user nova-compute[72351]: DEBUG nova.compute.manager [req-dacbe9aa-fa56-48d5-b051-fc7df8b53432 req-2423c88b-0764-4aff-8768-278519243b19 service nova] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Received event network-changed-02c9ff8a-e779-443c-9b3b-567d2e69cea0 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:28:34 user nova-compute[72351]: DEBUG nova.compute.manager [req-dacbe9aa-fa56-48d5-b051-fc7df8b53432 req-2423c88b-0764-4aff-8768-278519243b19 service nova] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Refreshing instance network info cache due to event network-changed-02c9ff8a-e779-443c-9b3b-567d2e69cea0. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:28:34 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-dacbe9aa-fa56-48d5-b051-fc7df8b53432 req-2423c88b-0764-4aff-8768-278519243b19 service nova] Acquiring lock "refresh_cache-831c864a-75d9-42b3-b952-749f4201e40e" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:28:34 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-dacbe9aa-fa56-48d5-b051-fc7df8b53432 req-2423c88b-0764-4aff-8768-278519243b19 service nova] Acquired lock "refresh_cache-831c864a-75d9-42b3-b952-749f4201e40e" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:28:34 user nova-compute[72351]: DEBUG nova.network.neutron [req-dacbe9aa-fa56-48d5-b051-fc7df8b53432 req-2423c88b-0764-4aff-8768-278519243b19 service nova] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Refreshing network info cache for port 02c9ff8a-e779-443c-9b3b-567d2e69cea0 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:28:34 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1e45f657-cc89-4c45-8660-3a2ef2d520f7 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Acquiring lock "831c864a-75d9-42b3-b952-749f4201e40e" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:28:34 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1e45f657-cc89-4c45-8660-3a2ef2d520f7 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "831c864a-75d9-42b3-b952-749f4201e40e" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:28:34 user nova-compute[72351]: INFO nova.compute.manager [None req-1e45f657-cc89-4c45-8660-3a2ef2d520f7 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Detaching volume 5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e
Aug 09 02:28:34 user nova-compute[72351]: INFO nova.virt.block_device [None req-1e45f657-cc89-4c45-8660-3a2ef2d520f7 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Attempting to driver detach volume 5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e from mountpoint /dev/vdb
Aug 09 02:28:34 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-1e45f657-cc89-4c45-8660-3a2ef2d520f7 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Attempting to detach device vdb from instance 831c864a-75d9-42b3-b952-749f4201e40e from the persistent domain config. {{(pid=72351) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2477}}
Aug 09 02:28:34 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-1e45f657-cc89-4c45-8660-3a2ef2d520f7 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 02:28:34 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:28:34 user nova-compute[72351]:   <source dev="/dev/sde"/>
Aug 09 02:28:34 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:28:34 user nova-compute[72351]:   <serial>5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e</serial>
Aug 09 02:28:34 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Aug 09 02:28:34 user nova-compute[72351]: </disk>
Aug 09 02:28:34 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 02:28:34 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-1e45f657-cc89-4c45-8660-3a2ef2d520f7 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Successfully detached device vdb from instance 831c864a-75d9-42b3-b952-749f4201e40e from the persistent domain config.
Aug 09 02:28:34 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-1e45f657-cc89-4c45-8660-3a2ef2d520f7 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 831c864a-75d9-42b3-b952-749f4201e40e from the live domain config. {{(pid=72351) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2513}}
Aug 09 02:28:34 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-1e45f657-cc89-4c45-8660-3a2ef2d520f7 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 02:28:34 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:28:34 user nova-compute[72351]:   <source dev="/dev/sde"/>
Aug 09 02:28:34 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:28:34 user nova-compute[72351]:   <serial>5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e</serial>
Aug 09 02:28:34 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Aug 09 02:28:34 user nova-compute[72351]: </disk>
Aug 09 02:28:34 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 02:28:34 user nova-compute[72351]: DEBUG nova.network.neutron [req-dacbe9aa-fa56-48d5-b051-fc7df8b53432 req-2423c88b-0764-4aff-8768-278519243b19 service nova] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Updated VIF entry in instance network info cache for port 02c9ff8a-e779-443c-9b3b-567d2e69cea0. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:28:34 user nova-compute[72351]: DEBUG nova.network.neutron [req-dacbe9aa-fa56-48d5-b051-fc7df8b53432 req-2423c88b-0764-4aff-8768-278519243b19 service nova] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Updating instance_info_cache with network_info: [{"id": "02c9ff8a-e779-443c-9b3b-567d2e69cea0", "address": "fa:16:3e:32:5d:c2", "network": {"id": "7024d1f6-a400-4482-9364-78ae3b6adc80", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1603258958-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c2f67da658d94fabbab82ee6a6171c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap02c9ff8a-e7", "ovs_interfaceid": "02c9ff8a-e779-443c-9b3b-567d2e69cea0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:28:34 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-dacbe9aa-fa56-48d5-b051-fc7df8b53432 req-2423c88b-0764-4aff-8768-278519243b19 service nova] Releasing lock "refresh_cache-831c864a-75d9-42b3-b952-749f4201e40e" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:28:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Received event <DeviceRemovedEvent: 1691548115.0866444, 831c864a-75d9-42b3-b952-749f4201e40e => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=72351) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2360}}
Aug 09 02:28:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-1e45f657-cc89-4c45-8660-3a2ef2d520f7 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 831c864a-75d9-42b3-b952-749f4201e40e {{(pid=72351) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2589}}
Aug 09 02:28:35 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-1e45f657-cc89-4c45-8660-3a2ef2d520f7 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Successfully detached device vdb from instance 831c864a-75d9-42b3-b952-749f4201e40e from the live domain config.
Aug 09 02:28:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-1e45f657-cc89-4c45-8660-3a2ef2d520f7 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 02:28:35 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1e45f657-cc89-4c45-8660-3a2ef2d520f7 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'HmkUMfVE', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sde'}, None), 'kwargs': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:28:35 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-1e45f657-cc89-4c45-8660-3a2ef2d520f7 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:28:35 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-1e45f657-cc89-4c45-8660-3a2ef2d520f7 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:28:35 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-1e45f657-cc89-4c45-8660-3a2ef2d520f7 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:28:35 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1e45f657-cc89-4c45-8660-3a2ef2d520f7 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 02:28:35 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:35 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:35 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a3aff68f-fed5-4cf5-9dee-78a69f4a7cae]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:28:35 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:35 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.027s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:35 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[2ebe3219-de46-451b-a3ce-dd425554a8e2]: (4, ('tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)\ntcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)\ntcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)\ntcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950 (non-flash)\ntcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:28:35 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1e45f657-cc89-4c45-8660-3a2ef2d520f7 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)
Aug 09 02:28:35 user nova-compute[72351]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)
Aug 09 02:28:35 user nova-compute[72351]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:28:35 user nova-compute[72351]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:28:35 user nova-compute[72351]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950 (non-flash)
Aug 09 02:28:35 user nova-compute[72351]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:28:35 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:28:35 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1e45f657-cc89-4c45-8660-3a2ef2d520f7 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] iscsi session list stdout=tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e (non-flash)
Aug 09 02:28:35 user nova-compute[72351]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)
Aug 09 02:28:35 user nova-compute[72351]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:28:35 user nova-compute[72351]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:28:35 user nova-compute[72351]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950 (non-flash)
Aug 09 02:28:35 user nova-compute[72351]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:28:35 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:28:35 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1e45f657-cc89-4c45-8660-3a2ef2d520f7 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fcfdc1a4940>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e'): ({'sde'}, set())}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 02:28:35 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-1e45f657-cc89-4c45-8660-3a2ef2d520f7 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Removing single pathed devices sde {{(pid=72351) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Aug 09 02:28:35 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:35 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:28:35 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.026s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:35 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[5670443a-9c6c-48fe-a034-a9216dc17d49]: (4, ('path checker states:\nup                  5\npending             1\n\npaths: 0\nbusy: False\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:28:35 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sde {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:35 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sde" returned: 0 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:35 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ac033c1c-e9cf-445f-b994-319dc90dc75c]: (4, ('ok\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:28:35 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-1e45f657-cc89-4c45-8660-3a2ef2d520f7 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Flushing IO for device /dev/sde {{(pid=72351) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Aug 09 02:28:35 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sde {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:35 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sde" returned: 0 in 0.016s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:35 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[3273dc83-d595-4427-a6e4-e98ee36ae284]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:28:35 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-1e45f657-cc89-4c45-8660-3a2ef2d520f7 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Remove SCSI device /dev/sde with /sys/block/sde/device/delete {{(pid=72351) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Aug 09 02:28:35 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sde/device/delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:35 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sde/device/delete" returned: 0 in 0.048s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:35 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[04990000-d9ef-43f0-a1bd-f8e214306502]: (4, ('1', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:28:35 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-1e45f657-cc89-4c45-8660-3a2ef2d520f7 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Checking to see if SCSI volumes sde have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Aug 09 02:28:35 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-1e45f657-cc89-4c45-8660-3a2ef2d520f7 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] SCSI volumes sde have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Aug 09 02:28:35 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1e45f657-cc89-4c45-8660-3a2ef2d520f7 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e')] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 02:28:35 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:35 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:35 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[0780dd41-b4f0-4440-a567-6001a2ddb5ec]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:28:35 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1e45f657-cc89-4c45-8660-3a2ef2d520f7 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:28:35 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e -p 172.16.0.220:3260 --logout {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:35 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e -p 172.16.0.220:3260 --logout" returned: 0 in 0.065s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:35 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[47150e0e-84e2-4c41-94c2-83e8fb5f0e6f]: (4, ('Logging out of session [sid: 24, target: iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e, portal: 172.16.0.220,3260]\nLogout of [sid: 24, target: iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:28:35 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1e45f657-cc89-4c45-8660-3a2ef2d520f7 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 24, target: iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e, portal: 172.16.0.220,3260]
Aug 09 02:28:35 user nova-compute[72351]: Logout of [sid: 24, target: iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e, portal: 172.16.0.220,3260] successful.
Aug 09 02:28:35 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:28:35 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e -p 172.16.0.220:3260 --op delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:28:35 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5d46f954-9538-4e4f-9a3f-a6c0cb9ffe7e -p 172.16.0.220:3260 --op delete" returned: 0 in 0.035s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:28:35 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[68b29563-2a5c-47c4-bce5-4cfa26e76013]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:28:35 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1e45f657-cc89-4c45-8660-3a2ef2d520f7 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:28:35 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-1e45f657-cc89-4c45-8660-3a2ef2d520f7 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.471s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:28:35 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1e45f657-cc89-4c45-8660-3a2ef2d520f7 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] <== disconnect_volume: return (473ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:28:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-1e45f657-cc89-4c45-8660-3a2ef2d520f7 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 02:28:36 user nova-compute[72351]: DEBUG nova.objects.instance [None req-1e45f657-cc89-4c45-8660-3a2ef2d520f7 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lazy-loading 'flavor' on Instance uuid 831c864a-75d9-42b3-b952-749f4201e40e {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:28:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1e45f657-cc89-4c45-8660-3a2ef2d520f7 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "831c864a-75d9-42b3-b952-749f4201e40e" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 2.016s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:28:38 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:28:40 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:28:43 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:28:45 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:28:48 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:28:50 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:28:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:28:55 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:28:56 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:28:58 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:28:58 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:29:00 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:29:00 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:29:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:29:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:29:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:29:00 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Auditing locally available compute resources for user (node: user) {{(pid=72351) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Aug 09 02:29:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json" returned: 0 in 0.147s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:01 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdf (vdb) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:29:01 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdg (vda) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:29:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/957c1510-1320-4492-9052-a58ff540319d/disk.rescue --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/957c1510-1320-4492-9052-a58ff540319d/disk.rescue --force-share --output=json" returned: 0 in 0.153s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/957c1510-1320-4492-9052-a58ff540319d/disk.rescue --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/957c1510-1320-4492-9052-a58ff540319d/disk.rescue --force-share --output=json" returned: 0 in 0.133s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:01 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdh (vda) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:29:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/47afce95-e5f7-4703-a99f-7ef8b28c4bd2/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/47afce95-e5f7-4703-a99f-7ef8b28c4bd2/disk --force-share --output=json" returned: 0 in 0.140s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:01 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/47afce95-e5f7-4703-a99f-7ef8b28c4bd2/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/47afce95-e5f7-4703-a99f-7ef8b28c4bd2/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.rescue --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.rescue --force-share --output=json" returned: 0 in 0.142s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.rescue --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.rescue --force-share --output=json" returned: 0 in 0.134s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json" returned: 0 in 0.132s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json" returned: 0 in 0.131s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json" returned: 0 in 0.130s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/16c1e558-baa1-4c2d-b13b-50585f74d587/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/16c1e558-baa1-4c2d-b13b-50585f74d587/disk --force-share --output=json" returned: 0 in 0.142s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/16c1e558-baa1-4c2d-b13b-50585f74d587/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:29:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/16c1e558-baa1-4c2d-b13b-50585f74d587/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:03 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdc (vdb) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:29:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json" returned: 0 in 0.145s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1b90316d-6638-4603-bb09-38bf0ed795e3/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1b90316d-6638-4603-bb09-38bf0ed795e3/disk --force-share --output=json" returned: 0 in 0.129s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1b90316d-6638-4603-bb09-38bf0ed795e3/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1b90316d-6638-4603-bb09-38bf0ed795e3/disk --force-share --output=json" returned: 0 in 0.127s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff/disk --force-share --output=json" returned: 0 in 0.147s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff/disk --force-share --output=json" returned: 0 in 0.147s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:04 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdb (vdb) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:29:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a03fbded-23ad-440b-97e7-bea07425e873/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a03fbded-23ad-440b-97e7-bea07425e873/disk --force-share --output=json" returned: 0 in 0.147s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a03fbded-23ad-440b-97e7-bea07425e873/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a03fbded-23ad-440b-97e7-bea07425e873/disk --force-share --output=json" returned: 0 in 0.132s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/831c864a-75d9-42b3-b952-749f4201e40e/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/831c864a-75d9-42b3-b952-749f4201e40e/disk --force-share --output=json" returned: 0 in 0.144s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/831c864a-75d9-42b3-b952-749f4201e40e/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:29:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/831c864a-75d9-42b3-b952-749f4201e40e/disk --force-share --output=json" returned: 0 in 0.142s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9df16e46-d097-4553-b44c-732ea24a4169/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9df16e46-d097-4553-b44c-732ea24a4169/disk --force-share --output=json" returned: 0 in 0.170s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9df16e46-d097-4553-b44c-732ea24a4169/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9df16e46-d097-4553-b44c-732ea24a4169/disk --force-share --output=json" returned: 0 in 0.140s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:29:05 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:29:05 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:29:05 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Hypervisor/Node resource view: name=user free_ram=6558MB free_disk=25.183551788330078GB free_vcpus=-1 pci_devices=[{"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72351) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Aug 09 02:29:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:29:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:29:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:29:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6a6ea4c2-5a7a-4151-9d47-e2602649997e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Acquiring lock "16c1e558-baa1-4c2d-b13b-50585f74d587" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:29:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6a6ea4c2-5a7a-4151-9d47-e2602649997e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Lock "16c1e558-baa1-4c2d-b13b-50585f74d587" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:29:05 user nova-compute[72351]: INFO nova.compute.manager [None req-6a6ea4c2-5a7a-4151-9d47-e2602649997e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Detaching volume a52af854-9d8e-4ec7-935a-b7b7b65d3950
Aug 09 02:29:05 user nova-compute[72351]: INFO nova.virt.block_device [None req-6a6ea4c2-5a7a-4151-9d47-e2602649997e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Attempting to driver detach volume a52af854-9d8e-4ec7-935a-b7b7b65d3950 from mountpoint /dev/vdb
Aug 09 02:29:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6a6ea4c2-5a7a-4151-9d47-e2602649997e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Attempting to detach device vdb from instance 16c1e558-baa1-4c2d-b13b-50585f74d587 from the persistent domain config. {{(pid=72351) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2477}}
Aug 09 02:29:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-6a6ea4c2-5a7a-4151-9d47-e2602649997e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 02:29:05 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:29:05 user nova-compute[72351]:   <source dev="/dev/sdc"/>
Aug 09 02:29:05 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:29:05 user nova-compute[72351]:   <serial>a52af854-9d8e-4ec7-935a-b7b7b65d3950</serial>
Aug 09 02:29:05 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Aug 09 02:29:05 user nova-compute[72351]: </disk>
Aug 09 02:29:05 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 02:29:05 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:29:05 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance eb86cb49-2018-42c2-bd97-c662b829775d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:29:05 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:29:05 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance a7f3d076-864b-4b68-9207-214ce2c21444 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:29:05 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 831c864a-75d9-42b3-b952-749f4201e40e actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:29:05 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 75bbbead-8628-448b-9484-19bf4ab922ff actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:29:05 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 47afce95-e5f7-4703-a99f-7ef8b28c4bd2 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:29:05 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 16c1e558-baa1-4c2d-b13b-50585f74d587 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:29:05 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 957c1510-1320-4492-9052-a58ff540319d actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:29:05 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 1b90316d-6638-4603-bb09-38bf0ed795e3 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:29:05 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance a03fbded-23ad-440b-97e7-bea07425e873 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:29:05 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 2a3bcdea-be36-4a5b-93cb-53396efdfa8e actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:29:05 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 9df16e46-d097-4553-b44c-732ea24a4169 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:29:05 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Total usable vcpus: 12, total allocated vcpus: 13 {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Aug 09 02:29:05 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Final resource view: name=user phys_ram=16011MB used_ram=7168MB phys_disk=40GB used_disk=11GB total_vcpus=12 used_vcpus=13 pci_stats=[] {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Aug 09 02:29:05 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-6a6ea4c2-5a7a-4151-9d47-e2602649997e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Successfully detached device vdb from instance 16c1e558-baa1-4c2d-b13b-50585f74d587 from the persistent domain config.
Aug 09 02:29:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6a6ea4c2-5a7a-4151-9d47-e2602649997e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 16c1e558-baa1-4c2d-b13b-50585f74d587 from the live domain config. {{(pid=72351) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2513}}
Aug 09 02:29:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-6a6ea4c2-5a7a-4151-9d47-e2602649997e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 02:29:05 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:29:05 user nova-compute[72351]:   <source dev="/dev/sdc"/>
Aug 09 02:29:05 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:29:05 user nova-compute[72351]:   <serial>a52af854-9d8e-4ec7-935a-b7b7b65d3950</serial>
Aug 09 02:29:05 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Aug 09 02:29:05 user nova-compute[72351]: </disk>
Aug 09 02:29:05 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 02:29:05 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Refreshing inventories for resource provider 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:804}}
Aug 09 02:29:05 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Updating ProviderTree inventory for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 from _refresh_and_get_inventory using data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) _refresh_and_get_inventory /opt/stack/nova/nova/scheduler/client/report.py:768}}
Aug 09 02:29:05 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Updating inventory in ProviderTree for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 with inventory: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:176}}
Aug 09 02:29:05 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Refreshing aggregate associations for resource provider 8eae9b37-1236-4e6d-91a5-b650de27b009, aggregates: None {{(pid=72351) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:813}}
Aug 09 02:29:05 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Refreshing trait associations for resource provider 8eae9b37-1236-4e6d-91a5-b650de27b009, traits: COMPUTE_VIOMMU_MODEL_INTEL,COMPUTE_STORAGE_BUS_IDE,COMPUTE_GRAPHICS_MODEL_NONE,COMPUTE_NET_VIF_MODEL_E1000E,COMPUTE_IMAGE_TYPE_AKI,COMPUTE_VOLUME_ATTACH_WITH_TAG,COMPUTE_NET_VIF_MODEL_VMXNET3,COMPUTE_SOCKET_PCI_NUMA_AFFINITY,COMPUTE_ACCELERATORS,HW_CPU_X86_SSE41,COMPUTE_NET_ATTACH_INTERFACE_WITH_TAG,COMPUTE_GRAPHICS_MODEL_VGA,COMPUTE_GRAPHICS_MODEL_QXL,COMPUTE_VIOMMU_MODEL_AUTO,COMPUTE_SECURITY_UEFI_SECURE_BOOT,COMPUTE_NET_VIF_MODEL_E1000,COMPUTE_TRUSTED_CERTS,COMPUTE_VOLUME_MULTI_ATTACH,COMPUTE_STORAGE_BUS_FDC,COMPUTE_VOLUME_EXTEND,COMPUTE_GRAPHICS_MODEL_CIRRUS,COMPUTE_GRAPHICS_MODEL_VMVGA,COMPUTE_GRAPHICS_MODEL_BOCHS,COMPUTE_DEVICE_TAGGING,COMPUTE_STORAGE_BUS_SCSI,COMPUTE_STORAGE_BUS_VIRTIO,HW_CPU_X86_SSSE3,HW_CPU_X86_SSE2,COMPUTE_IMAGE_TYPE_AMI,COMPUTE_STORAGE_BUS_USB,COMPUTE_NET_VIF_MODEL_PCNET,COMPUTE_NET_VIF_MODEL_VIRTIO,COMPUTE_IMAGE_TYPE_ISO,HW_CPU_X86_SSE42,HW_CPU_X86_SSE,COMPUTE_IMAGE_TYPE_ARI,COMPUTE_IMAGE_TYPE_RAW,COMPUTE_IMAGE_TYPE_QCOW2,COMPUTE_NET_VIF_MODEL_NE2K_PCI,COMPUTE_NODE,HW_CPU_X86_MMX,COMPUTE_NET_VIF_MODEL_RTL8139,COMPUTE_GRAPHICS_MODEL_VIRTIO,COMPUTE_NET_VIF_MODEL_SPAPR_VLAN,COMPUTE_STORAGE_BUS_SATA,COMPUTE_NET_ATTACH_INTERFACE,COMPUTE_RESCUE_BFV {{(pid=72351) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:825}}
Aug 09 02:29:06 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Received event <DeviceRemovedEvent: 1691548146.2112377, 16c1e558-baa1-4c2d-b13b-50585f74d587 => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=72351) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2360}}
Aug 09 02:29:06 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6a6ea4c2-5a7a-4151-9d47-e2602649997e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 16c1e558-baa1-4c2d-b13b-50585f74d587 {{(pid=72351) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2589}}
Aug 09 02:29:06 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-6a6ea4c2-5a7a-4151-9d47-e2602649997e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Successfully detached device vdb from instance 16c1e558-baa1-4c2d-b13b-50585f74d587 from the live domain config.
Aug 09 02:29:06 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-6a6ea4c2-5a7a-4151-9d47-e2602649997e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 02:29:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-6a6ea4c2-5a7a-4151-9d47-e2602649997e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'kyoaqdZV', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc'}, None), 'kwargs': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:29:06 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-6a6ea4c2-5a7a-4151-9d47-e2602649997e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:29:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-6a6ea4c2-5a7a-4151-9d47-e2602649997e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:29:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-6a6ea4c2-5a7a-4151-9d47-e2602649997e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:29:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-6a6ea4c2-5a7a-4151-9d47-e2602649997e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 02:29:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:06 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[f60efaa3-7f31-4497-9b92-5c54d573ac10]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:06 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:29:06 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:29:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.030s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:06 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d2625e73-e772-43d5-9ff3-363ea8ebabdf]: (4, ('tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)\ntcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)\ntcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950 (non-flash)\ntcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-6a6ea4c2-5a7a-4151-9d47-e2602649997e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)
Aug 09 02:29:06 user nova-compute[72351]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:29:06 user nova-compute[72351]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:29:06 user nova-compute[72351]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950 (non-flash)
Aug 09 02:29:06 user nova-compute[72351]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:29:06 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:29:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-6a6ea4c2-5a7a-4151-9d47-e2602649997e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] iscsi session list stdout=tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)
Aug 09 02:29:06 user nova-compute[72351]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:29:06 user nova-compute[72351]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:29:06 user nova-compute[72351]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950 (non-flash)
Aug 09 02:29:06 user nova-compute[72351]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:29:06 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:29:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-6a6ea4c2-5a7a-4151-9d47-e2602649997e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fcfdc421990>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950'): ({'sdc'}, set())}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 02:29:06 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-6a6ea4c2-5a7a-4151-9d47-e2602649997e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Removing single pathed devices sdc {{(pid=72351) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Aug 09 02:29:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:06 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Compute_service record updated for user:user {{(pid=72351) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Aug 09 02:29:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.825s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:29:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:06 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[56ef0116-678c-4486-9c3c-5ac7205cffbc]: (4, ('path checker states:\nup                  4\npending             1\n\npaths: 0\nbusy: False\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdc {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdc" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:06 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[9831b14f-594b-4c1b-9b61-8c77129e7a7b]: (4, ('ok\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:06 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-6a6ea4c2-5a7a-4151-9d47-e2602649997e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Flushing IO for device /dev/sdc {{(pid=72351) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Aug 09 02:29:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdc {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdc" returned: 0 in 0.013s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:06 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[02dcdd09-6c0a-409d-8412-859489e765d8]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:06 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-6a6ea4c2-5a7a-4151-9d47-e2602649997e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Remove SCSI device /dev/sdc with /sys/block/sdc/device/delete {{(pid=72351) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Aug 09 02:29:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdc/device/delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdc/device/delete" returned: 0 in 0.038s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:06 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[4265016c-907c-4fee-ad24-ef5a9446e59d]: (4, ('1', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:06 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-6a6ea4c2-5a7a-4151-9d47-e2602649997e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Checking to see if SCSI volumes sdc have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Aug 09 02:29:06 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-6a6ea4c2-5a7a-4151-9d47-e2602649997e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] SCSI volumes sdc have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Aug 09 02:29:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-6a6ea4c2-5a7a-4151-9d47-e2602649997e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950')] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 02:29:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.037s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:06 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[bb67e585-e692-4d55-beed-8b6d5b21a3b6]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-6a6ea4c2-5a7a-4151-9d47-e2602649997e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:29:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950 -p 172.16.0.220:3260 --logout {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950 -p 172.16.0.220:3260 --logout" returned: 0 in 0.065s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:06 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[5b237387-e35b-4f06-87e2-5cee708c39d8]: (4, ('Logging out of session [sid: 29, target: iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950, portal: 172.16.0.220,3260]\nLogout of [sid: 29, target: iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-6a6ea4c2-5a7a-4151-9d47-e2602649997e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 29, target: iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950, portal: 172.16.0.220,3260]
Aug 09 02:29:06 user nova-compute[72351]: Logout of [sid: 29, target: iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950, portal: 172.16.0.220,3260] successful.
Aug 09 02:29:06 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:29:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950 -p 172.16.0.220:3260 --op delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a52af854-9d8e-4ec7-935a-b7b7b65d3950 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.028s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:06 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[3f3ddc53-a643-49ce-a792-7d2d3a6d91b2]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-6a6ea4c2-5a7a-4151-9d47-e2602649997e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:29:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-6a6ea4c2-5a7a-4151-9d47-e2602649997e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.319s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:29:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-6a6ea4c2-5a7a-4151-9d47-e2602649997e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] <== disconnect_volume: return (320ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:29:06 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-6a6ea4c2-5a7a-4151-9d47-e2602649997e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 02:29:07 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:29:07 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:29:07 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:29:07 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Starting heal instance info cache {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9842}}
Aug 09 02:29:07 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Rebuilding the list of instances to heal {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9846}}
Aug 09 02:29:07 user nova-compute[72351]: DEBUG nova.objects.instance [None req-6a6ea4c2-5a7a-4151-9d47-e2602649997e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Lazy-loading 'flavor' on Instance uuid 16c1e558-baa1-4c2d-b13b-50585f74d587 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:29:07 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6a6ea4c2-5a7a-4151-9d47-e2602649997e tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Lock "16c1e558-baa1-4c2d-b13b-50585f74d587" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.807s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:29:07 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "refresh_cache-75bbbead-8628-448b-9484-19bf4ab922ff" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:29:07 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquired lock "refresh_cache-75bbbead-8628-448b-9484-19bf4ab922ff" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:29:07 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Forcefully refreshing network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1996}}
Aug 09 02:29:07 user nova-compute[72351]: DEBUG nova.objects.instance [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lazy-loading 'info_cache' on Instance uuid 75bbbead-8628-448b-9484-19bf4ab922ff {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:29:07 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Updating instance_info_cache with network_info: [{"id": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "address": "fa:16:3e:8b:b5:2e", "network": {"id": "8231f2c2-237e-41c6-8892-720b950c1453", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherB-1917380761-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.30", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fc90e94bbcb54fb6b86a9ac45edb596a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbe4ec483-85", "ovs_interfaceid": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:29:07 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Releasing lock "refresh_cache-75bbbead-8628-448b-9484-19bf4ab922ff" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:29:07 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Updated the network info_cache for instance {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9913}}
Aug 09 02:29:07 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:29:07 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:29:07 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:29:07 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:29:07 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72351) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10461}}
Aug 09 02:29:07 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:29:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:29:09 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-cd29fcf3-a7ec-4358-b81a-57395b23201d tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Acquiring lock "1b90316d-6638-4603-bb09-38bf0ed795e3" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:29:09 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-cd29fcf3-a7ec-4358-b81a-57395b23201d tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Lock "1b90316d-6638-4603-bb09-38bf0ed795e3" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:29:09 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-cd29fcf3-a7ec-4358-b81a-57395b23201d tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Acquiring lock "1b90316d-6638-4603-bb09-38bf0ed795e3-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:29:09 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-cd29fcf3-a7ec-4358-b81a-57395b23201d tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Lock "1b90316d-6638-4603-bb09-38bf0ed795e3-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:29:09 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-cd29fcf3-a7ec-4358-b81a-57395b23201d tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Lock "1b90316d-6638-4603-bb09-38bf0ed795e3-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:29:09 user nova-compute[72351]: INFO nova.compute.manager [None req-cd29fcf3-a7ec-4358-b81a-57395b23201d tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] Terminating instance
Aug 09 02:29:09 user nova-compute[72351]: DEBUG nova.compute.manager [None req-cd29fcf3-a7ec-4358-b81a-57395b23201d tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] Start destroying the instance on the hypervisor. {{(pid=72351) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3124}}
Aug 09 02:29:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:29:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:29:10 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:29:10 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:29:10 user nova-compute[72351]: DEBUG nova.compute.manager [req-d02213c4-f9de-4a73-9b6c-b23b6dbc4afe req-75ceedbd-61a1-4415-9dad-10b1190d9548 service nova] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] Received event network-vif-unplugged-fc1e01cd-012e-4aca-992a-6c6e6709f4a1 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:29:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d02213c4-f9de-4a73-9b6c-b23b6dbc4afe req-75ceedbd-61a1-4415-9dad-10b1190d9548 service nova] Acquiring lock "1b90316d-6638-4603-bb09-38bf0ed795e3-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:29:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d02213c4-f9de-4a73-9b6c-b23b6dbc4afe req-75ceedbd-61a1-4415-9dad-10b1190d9548 service nova] Lock "1b90316d-6638-4603-bb09-38bf0ed795e3-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:29:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d02213c4-f9de-4a73-9b6c-b23b6dbc4afe req-75ceedbd-61a1-4415-9dad-10b1190d9548 service nova] Lock "1b90316d-6638-4603-bb09-38bf0ed795e3-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:29:10 user nova-compute[72351]: DEBUG nova.compute.manager [req-d02213c4-f9de-4a73-9b6c-b23b6dbc4afe req-75ceedbd-61a1-4415-9dad-10b1190d9548 service nova] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] No waiting events found dispatching network-vif-unplugged-fc1e01cd-012e-4aca-992a-6c6e6709f4a1 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:29:10 user nova-compute[72351]: DEBUG nova.compute.manager [req-d02213c4-f9de-4a73-9b6c-b23b6dbc4afe req-75ceedbd-61a1-4415-9dad-10b1190d9548 service nova] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] Received event network-vif-unplugged-fc1e01cd-012e-4aca-992a-6c6e6709f4a1 for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:29:10 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] Instance destroyed successfully.
Aug 09 02:29:10 user nova-compute[72351]: DEBUG nova.objects.instance [None req-cd29fcf3-a7ec-4358-b81a-57395b23201d tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Lazy-loading 'resources' on Instance uuid 1b90316d-6638-4603-bb09-38bf0ed795e3 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:29:10 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-cd29fcf3-a7ec-4358-b81a-57395b23201d tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:27:19Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServersNegativeTestJSON-server-132169414',display_name='tempest-ServersNegativeTestJSON-server-132169414',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serversnegativetestjson-server-132169414',id=29,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:27:26Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='eebc3a715ae04d48918b6c68b135935b',ramdisk_id='',reservation_id='r-1y0yt363',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServersNegativeTestJSON-1773070288',owner_user_name='tempest-ServersNegativeTestJSON-1773070288-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:27:26Z,user_data=None,user_id='ebc62bdcafdd4131be9abb7fe4ed2422',uuid=1b90316d-6638-4603-bb09-38bf0ed795e3,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "fc1e01cd-012e-4aca-992a-6c6e6709f4a1", "address": "fa:16:3e:69:d6:e1", "network": {"id": "9785f84c-cf45-464e-9127-332ce46abd11", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-87568186-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "eebc3a715ae04d48918b6c68b135935b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfc1e01cd-01", "ovs_interfaceid": "fc1e01cd-012e-4aca-992a-6c6e6709f4a1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:29:10 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-cd29fcf3-a7ec-4358-b81a-57395b23201d tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Converting VIF {"id": "fc1e01cd-012e-4aca-992a-6c6e6709f4a1", "address": "fa:16:3e:69:d6:e1", "network": {"id": "9785f84c-cf45-464e-9127-332ce46abd11", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-87568186-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "eebc3a715ae04d48918b6c68b135935b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfc1e01cd-01", "ovs_interfaceid": "fc1e01cd-012e-4aca-992a-6c6e6709f4a1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:29:10 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-cd29fcf3-a7ec-4358-b81a-57395b23201d tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:69:d6:e1,bridge_name='br-int',has_traffic_filtering=True,id=fc1e01cd-012e-4aca-992a-6c6e6709f4a1,network=Network(9785f84c-cf45-464e-9127-332ce46abd11),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfc1e01cd-01') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:29:10 user nova-compute[72351]: DEBUG os_vif [None req-cd29fcf3-a7ec-4358-b81a-57395b23201d tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:69:d6:e1,bridge_name='br-int',has_traffic_filtering=True,id=fc1e01cd-012e-4aca-992a-6c6e6709f4a1,network=Network(9785f84c-cf45-464e-9127-332ce46abd11),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfc1e01cd-01') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:29:10 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:29:10 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapfc1e01cd-01, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:29:10 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:29:10 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:29:10 user nova-compute[72351]: INFO os_vif [None req-cd29fcf3-a7ec-4358-b81a-57395b23201d tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:69:d6:e1,bridge_name='br-int',has_traffic_filtering=True,id=fc1e01cd-012e-4aca-992a-6c6e6709f4a1,network=Network(9785f84c-cf45-464e-9127-332ce46abd11),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfc1e01cd-01')
Aug 09 02:29:10 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-cd29fcf3-a7ec-4358-b81a-57395b23201d tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] Deleting instance files /opt/stack/data/nova/instances/1b90316d-6638-4603-bb09-38bf0ed795e3_del
Aug 09 02:29:10 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-cd29fcf3-a7ec-4358-b81a-57395b23201d tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] Deletion of /opt/stack/data/nova/instances/1b90316d-6638-4603-bb09-38bf0ed795e3_del complete
Aug 09 02:29:10 user nova-compute[72351]: INFO nova.compute.manager [None req-cd29fcf3-a7ec-4358-b81a-57395b23201d tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] Took 0.72 seconds to destroy the instance on the hypervisor.
Aug 09 02:29:10 user nova-compute[72351]: DEBUG oslo.service.loopingcall [None req-cd29fcf3-a7ec-4358-b81a-57395b23201d tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72351) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Aug 09 02:29:10 user nova-compute[72351]: DEBUG nova.compute.manager [-] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] Deallocating network for instance {{(pid=72351) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2263}}
Aug 09 02:29:10 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] deallocate_for_instance() {{(pid=72351) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1795}}
Aug 09 02:29:11 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:29:11 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] Took 0.55 seconds to deallocate network for instance.
Aug 09 02:29:11 user nova-compute[72351]: DEBUG nova.compute.manager [req-e61f4a5c-ede7-4678-98dd-a48ddae76e2b req-442b5b21-0100-4e75-9327-4321d9e24ce7 service nova] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] Received event network-vif-deleted-fc1e01cd-012e-4aca-992a-6c6e6709f4a1 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:29:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-cd29fcf3-a7ec-4358-b81a-57395b23201d tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:29:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-cd29fcf3-a7ec-4358-b81a-57395b23201d tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:29:11 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-cd29fcf3-a7ec-4358-b81a-57395b23201d tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:29:11 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-cd29fcf3-a7ec-4358-b81a-57395b23201d tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:29:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-cd29fcf3-a7ec-4358-b81a-57395b23201d tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.418s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:29:11 user nova-compute[72351]: INFO nova.scheduler.client.report [None req-cd29fcf3-a7ec-4358-b81a-57395b23201d tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Deleted allocations for instance 1b90316d-6638-4603-bb09-38bf0ed795e3
Aug 09 02:29:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-cd29fcf3-a7ec-4358-b81a-57395b23201d tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Lock "1b90316d-6638-4603-bb09-38bf0ed795e3" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 1.880s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:29:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-8d3d8900-6181-4de4-b361-f5546bd396b1 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Acquiring lock "585c81a5-ac24-4d12-a8a6-1e7b12a2d53a" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:29:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-8d3d8900-6181-4de4-b361-f5546bd396b1 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "585c81a5-ac24-4d12-a8a6-1e7b12a2d53a" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:29:11 user nova-compute[72351]: INFO nova.compute.manager [None req-8d3d8900-6181-4de4-b361-f5546bd396b1 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Detaching volume 40cdcbc5-17b6-490e-8989-3d6ab80bdd80
Aug 09 02:29:12 user nova-compute[72351]: INFO nova.virt.block_device [None req-8d3d8900-6181-4de4-b361-f5546bd396b1 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Attempting to driver detach volume 40cdcbc5-17b6-490e-8989-3d6ab80bdd80 from mountpoint /dev/vdb
Aug 09 02:29:12 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-8d3d8900-6181-4de4-b361-f5546bd396b1 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Attempting to detach device vdb from instance 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a from the persistent domain config. {{(pid=72351) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2477}}
Aug 09 02:29:12 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-8d3d8900-6181-4de4-b361-f5546bd396b1 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 02:29:12 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:29:12 user nova-compute[72351]:   <source dev="/dev/sdf"/>
Aug 09 02:29:12 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:29:12 user nova-compute[72351]:   <serial>40cdcbc5-17b6-490e-8989-3d6ab80bdd80</serial>
Aug 09 02:29:12 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x05" function="0x0"/>
Aug 09 02:29:12 user nova-compute[72351]: </disk>
Aug 09 02:29:12 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 02:29:12 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-8d3d8900-6181-4de4-b361-f5546bd396b1 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Successfully detached device vdb from instance 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a from the persistent domain config.
Aug 09 02:29:12 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-8d3d8900-6181-4de4-b361-f5546bd396b1 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a from the live domain config. {{(pid=72351) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2513}}
Aug 09 02:29:12 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-8d3d8900-6181-4de4-b361-f5546bd396b1 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 02:29:12 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:29:12 user nova-compute[72351]:   <source dev="/dev/sdf"/>
Aug 09 02:29:12 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:29:12 user nova-compute[72351]:   <serial>40cdcbc5-17b6-490e-8989-3d6ab80bdd80</serial>
Aug 09 02:29:12 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x05" function="0x0"/>
Aug 09 02:29:12 user nova-compute[72351]: </disk>
Aug 09 02:29:12 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 02:29:12 user nova-compute[72351]: DEBUG nova.compute.manager [req-48107774-e0d3-47b0-9bae-d3f878135321 req-f0cf4a93-3bfc-4cc4-97b1-6856389b6608 service nova] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] Received event network-vif-plugged-fc1e01cd-012e-4aca-992a-6c6e6709f4a1 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:29:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-48107774-e0d3-47b0-9bae-d3f878135321 req-f0cf4a93-3bfc-4cc4-97b1-6856389b6608 service nova] Acquiring lock "1b90316d-6638-4603-bb09-38bf0ed795e3-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:29:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-48107774-e0d3-47b0-9bae-d3f878135321 req-f0cf4a93-3bfc-4cc4-97b1-6856389b6608 service nova] Lock "1b90316d-6638-4603-bb09-38bf0ed795e3-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:29:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-48107774-e0d3-47b0-9bae-d3f878135321 req-f0cf4a93-3bfc-4cc4-97b1-6856389b6608 service nova] Lock "1b90316d-6638-4603-bb09-38bf0ed795e3-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:29:12 user nova-compute[72351]: DEBUG nova.compute.manager [req-48107774-e0d3-47b0-9bae-d3f878135321 req-f0cf4a93-3bfc-4cc4-97b1-6856389b6608 service nova] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] No waiting events found dispatching network-vif-plugged-fc1e01cd-012e-4aca-992a-6c6e6709f4a1 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:29:12 user nova-compute[72351]: WARNING nova.compute.manager [req-48107774-e0d3-47b0-9bae-d3f878135321 req-f0cf4a93-3bfc-4cc4-97b1-6856389b6608 service nova] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] Received unexpected event network-vif-plugged-fc1e01cd-012e-4aca-992a-6c6e6709f4a1 for instance with vm_state deleted and task_state None.
Aug 09 02:29:12 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Received event <DeviceRemovedEvent: 1691548152.502823, 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=72351) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2360}}
Aug 09 02:29:12 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-8d3d8900-6181-4de4-b361-f5546bd396b1 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a {{(pid=72351) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2589}}
Aug 09 02:29:12 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-8d3d8900-6181-4de4-b361-f5546bd396b1 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Successfully detached device vdb from instance 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a from the live domain config.
Aug 09 02:29:12 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-8d3d8900-6181-4de4-b361-f5546bd396b1 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 02:29:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-8d3d8900-6181-4de4-b361-f5546bd396b1 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'dEclGUPX', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdf'}, None), 'kwargs': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:29:12 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-8d3d8900-6181-4de4-b361-f5546bd396b1 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:29:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-8d3d8900-6181-4de4-b361-f5546bd396b1 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:29:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-8d3d8900-6181-4de4-b361-f5546bd396b1 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:29:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-8d3d8900-6181-4de4-b361-f5546bd396b1 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 02:29:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:12 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[0dcbb7cb-6ebd-4498-849e-bee92ef2c16c]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:12 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[571d65ff-9530-4118-8624-05e9e675d9a8]: (4, ('tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)\ntcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)\ntcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-8d3d8900-6181-4de4-b361-f5546bd396b1 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)
Aug 09 02:29:12 user nova-compute[72351]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:29:12 user nova-compute[72351]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:29:12 user nova-compute[72351]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:29:12 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:29:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-8d3d8900-6181-4de4-b361-f5546bd396b1 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsi session list stdout=tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 (non-flash)
Aug 09 02:29:12 user nova-compute[72351]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:29:12 user nova-compute[72351]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:29:12 user nova-compute[72351]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:29:12 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:29:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-8d3d8900-6181-4de4-b361-f5546bd396b1 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fcfdc421cf0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80'): ({'sdf'}, set())}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 02:29:12 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-8d3d8900-6181-4de4-b361-f5546bd396b1 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Removing single pathed devices sdf {{(pid=72351) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Aug 09 02:29:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:12 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[c6ca11a3-e464-46e1-809a-1cb8f200f25c]: (4, ('path checker states:\nup                  3\npending             1\n\npaths: 0\nbusy: False\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdf {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdf" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:12 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[50d32829-28e7-4052-a7b1-2dc3886f7aa2]: (4, ('ok\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:12 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-8d3d8900-6181-4de4-b361-f5546bd396b1 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Flushing IO for device /dev/sdf {{(pid=72351) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Aug 09 02:29:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdf {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdf" returned: 0 in 0.013s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:12 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[9758770a-71b0-4e32-bc9f-fd9ae7acb0f5]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:12 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-8d3d8900-6181-4de4-b361-f5546bd396b1 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Remove SCSI device /dev/sdf with /sys/block/sdf/device/delete {{(pid=72351) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Aug 09 02:29:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdf/device/delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdf/device/delete" returned: 0 in 0.053s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:12 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[fbb041dd-cc52-49c6-8afb-c867839ff1e2]: (4, ('1', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:12 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-8d3d8900-6181-4de4-b361-f5546bd396b1 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Checking to see if SCSI volumes sdf have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Aug 09 02:29:12 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-8d3d8900-6181-4de4-b361-f5546bd396b1 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] SCSI volumes sdf have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Aug 09 02:29:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-8d3d8900-6181-4de4-b361-f5546bd396b1 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80')] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 02:29:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:12 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[2069d4d9-fb6c-4572-8f55-61d9182e7f5a]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-8d3d8900-6181-4de4-b361-f5546bd396b1 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:29:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 -p 172.16.0.220:3260 --logout {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 -p 172.16.0.220:3260 --logout" returned: 0 in 0.059s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:12 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[66972f17-b177-4bc1-8883-d896c8f833bc]: (4, ('Logging out of session [sid: 25, target: iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80, portal: 172.16.0.220,3260]\nLogout of [sid: 25, target: iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-8d3d8900-6181-4de4-b361-f5546bd396b1 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 25, target: iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80, portal: 172.16.0.220,3260]
Aug 09 02:29:12 user nova-compute[72351]: Logout of [sid: 25, target: iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80, portal: 172.16.0.220,3260] successful.
Aug 09 02:29:12 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:29:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 -p 172.16.0.220:3260 --op delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:40cdcbc5-17b6-490e-8989-3d6ab80bdd80 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.024s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:12 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[4bf5d666-909a-4599-95ea-cf5ad92eb122]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-8d3d8900-6181-4de4-b361-f5546bd396b1 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:29:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-8d3d8900-6181-4de4-b361-f5546bd396b1 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.289s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:29:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-8d3d8900-6181-4de4-b361-f5546bd396b1 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] <== disconnect_volume: return (290ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:29:12 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-8d3d8900-6181-4de4-b361-f5546bd396b1 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 02:29:13 user nova-compute[72351]: DEBUG nova.objects.instance [None req-8d3d8900-6181-4de4-b361-f5546bd396b1 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lazy-loading 'flavor' on Instance uuid 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:29:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-8d3d8900-6181-4de4-b361-f5546bd396b1 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "585c81a5-ac24-4d12-a8a6-1e7b12a2d53a" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.635s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:29:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:29:15 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:29:17 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:29:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:29:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:29:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:29:23 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:29:23 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:29:25 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691548150.3508565, 1b90316d-6638-4603-bb09-38bf0ed795e3 => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:29:25 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] VM Stopped (Lifecycle Event)
Aug 09 02:29:25 user nova-compute[72351]: DEBUG nova.compute.manager [None req-46b01840-4c30-490d-b1f6-4dd96b0d4dc5 None None] [instance: 1b90316d-6638-4603-bb09-38bf0ed795e3] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:29:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:29:28 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:29:29 user nova-compute[72351]: DEBUG nova.compute.manager [req-b88540a3-a343-4384-abcf-ba06ed7ac420 req-4a108230-51c8-493e-8f29-f58d83e4bf2f service nova] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Received event network-changed-e58740da-bc24-4309-8e35-48fc70bece21 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:29:29 user nova-compute[72351]: DEBUG nova.compute.manager [req-b88540a3-a343-4384-abcf-ba06ed7ac420 req-4a108230-51c8-493e-8f29-f58d83e4bf2f service nova] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Refreshing instance network info cache due to event network-changed-e58740da-bc24-4309-8e35-48fc70bece21. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:29:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-b88540a3-a343-4384-abcf-ba06ed7ac420 req-4a108230-51c8-493e-8f29-f58d83e4bf2f service nova] Acquiring lock "refresh_cache-a03fbded-23ad-440b-97e7-bea07425e873" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:29:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-b88540a3-a343-4384-abcf-ba06ed7ac420 req-4a108230-51c8-493e-8f29-f58d83e4bf2f service nova] Acquired lock "refresh_cache-a03fbded-23ad-440b-97e7-bea07425e873" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:29:29 user nova-compute[72351]: DEBUG nova.network.neutron [req-b88540a3-a343-4384-abcf-ba06ed7ac420 req-4a108230-51c8-493e-8f29-f58d83e4bf2f service nova] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Refreshing network info cache for port e58740da-bc24-4309-8e35-48fc70bece21 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:29:30 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6be2a94a-0d46-4a64-b5c1-7815853bcfe6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Acquiring lock "831c864a-75d9-42b3-b952-749f4201e40e" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:29:30 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6be2a94a-0d46-4a64-b5c1-7815853bcfe6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "831c864a-75d9-42b3-b952-749f4201e40e" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:29:30 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6be2a94a-0d46-4a64-b5c1-7815853bcfe6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Acquiring lock "831c864a-75d9-42b3-b952-749f4201e40e-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:29:30 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6be2a94a-0d46-4a64-b5c1-7815853bcfe6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "831c864a-75d9-42b3-b952-749f4201e40e-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:29:30 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6be2a94a-0d46-4a64-b5c1-7815853bcfe6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "831c864a-75d9-42b3-b952-749f4201e40e-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:29:30 user nova-compute[72351]: INFO nova.compute.manager [None req-6be2a94a-0d46-4a64-b5c1-7815853bcfe6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Terminating instance
Aug 09 02:29:30 user nova-compute[72351]: DEBUG nova.compute.manager [None req-6be2a94a-0d46-4a64-b5c1-7815853bcfe6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Start destroying the instance on the hypervisor. {{(pid=72351) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3124}}
Aug 09 02:29:30 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-4a33640f-2faa-4a61-9565-7ff024fe8b19 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Acquiring lock "a03fbded-23ad-440b-97e7-bea07425e873" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:29:30 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-4a33640f-2faa-4a61-9565-7ff024fe8b19 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lock "a03fbded-23ad-440b-97e7-bea07425e873" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:29:30 user nova-compute[72351]: DEBUG nova.objects.instance [None req-4a33640f-2faa-4a61-9565-7ff024fe8b19 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lazy-loading 'flavor' on Instance uuid a03fbded-23ad-440b-97e7-bea07425e873 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:29:30 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:29:30 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:29:30 user nova-compute[72351]: DEBUG nova.network.neutron [req-b88540a3-a343-4384-abcf-ba06ed7ac420 req-4a108230-51c8-493e-8f29-f58d83e4bf2f service nova] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Updated VIF entry in instance network info cache for port e58740da-bc24-4309-8e35-48fc70bece21. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:29:30 user nova-compute[72351]: DEBUG nova.network.neutron [req-b88540a3-a343-4384-abcf-ba06ed7ac420 req-4a108230-51c8-493e-8f29-f58d83e4bf2f service nova] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Updating instance_info_cache with network_info: [{"id": "e58740da-bc24-4309-8e35-48fc70bece21", "address": "fa:16:3e:f3:4e:5e", "network": {"id": "49bfc70e-93c7-4607-9507-4711bda1a686", "bridge": "br-int", "label": "tempest-TestStampPattern-1656229455-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.74", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fc12f9037cd54bb6a2fdff3071132dd4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape58740da-bc", "ovs_interfaceid": "e58740da-bc24-4309-8e35-48fc70bece21", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:29:30 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-4a33640f-2faa-4a61-9565-7ff024fe8b19 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lock "a03fbded-23ad-440b-97e7-bea07425e873" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.073s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:29:30 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-b88540a3-a343-4384-abcf-ba06ed7ac420 req-4a108230-51c8-493e-8f29-f58d83e4bf2f service nova] Releasing lock "refresh_cache-a03fbded-23ad-440b-97e7-bea07425e873" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:29:30 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:29:30 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:29:30 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:29:30 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-4a33640f-2faa-4a61-9565-7ff024fe8b19 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Acquiring lock "a03fbded-23ad-440b-97e7-bea07425e873" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:29:30 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-4a33640f-2faa-4a61-9565-7ff024fe8b19 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lock "a03fbded-23ad-440b-97e7-bea07425e873" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:29:30 user nova-compute[72351]: INFO nova.compute.manager [None req-4a33640f-2faa-4a61-9565-7ff024fe8b19 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Attaching volume 87e3b4cc-f41c-4e23-bc8a-5dc14a5788b6 to /dev/vdb
Aug 09 02:29:30 user nova-compute[72351]: DEBUG os_brick.utils [None req-4a33640f-2faa-4a61-9565-7ff024fe8b19 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:29:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.016s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:30 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[fc58db2e-6842-47e1-acd8-e8ec14fce91d]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:a52269602ff4\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:30 user nova-compute[72351]: DEBUG nova.compute.manager [req-d276be20-bd05-4b8a-84eb-162d79bb7b57 req-775f8db2-38b5-4a9f-920e-8fe6082c046e service nova] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Received event network-vif-unplugged-02c9ff8a-e779-443c-9b3b-567d2e69cea0 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:29:30 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d276be20-bd05-4b8a-84eb-162d79bb7b57 req-775f8db2-38b5-4a9f-920e-8fe6082c046e service nova] Acquiring lock "831c864a-75d9-42b3-b952-749f4201e40e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:29:30 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d276be20-bd05-4b8a-84eb-162d79bb7b57 req-775f8db2-38b5-4a9f-920e-8fe6082c046e service nova] Lock "831c864a-75d9-42b3-b952-749f4201e40e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:29:30 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d276be20-bd05-4b8a-84eb-162d79bb7b57 req-775f8db2-38b5-4a9f-920e-8fe6082c046e service nova] Lock "831c864a-75d9-42b3-b952-749f4201e40e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:29:30 user nova-compute[72351]: DEBUG nova.compute.manager [req-d276be20-bd05-4b8a-84eb-162d79bb7b57 req-775f8db2-38b5-4a9f-920e-8fe6082c046e service nova] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] No waiting events found dispatching network-vif-unplugged-02c9ff8a-e779-443c-9b3b-567d2e69cea0 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:29:30 user nova-compute[72351]: DEBUG nova.compute.manager [req-d276be20-bd05-4b8a-84eb-162d79bb7b57 req-775f8db2-38b5-4a9f-920e-8fe6082c046e service nova] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Received event network-vif-unplugged-02c9ff8a-e779-443c-9b3b-567d2e69cea0 for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:29:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:30 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[427b8d2b-df01-4491-9ae0-6042a3c27fce]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.016s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:30 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[1706c4f9-c607-4b7e-8c44-a0f53e5ffc35]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:30 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[8e048deb-2dc0-479a-a858-7746187e70cd]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-4a33640f-2faa-4a61-9565-7ff024fe8b19 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Running cmd (subprocess): nvme version {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-4a33640f-2faa-4a61-9565-7ff024fe8b19 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] 'nvme version' failed. Not Retrying. {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:29:30 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.nvmeof [None req-4a33640f-2faa-4a61-9565-7ff024fe8b19 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] nvme not present on system {{(pid=72351) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Aug 09 02:29:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:29:30 user nova-compute[72351]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Aug 09 02:29:30 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[afbc39ca-0d48-465f-ae28-4ab7df00116a]: (4, '') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:30 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-4a33640f-2faa-4a61-9565-7ff024fe8b19 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72351) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Aug 09 02:29:30 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-4a33640f-2faa-4a61-9565-7ff024fe8b19 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Aug 09 02:29:30 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-4a33640f-2faa-4a61-9565-7ff024fe8b19 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] LIGHTOS: no hostnqn found. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Aug 09 02:29:30 user nova-compute[72351]: DEBUG os_brick.utils [None req-4a33640f-2faa-4a61-9565-7ff024fe8b19 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] <== get_connector_properties: return (131ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:a52269602ff4', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:29:30 user nova-compute[72351]: DEBUG nova.virt.block_device [None req-4a33640f-2faa-4a61-9565-7ff024fe8b19 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Updating existing volume attachment record: b3862184-1012-43ff-a99f-39de794f9eda {{(pid=72351) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Aug 09 02:29:31 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:29:31 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:29:31 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Instance destroyed successfully.
Aug 09 02:29:31 user nova-compute[72351]: DEBUG nova.objects.instance [None req-6be2a94a-0d46-4a64-b5c1-7815853bcfe6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lazy-loading 'resources' on Instance uuid 831c864a-75d9-42b3-b952-749f4201e40e {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:29:31 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-6be2a94a-0d46-4a64-b5c1-7815853bcfe6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:24:18Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestMinimumBasicScenario-server-1449267845',display_name='tempest-TestMinimumBasicScenario-server-1449267845',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testminimumbasicscenario-server-1449267845',id=23,image_ref='16da0824-5647-4984-9a67-fc86d723ef34',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMvWYiT3PAj8t6XXpOzeSg0zsvJtWfK76LZNXd5TZSvLf67t8TWrmwgsa6xs6zEQiNUAl4jqZiCkH4fjuSHiqFnl0NsYYcCcqIbK07RAKSi5ixt23WHp3vhu37izfOEYbg==',key_name='tempest-TestMinimumBasicScenario-1710540764',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:24:29Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='c2f67da658d94fabbab82ee6a6171c4b',ramdisk_id='',reservation_id='r-c33grfp2',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='16da0824-5647-4984-9a67-fc86d723ef34',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',owner_project_name='tempest-TestMinimumBasicScenario-1380132717',owner_user_name='tempest-TestMinimumBasicScenario-1380132717-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:26:50Z,user_data=None,user_id='bec5b5704e9b4e3480f428d5c037c20d',uuid=831c864a-75d9-42b3-b952-749f4201e40e,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "02c9ff8a-e779-443c-9b3b-567d2e69cea0", "address": "fa:16:3e:32:5d:c2", "network": {"id": "7024d1f6-a400-4482-9364-78ae3b6adc80", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1603258958-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c2f67da658d94fabbab82ee6a6171c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap02c9ff8a-e7", "ovs_interfaceid": "02c9ff8a-e779-443c-9b3b-567d2e69cea0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:29:31 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-6be2a94a-0d46-4a64-b5c1-7815853bcfe6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Converting VIF {"id": "02c9ff8a-e779-443c-9b3b-567d2e69cea0", "address": "fa:16:3e:32:5d:c2", "network": {"id": "7024d1f6-a400-4482-9364-78ae3b6adc80", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1603258958-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c2f67da658d94fabbab82ee6a6171c4b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap02c9ff8a-e7", "ovs_interfaceid": "02c9ff8a-e779-443c-9b3b-567d2e69cea0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:29:31 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-6be2a94a-0d46-4a64-b5c1-7815853bcfe6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:32:5d:c2,bridge_name='br-int',has_traffic_filtering=True,id=02c9ff8a-e779-443c-9b3b-567d2e69cea0,network=Network(7024d1f6-a400-4482-9364-78ae3b6adc80),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap02c9ff8a-e7') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:29:31 user nova-compute[72351]: DEBUG os_vif [None req-6be2a94a-0d46-4a64-b5c1-7815853bcfe6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:32:5d:c2,bridge_name='br-int',has_traffic_filtering=True,id=02c9ff8a-e779-443c-9b3b-567d2e69cea0,network=Network(7024d1f6-a400-4482-9364-78ae3b6adc80),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap02c9ff8a-e7') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:29:31 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:29:31 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap02c9ff8a-e7, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:29:31 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:29:31 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:29:31 user nova-compute[72351]: INFO os_vif [None req-6be2a94a-0d46-4a64-b5c1-7815853bcfe6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:32:5d:c2,bridge_name='br-int',has_traffic_filtering=True,id=02c9ff8a-e779-443c-9b3b-567d2e69cea0,network=Network(7024d1f6-a400-4482-9364-78ae3b6adc80),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap02c9ff8a-e7')
Aug 09 02:29:31 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-6be2a94a-0d46-4a64-b5c1-7815853bcfe6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Deleting instance files /opt/stack/data/nova/instances/831c864a-75d9-42b3-b952-749f4201e40e_del
Aug 09 02:29:31 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-6be2a94a-0d46-4a64-b5c1-7815853bcfe6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Deletion of /opt/stack/data/nova/instances/831c864a-75d9-42b3-b952-749f4201e40e_del complete
Aug 09 02:29:31 user nova-compute[72351]: INFO nova.compute.manager [None req-6be2a94a-0d46-4a64-b5c1-7815853bcfe6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Took 0.87 seconds to destroy the instance on the hypervisor.
Aug 09 02:29:31 user nova-compute[72351]: DEBUG oslo.service.loopingcall [None req-6be2a94a-0d46-4a64-b5c1-7815853bcfe6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72351) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Aug 09 02:29:31 user nova-compute[72351]: DEBUG nova.compute.manager [-] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Deallocating network for instance {{(pid=72351) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2263}}
Aug 09 02:29:31 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] deallocate_for_instance() {{(pid=72351) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1795}}
Aug 09 02:29:31 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:29:31 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Took 0.55 seconds to deallocate network for instance.
Aug 09 02:29:31 user nova-compute[72351]: DEBUG nova.compute.manager [req-76be7852-c417-4c63-b7bb-f6a82d2becdd req-57e0d58e-8697-4090-8177-7f40cd3074cf service nova] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Received event network-vif-deleted-02c9ff8a-e779-443c-9b3b-567d2e69cea0 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:29:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6be2a94a-0d46-4a64-b5c1-7815853bcfe6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:29:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6be2a94a-0d46-4a64-b5c1-7815853bcfe6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:29:32 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-6be2a94a-0d46-4a64-b5c1-7815853bcfe6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:29:32 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-6be2a94a-0d46-4a64-b5c1-7815853bcfe6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:29:32 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6be2a94a-0d46-4a64-b5c1-7815853bcfe6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.383s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:29:32 user nova-compute[72351]: INFO nova.scheduler.client.report [None req-6be2a94a-0d46-4a64-b5c1-7815853bcfe6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Deleted allocations for instance 831c864a-75d9-42b3-b952-749f4201e40e
Aug 09 02:29:32 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6be2a94a-0d46-4a64-b5c1-7815853bcfe6 tempest-TestMinimumBasicScenario-1380132717 tempest-TestMinimumBasicScenario-1380132717-project-member] Lock "831c864a-75d9-42b3-b952-749f4201e40e" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.014s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:29:32 user nova-compute[72351]: DEBUG nova.compute.manager [req-1a6f00d9-adcc-4b75-a19c-2b5b4d359633 req-5f2414d2-5789-4fe1-baf8-24b922bed815 service nova] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Received event network-vif-plugged-02c9ff8a-e779-443c-9b3b-567d2e69cea0 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:29:32 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-1a6f00d9-adcc-4b75-a19c-2b5b4d359633 req-5f2414d2-5789-4fe1-baf8-24b922bed815 service nova] Acquiring lock "831c864a-75d9-42b3-b952-749f4201e40e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:29:32 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-1a6f00d9-adcc-4b75-a19c-2b5b4d359633 req-5f2414d2-5789-4fe1-baf8-24b922bed815 service nova] Lock "831c864a-75d9-42b3-b952-749f4201e40e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:29:32 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-1a6f00d9-adcc-4b75-a19c-2b5b4d359633 req-5f2414d2-5789-4fe1-baf8-24b922bed815 service nova] Lock "831c864a-75d9-42b3-b952-749f4201e40e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:29:32 user nova-compute[72351]: DEBUG nova.compute.manager [req-1a6f00d9-adcc-4b75-a19c-2b5b4d359633 req-5f2414d2-5789-4fe1-baf8-24b922bed815 service nova] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] No waiting events found dispatching network-vif-plugged-02c9ff8a-e779-443c-9b3b-567d2e69cea0 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:29:32 user nova-compute[72351]: WARNING nova.compute.manager [req-1a6f00d9-adcc-4b75-a19c-2b5b4d359633 req-5f2414d2-5789-4fe1-baf8-24b922bed815 service nova] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Received unexpected event network-vif-plugged-02c9ff8a-e779-443c-9b3b-567d2e69cea0 for instance with vm_state deleted and task_state None.
Aug 09 02:29:33 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:29:36 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:29:38 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:29:39 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-4a33640f-2faa-4a61-9565-7ff024fe8b19 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 02:29:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a33640f-2faa-4a61-9565-7ff024fe8b19 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:87e3b4cc-f41c-4e23-bc8a-5dc14a5788b6'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'qMbEwGvQ', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:29:39 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-4a33640f-2faa-4a61-9565-7ff024fe8b19 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:29:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-4a33640f-2faa-4a61-9565-7ff024fe8b19 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:29:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-4a33640f-2faa-4a61-9565-7ff024fe8b19 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:29:39 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-4a33640f-2faa-4a61-9565-7ff024fe8b19 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:29:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-4a33640f-2faa-4a61-9565-7ff024fe8b19 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:87e3b4cc-f41c-4e23-bc8a-5dc14a5788b6" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:29:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-4a33640f-2faa-4a61-9565-7ff024fe8b19 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:87e3b4cc-f41c-4e23-bc8a-5dc14a5788b6" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:29:39 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-4a33640f-2faa-4a61-9565-7ff024fe8b19 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 02:29:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:87e3b4cc-f41c-4e23-bc8a-5dc14a5788b6 -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:87e3b4cc-f41c-4e23-bc8a-5dc14a5788b6 -p 172.16.0.220:3260" returned: 21 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:39 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[72f4fb8a-a4dd-4ba2-b8b7-05caf357ef88]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a33640f-2faa-4a61-9565-7ff024fe8b19 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Aug 09 02:29:39 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:29:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:87e3b4cc-f41c-4e23-bc8a-5dc14a5788b6 -p 172.16.0.220:3260 --interface default --op new {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:87e3b4cc-f41c-4e23-bc8a-5dc14a5788b6 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:39 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[fa24b6b8-8840-41ed-8b2b-58167de696f6]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:87e3b4cc-f41c-4e23-bc8a-5dc14a5788b6] added\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a33640f-2faa-4a61-9565-7ff024fe8b19 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:87e3b4cc-f41c-4e23-bc8a-5dc14a5788b6] added
Aug 09 02:29:39 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:29:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:87e3b4cc-f41c-4e23-bc8a-5dc14a5788b6 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:87e3b4cc-f41c-4e23-bc8a-5dc14a5788b6 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:39 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[bd54ad25-db1d-4417-8f36-1e62175473b6]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a33640f-2faa-4a61-9565-7ff024fe8b19 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:29:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:87e3b4cc-f41c-4e23-bc8a-5dc14a5788b6 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:87e3b4cc-f41c-4e23-bc8a-5dc14a5788b6 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:39 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[2156f56a-31f1-45c8-a718-8f71f8bb641f]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a33640f-2faa-4a61-9565-7ff024fe8b19 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:29:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:87e3b4cc-f41c-4e23-bc8a-5dc14a5788b6 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v qMbEwGvQ {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:87e3b4cc-f41c-4e23-bc8a-5dc14a5788b6 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v qMbEwGvQ" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:39 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e35f4c00-605d-4bfb-98b7-37fd6b0feccd]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a33640f-2faa-4a61-9565-7ff024fe8b19 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'qMbEwGvQ'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:29:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:87e3b4cc-f41c-4e23-bc8a-5dc14a5788b6 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:87e3b4cc-f41c-4e23-bc8a-5dc14a5788b6 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:39 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[62a0ae4b-ff55-4cf5-a932-a38099bf3f04]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a33640f-2faa-4a61-9565-7ff024fe8b19 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:29:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:39 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[edaf11f0-355e-4c42-823c-14ddf3248631]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)\ntcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a33640f-2faa-4a61-9565-7ff024fe8b19 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:29:39 user nova-compute[72351]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:29:39 user nova-compute[72351]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:29:39 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:29:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a33640f-2faa-4a61-9565-7ff024fe8b19 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:29:39 user nova-compute[72351]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:29:39 user nova-compute[72351]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:29:39 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:29:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:87e3b4cc-f41c-4e23-bc8a-5dc14a5788b6 -p 172.16.0.220:3260 --login {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:87e3b4cc-f41c-4e23-bc8a-5dc14a5788b6 -p 172.16.0.220:3260 --login" returned: 0 in 0.083s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:39 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[49430493-29ed-4d2b-9c25-1e2f02250d05]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:87e3b4cc-f41c-4e23-bc8a-5dc14a5788b6, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:87e3b4cc-f41c-4e23-bc8a-5dc14a5788b6, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a33640f-2faa-4a61-9565-7ff024fe8b19 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:87e3b4cc-f41c-4e23-bc8a-5dc14a5788b6, portal: 172.16.0.220,3260]
Aug 09 02:29:39 user nova-compute[72351]: Login to [iface: default, target: iqn.2010-10.org.openstack:87e3b4cc-f41c-4e23-bc8a-5dc14a5788b6, portal: 172.16.0.220,3260] successful.
Aug 09 02:29:39 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:29:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:87e3b4cc-f41c-4e23-bc8a-5dc14a5788b6 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:87e3b4cc-f41c-4e23-bc8a-5dc14a5788b6 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:39 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[81fc06ff-71d7-4ef8-afa8-210bbeb7ac48]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a33640f-2faa-4a61-9565-7ff024fe8b19 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:29:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:39 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[869788a6-1a25-40c6-b2c7-b8cba09c843c]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)\ntcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)\ntcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:87e3b4cc-f41c-4e23-bc8a-5dc14a5788b6 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a33640f-2faa-4a61-9565-7ff024fe8b19 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:29:39 user nova-compute[72351]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:29:39 user nova-compute[72351]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:29:39 user nova-compute[72351]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:87e3b4cc-f41c-4e23-bc8a-5dc14a5788b6 (non-flash)
Aug 09 02:29:39 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:29:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a33640f-2faa-4a61-9565-7ff024fe8b19 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:29:39 user nova-compute[72351]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:29:39 user nova-compute[72351]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:29:39 user nova-compute[72351]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:87e3b4cc-f41c-4e23-bc8a-5dc14a5788b6 (non-flash)
Aug 09 02:29:39 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:29:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-4a33640f-2faa-4a61-9565-7ff024fe8b19 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:87e3b4cc-f41c-4e23-bc8a-5dc14a5788b6" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.310s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:29:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a33640f-2faa-4a61-9565-7ff024fe8b19 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 02:29:39 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-4a33640f-2faa-4a61-9565-7ff024fe8b19 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] HCTL ('34', '-', '-', 0) found on session 32 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 02:29:39 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-4a33640f-2faa-4a61-9565-7ff024fe8b19 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Scanning host 34 c: -, t: -, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 02:29:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.026s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:39 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[225b5aa5-a6ba-4640-8365-21fce5a020d6]: (4, ('- - 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:39 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-4a33640f-2faa-4a61-9565-7ff024fe8b19 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Searching for a device in session 32 and hctl ['34', '*', '*', 0] yield: sdc {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:29:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a33640f-2faa-4a61-9565-7ff024fe8b19 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Connected to sdc using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'qMbEwGvQ', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 02:29:40 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-4a33640f-2faa-4a61-9565-7ff024fe8b19 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.349s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:29:40 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a33640f-2faa-4a61-9565-7ff024fe8b19 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] <== connect_volume: return (1350ms) {'type': 'block', 'scsi_wwn': '23562313263653331', 'path': '/dev/sdc'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:29:40 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-4a33640f-2faa-4a61-9565-7ff024fe8b19 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23562313263653331', 'path': '/dev/sdc'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 02:29:40 user nova-compute[72351]: DEBUG nova.objects.instance [None req-4a33640f-2faa-4a61-9565-7ff024fe8b19 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lazy-loading 'flavor' on Instance uuid a03fbded-23ad-440b-97e7-bea07425e873 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:29:40 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-4a33640f-2faa-4a61-9565-7ff024fe8b19 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] attach device xml: <disk type="block" device="disk">
Aug 09 02:29:40 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:29:40 user nova-compute[72351]:   <source dev="/dev/sdc"/>
Aug 09 02:29:40 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:29:40 user nova-compute[72351]:   <serial>87e3b4cc-f41c-4e23-bc8a-5dc14a5788b6</serial>
Aug 09 02:29:40 user nova-compute[72351]: </disk>
Aug 09 02:29:40 user nova-compute[72351]:  {{(pid=72351) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Aug 09 02:29:41 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:29:41 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-4a33640f-2faa-4a61-9565-7ff024fe8b19 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:29:41 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-4a33640f-2faa-4a61-9565-7ff024fe8b19 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:29:41 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-4a33640f-2faa-4a61-9565-7ff024fe8b19 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] No VIF found with MAC fa:16:3e:f3:4e:5e, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:29:41 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-4a33640f-2faa-4a61-9565-7ff024fe8b19 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lock "a03fbded-23ad-440b-97e7-bea07425e873" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 10.704s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:29:43 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:29:43 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-4f4f46f4-d71c-4cca-9521-ebb210160d3b tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Acquiring lock "a03fbded-23ad-440b-97e7-bea07425e873" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:29:43 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-4f4f46f4-d71c-4cca-9521-ebb210160d3b tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lock "a03fbded-23ad-440b-97e7-bea07425e873" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:29:43 user nova-compute[72351]: INFO nova.compute.manager [None req-4f4f46f4-d71c-4cca-9521-ebb210160d3b tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Detaching volume 87e3b4cc-f41c-4e23-bc8a-5dc14a5788b6
Aug 09 02:29:43 user nova-compute[72351]: INFO nova.virt.block_device [None req-4f4f46f4-d71c-4cca-9521-ebb210160d3b tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Attempting to driver detach volume 87e3b4cc-f41c-4e23-bc8a-5dc14a5788b6 from mountpoint /dev/vdb
Aug 09 02:29:43 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-4f4f46f4-d71c-4cca-9521-ebb210160d3b tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Attempting to detach device vdb from instance a03fbded-23ad-440b-97e7-bea07425e873 from the persistent domain config. {{(pid=72351) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2477}}
Aug 09 02:29:43 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-4f4f46f4-d71c-4cca-9521-ebb210160d3b tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 02:29:43 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:29:43 user nova-compute[72351]:   <source dev="/dev/sdc"/>
Aug 09 02:29:43 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:29:43 user nova-compute[72351]:   <serial>87e3b4cc-f41c-4e23-bc8a-5dc14a5788b6</serial>
Aug 09 02:29:43 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Aug 09 02:29:43 user nova-compute[72351]: </disk>
Aug 09 02:29:43 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 02:29:43 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-4f4f46f4-d71c-4cca-9521-ebb210160d3b tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Successfully detached device vdb from instance a03fbded-23ad-440b-97e7-bea07425e873 from the persistent domain config.
Aug 09 02:29:43 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-4f4f46f4-d71c-4cca-9521-ebb210160d3b tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance a03fbded-23ad-440b-97e7-bea07425e873 from the live domain config. {{(pid=72351) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2513}}
Aug 09 02:29:43 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-4f4f46f4-d71c-4cca-9521-ebb210160d3b tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 02:29:43 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:29:43 user nova-compute[72351]:   <source dev="/dev/sdc"/>
Aug 09 02:29:43 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:29:43 user nova-compute[72351]:   <serial>87e3b4cc-f41c-4e23-bc8a-5dc14a5788b6</serial>
Aug 09 02:29:43 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Aug 09 02:29:43 user nova-compute[72351]: </disk>
Aug 09 02:29:43 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 02:29:43 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Received event <DeviceRemovedEvent: 1691548183.6877365, a03fbded-23ad-440b-97e7-bea07425e873 => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=72351) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2360}}
Aug 09 02:29:43 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-4f4f46f4-d71c-4cca-9521-ebb210160d3b tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance a03fbded-23ad-440b-97e7-bea07425e873 {{(pid=72351) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2589}}
Aug 09 02:29:43 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-4f4f46f4-d71c-4cca-9521-ebb210160d3b tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Successfully detached device vdb from instance a03fbded-23ad-440b-97e7-bea07425e873 from the live domain config.
Aug 09 02:29:43 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-4f4f46f4-d71c-4cca-9521-ebb210160d3b tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: a03fbded-23ad-440b-97e7-bea07425e873] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 02:29:43 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f4f46f4-d71c-4cca-9521-ebb210160d3b tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:87e3b4cc-f41c-4e23-bc8a-5dc14a5788b6'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'qMbEwGvQ', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc'}, None), 'kwargs': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:29:43 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-4f4f46f4-d71c-4cca-9521-ebb210160d3b tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:29:43 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-4f4f46f4-d71c-4cca-9521-ebb210160d3b tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:29:43 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-4f4f46f4-d71c-4cca-9521-ebb210160d3b tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:29:43 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f4f46f4-d71c-4cca-9521-ebb210160d3b tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:87e3b4cc-f41c-4e23-bc8a-5dc14a5788b6', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 02:29:43 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:43 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:43 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[c5d35bdd-6837-45b8-aad9-405edca8a758]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:87e3b4cc-f41c-4e23-bc8a-5dc14a5788b6\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:43 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:43 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:43 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[6a706824-5e2b-4336-87c2-a5643a72bb4e]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)\ntcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)\ntcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:87e3b4cc-f41c-4e23-bc8a-5dc14a5788b6 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:43 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f4f46f4-d71c-4cca-9521-ebb210160d3b tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:29:43 user nova-compute[72351]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:29:43 user nova-compute[72351]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:29:43 user nova-compute[72351]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:87e3b4cc-f41c-4e23-bc8a-5dc14a5788b6 (non-flash)
Aug 09 02:29:43 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:29:43 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f4f46f4-d71c-4cca-9521-ebb210160d3b tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:29:43 user nova-compute[72351]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:29:43 user nova-compute[72351]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:29:43 user nova-compute[72351]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:87e3b4cc-f41c-4e23-bc8a-5dc14a5788b6 (non-flash)
Aug 09 02:29:43 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:29:43 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f4f46f4-d71c-4cca-9521-ebb210160d3b tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fd0001784c0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:87e3b4cc-f41c-4e23-bc8a-5dc14a5788b6'): ({'sdc'}, set())}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 02:29:43 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-4f4f46f4-d71c-4cca-9521-ebb210160d3b tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Removing single pathed devices sdc {{(pid=72351) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Aug 09 02:29:43 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:43 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:43 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[dcfe2dd8-94c6-4c32-a9e3-1976543b064c]: (4, ('path checker states:\nup                  3\npending             1\n\npaths: 0\nbusy: False\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:43 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdc {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:43 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdc" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:43 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[242d8668-5601-4607-b540-f192febbe0f9]: (4, ('ok\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:43 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-4f4f46f4-d71c-4cca-9521-ebb210160d3b tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Flushing IO for device /dev/sdc {{(pid=72351) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Aug 09 02:29:43 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdc {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:43 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdc" returned: 0 in 0.015s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:43 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[7679fa77-42ac-4f55-8b40-3cb1ee83d571]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:43 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-4f4f46f4-d71c-4cca-9521-ebb210160d3b tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Remove SCSI device /dev/sdc with /sys/block/sdc/device/delete {{(pid=72351) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Aug 09 02:29:43 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdc/device/delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:43 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdc/device/delete" returned: 0 in 0.032s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:43 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[2f6741a4-7986-434f-ad14-cf17204aff87]: (4, ('1', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:43 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-4f4f46f4-d71c-4cca-9521-ebb210160d3b tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Checking to see if SCSI volumes sdc have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Aug 09 02:29:43 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-4f4f46f4-d71c-4cca-9521-ebb210160d3b tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] SCSI volumes sdc have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Aug 09 02:29:43 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f4f46f4-d71c-4cca-9521-ebb210160d3b tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:87e3b4cc-f41c-4e23-bc8a-5dc14a5788b6')] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 02:29:43 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:87e3b4cc-f41c-4e23-bc8a-5dc14a5788b6 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:43 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:87e3b4cc-f41c-4e23-bc8a-5dc14a5788b6 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:43 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[10db77db-7552-40f1-8265-9455ca56c864]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:43 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f4f46f4-d71c-4cca-9521-ebb210160d3b tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:29:43 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:87e3b4cc-f41c-4e23-bc8a-5dc14a5788b6 -p 172.16.0.220:3260 --logout {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:43 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:87e3b4cc-f41c-4e23-bc8a-5dc14a5788b6 -p 172.16.0.220:3260 --logout" returned: 0 in 0.060s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:43 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[66a6dc84-475b-4c0c-97bf-049039beeab8]: (4, ('Logging out of session [sid: 32, target: iqn.2010-10.org.openstack:87e3b4cc-f41c-4e23-bc8a-5dc14a5788b6, portal: 172.16.0.220,3260]\nLogout of [sid: 32, target: iqn.2010-10.org.openstack:87e3b4cc-f41c-4e23-bc8a-5dc14a5788b6, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:43 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f4f46f4-d71c-4cca-9521-ebb210160d3b tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 32, target: iqn.2010-10.org.openstack:87e3b4cc-f41c-4e23-bc8a-5dc14a5788b6, portal: 172.16.0.220,3260]
Aug 09 02:29:43 user nova-compute[72351]: Logout of [sid: 32, target: iqn.2010-10.org.openstack:87e3b4cc-f41c-4e23-bc8a-5dc14a5788b6, portal: 172.16.0.220,3260] successful.
Aug 09 02:29:43 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:29:43 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:87e3b4cc-f41c-4e23-bc8a-5dc14a5788b6 -p 172.16.0.220:3260 --op delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:44 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:87e3b4cc-f41c-4e23-bc8a-5dc14a5788b6 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.027s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:44 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[2cfa3f64-d2ba-426c-91f4-2edff9399566]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:44 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f4f46f4-d71c-4cca-9521-ebb210160d3b tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:29:44 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-4f4f46f4-d71c-4cca-9521-ebb210160d3b tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.298s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:29:44 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f4f46f4-d71c-4cca-9521-ebb210160d3b tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] <== disconnect_volume: return (299ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:29:44 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-4f4f46f4-d71c-4cca-9521-ebb210160d3b tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 02:29:44 user nova-compute[72351]: DEBUG nova.objects.instance [None req-4f4f46f4-d71c-4cca-9521-ebb210160d3b tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lazy-loading 'flavor' on Instance uuid a03fbded-23ad-440b-97e7-bea07425e873 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:29:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-4f4f46f4-d71c-4cca-9521-ebb210160d3b tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lock "a03fbded-23ad-440b-97e7-bea07425e873" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.625s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:29:44 user nova-compute[72351]: DEBUG nova.compute.manager [req-fbcb9773-9234-48f6-a8fe-2d10615b468e req-3b791ffa-8107-4b39-ac72-549b77b33729 service nova] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Received event network-changed-4d36d764-e86b-40ee-81ed-a9f3adfadc48 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:29:44 user nova-compute[72351]: DEBUG nova.compute.manager [req-fbcb9773-9234-48f6-a8fe-2d10615b468e req-3b791ffa-8107-4b39-ac72-549b77b33729 service nova] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Refreshing instance network info cache due to event network-changed-4d36d764-e86b-40ee-81ed-a9f3adfadc48. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:29:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-fbcb9773-9234-48f6-a8fe-2d10615b468e req-3b791ffa-8107-4b39-ac72-549b77b33729 service nova] Acquiring lock "refresh_cache-2a3bcdea-be36-4a5b-93cb-53396efdfa8e" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:29:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-fbcb9773-9234-48f6-a8fe-2d10615b468e req-3b791ffa-8107-4b39-ac72-549b77b33729 service nova] Acquired lock "refresh_cache-2a3bcdea-be36-4a5b-93cb-53396efdfa8e" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:29:44 user nova-compute[72351]: DEBUG nova.network.neutron [req-fbcb9773-9234-48f6-a8fe-2d10615b468e req-3b791ffa-8107-4b39-ac72-549b77b33729 service nova] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Refreshing network info cache for port 4d36d764-e86b-40ee-81ed-a9f3adfadc48 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:29:45 user nova-compute[72351]: DEBUG nova.network.neutron [req-fbcb9773-9234-48f6-a8fe-2d10615b468e req-3b791ffa-8107-4b39-ac72-549b77b33729 service nova] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Updated VIF entry in instance network info cache for port 4d36d764-e86b-40ee-81ed-a9f3adfadc48. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:29:45 user nova-compute[72351]: DEBUG nova.network.neutron [req-fbcb9773-9234-48f6-a8fe-2d10615b468e req-3b791ffa-8107-4b39-ac72-549b77b33729 service nova] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Updating instance_info_cache with network_info: [{"id": "4d36d764-e86b-40ee-81ed-a9f3adfadc48", "address": "fa:16:3e:28:03:d9", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.162", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4d36d764-e8", "ovs_interfaceid": "4d36d764-e86b-40ee-81ed-a9f3adfadc48", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:29:45 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5c4d21a3-c6a5-4555-918a-9267db1d0851 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquiring lock "2a3bcdea-be36-4a5b-93cb-53396efdfa8e" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:29:45 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5c4d21a3-c6a5-4555-918a-9267db1d0851 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "2a3bcdea-be36-4a5b-93cb-53396efdfa8e" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:29:45 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5c4d21a3-c6a5-4555-918a-9267db1d0851 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquiring lock "2a3bcdea-be36-4a5b-93cb-53396efdfa8e-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:29:45 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5c4d21a3-c6a5-4555-918a-9267db1d0851 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "2a3bcdea-be36-4a5b-93cb-53396efdfa8e-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:29:45 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5c4d21a3-c6a5-4555-918a-9267db1d0851 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "2a3bcdea-be36-4a5b-93cb-53396efdfa8e-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:29:45 user nova-compute[72351]: INFO nova.compute.manager [None req-5c4d21a3-c6a5-4555-918a-9267db1d0851 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Terminating instance
Aug 09 02:29:45 user nova-compute[72351]: DEBUG nova.compute.manager [None req-5c4d21a3-c6a5-4555-918a-9267db1d0851 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Start destroying the instance on the hypervisor. {{(pid=72351) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3124}}
Aug 09 02:29:45 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-fbcb9773-9234-48f6-a8fe-2d10615b468e req-3b791ffa-8107-4b39-ac72-549b77b33729 service nova] Releasing lock "refresh_cache-2a3bcdea-be36-4a5b-93cb-53396efdfa8e" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:29:45 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:29:45 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:29:45 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:29:45 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:29:45 user nova-compute[72351]: DEBUG nova.compute.manager [req-85f24b84-bdef-43c9-b18b-4fee3545344d req-143e33a9-08a5-4d0d-848d-93e6b28726de service nova] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Received event network-vif-unplugged-4d36d764-e86b-40ee-81ed-a9f3adfadc48 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:29:45 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-85f24b84-bdef-43c9-b18b-4fee3545344d req-143e33a9-08a5-4d0d-848d-93e6b28726de service nova] Acquiring lock "2a3bcdea-be36-4a5b-93cb-53396efdfa8e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:29:45 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-85f24b84-bdef-43c9-b18b-4fee3545344d req-143e33a9-08a5-4d0d-848d-93e6b28726de service nova] Lock "2a3bcdea-be36-4a5b-93cb-53396efdfa8e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:29:45 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-85f24b84-bdef-43c9-b18b-4fee3545344d req-143e33a9-08a5-4d0d-848d-93e6b28726de service nova] Lock "2a3bcdea-be36-4a5b-93cb-53396efdfa8e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:29:45 user nova-compute[72351]: DEBUG nova.compute.manager [req-85f24b84-bdef-43c9-b18b-4fee3545344d req-143e33a9-08a5-4d0d-848d-93e6b28726de service nova] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] No waiting events found dispatching network-vif-unplugged-4d36d764-e86b-40ee-81ed-a9f3adfadc48 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:29:45 user nova-compute[72351]: DEBUG nova.compute.manager [req-85f24b84-bdef-43c9-b18b-4fee3545344d req-143e33a9-08a5-4d0d-848d-93e6b28726de service nova] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Received event network-vif-unplugged-4d36d764-e86b-40ee-81ed-a9f3adfadc48 for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:29:46 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Instance destroyed successfully.
Aug 09 02:29:46 user nova-compute[72351]: DEBUG nova.objects.instance [None req-5c4d21a3-c6a5-4555-918a-9267db1d0851 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lazy-loading 'resources' on Instance uuid 2a3bcdea-be36-4a5b-93cb-53396efdfa8e {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:29:46 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-5c4d21a3-c6a5-4555-918a-9267db1d0851 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:27:52Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-server-510987620',display_name='tempest-TestVolumeBootPattern-server-510987620',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testvolumebootpattern-server-510987620',id=31,image_ref='',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBM+1oHdu/+hmVTCW73FChAt+AvG2bL++YbEuGYMvrUaUYAyZQrt9Vwv7j34dhFmoVuCI0Rk18KgJ1X07X8pPcTOnOW0XFALfY2BbRHpJcmwARDVbYh8uPoNVxqSRoIR2gA==',key_name='tempest-TestVolumeBootPattern-805107831',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:28:21Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='a4c6bbfd8d904ba1ac69ba5da4564513',ramdisk_id='',reservation_id='r-65paxd61',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TestVolumeBootPattern-350743460',owner_user_name='tempest-TestVolumeBootPattern-350743460-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:28:21Z,user_data=None,user_id='07d8d5a634274df19c2db769d2d27b29',uuid=2a3bcdea-be36-4a5b-93cb-53396efdfa8e,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "4d36d764-e86b-40ee-81ed-a9f3adfadc48", "address": "fa:16:3e:28:03:d9", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4d36d764-e8", "ovs_interfaceid": "4d36d764-e86b-40ee-81ed-a9f3adfadc48", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:29:46 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-5c4d21a3-c6a5-4555-918a-9267db1d0851 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Converting VIF {"id": "4d36d764-e86b-40ee-81ed-a9f3adfadc48", "address": "fa:16:3e:28:03:d9", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4d36d764-e8", "ovs_interfaceid": "4d36d764-e86b-40ee-81ed-a9f3adfadc48", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:29:46 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-5c4d21a3-c6a5-4555-918a-9267db1d0851 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:28:03:d9,bridge_name='br-int',has_traffic_filtering=True,id=4d36d764-e86b-40ee-81ed-a9f3adfadc48,network=Network(5ef428cf-e408-490c-8f60-70934f760f4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap4d36d764-e8') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:29:46 user nova-compute[72351]: DEBUG os_vif [None req-5c4d21a3-c6a5-4555-918a-9267db1d0851 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:28:03:d9,bridge_name='br-int',has_traffic_filtering=True,id=4d36d764-e86b-40ee-81ed-a9f3adfadc48,network=Network(5ef428cf-e408-490c-8f60-70934f760f4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap4d36d764-e8') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:29:46 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:29:46 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap4d36d764-e8, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:29:46 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:29:46 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:29:46 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691548171.0649984, 831c864a-75d9-42b3-b952-749f4201e40e => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:29:46 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] VM Stopped (Lifecycle Event)
Aug 09 02:29:46 user nova-compute[72351]: INFO os_vif [None req-5c4d21a3-c6a5-4555-918a-9267db1d0851 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:28:03:d9,bridge_name='br-int',has_traffic_filtering=True,id=4d36d764-e86b-40ee-81ed-a9f3adfadc48,network=Network(5ef428cf-e408-490c-8f60-70934f760f4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap4d36d764-e8')
Aug 09 02:29:46 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-5c4d21a3-c6a5-4555-918a-9267db1d0851 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 02:29:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5c4d21a3-c6a5-4555-918a-9267db1d0851 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'CdzJhfwW', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdh'}, None), 'kwargs': True}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:29:46 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-5c4d21a3-c6a5-4555-918a-9267db1d0851 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:29:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-5c4d21a3-c6a5-4555-918a-9267db1d0851 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:29:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-5c4d21a3-c6a5-4555-918a-9267db1d0851 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:29:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5c4d21a3-c6a5-4555-918a-9267db1d0851 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 02:29:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:46 user nova-compute[72351]: DEBUG nova.compute.manager [None req-b94af0e9-287b-49c6-8963-ffaff7d03bd1 None None] [instance: 831c864a-75d9-42b3-b952-749f4201e40e] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:29:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:46 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[67814fda-a74c-457c-98db-771b8ca6b6aa]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:46 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[76e6a94c-bc03-4702-82a4-47f846a6f736]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)\ntcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5c4d21a3-c6a5-4555-918a-9267db1d0851 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:29:46 user nova-compute[72351]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:29:46 user nova-compute[72351]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:29:46 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:29:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5c4d21a3-c6a5-4555-918a-9267db1d0851 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:29:46 user nova-compute[72351]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:29:46 user nova-compute[72351]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:29:46 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:29:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5c4d21a3-c6a5-4555-918a-9267db1d0851 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fcfbc63e170>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448'): ({'sdh'}, set())}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 02:29:46 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-5c4d21a3-c6a5-4555-918a-9267db1d0851 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Removing single pathed devices sdh {{(pid=72351) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Aug 09 02:29:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:46 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[25dbb5f7-5502-4770-97ad-3a58ec3892cb]: (4, ('path checker states:\nup                  2\npending             1\n\npaths: 0\nbusy: False\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdh {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdh" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:46 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[7ca97aad-37a9-429a-8145-df5aa8b29840]: (4, ('ok\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:46 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-5c4d21a3-c6a5-4555-918a-9267db1d0851 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Flushing IO for device /dev/sdh {{(pid=72351) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Aug 09 02:29:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdh {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdh" returned: 0 in 0.014s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:46 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[95928ac7-1c5a-46d7-851b-bde7a085b7db]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:46 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-5c4d21a3-c6a5-4555-918a-9267db1d0851 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Remove SCSI device /dev/sdh with /sys/block/sdh/device/delete {{(pid=72351) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Aug 09 02:29:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdh/device/delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdh/device/delete" returned: 0 in 0.053s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:46 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[20629114-a950-425f-b997-01999dcf62e9]: (4, ('1', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:46 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-5c4d21a3-c6a5-4555-918a-9267db1d0851 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Checking to see if SCSI volumes sdh have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Aug 09 02:29:46 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-5c4d21a3-c6a5-4555-918a-9267db1d0851 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] SCSI volumes sdh have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Aug 09 02:29:46 user nova-compute[72351]: DEBUG os_brick.privileged.rootwrap [-] Unlinking ('/dev/disk/by-id/scsi-23931306338316130', '/dev/disk/by-id/scsi-1SCST_BIO26913bcd-910c81a04590072a', '/dev/disk/by-id/scsi-SSCST_BIO_Storage_26913bcd') {{(pid=83366) unlink_root /usr/local/lib/python3.10/dist-packages/os_brick/privileged/rootwrap.py:219}}
Aug 09 02:29:46 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a2cc1d04-b685-40b8-9c2a-79a26f24febd]: (4, None) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5c4d21a3-c6a5-4555-918a-9267db1d0851 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448')] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 02:29:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:46 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[63a1a1ad-ee20-49fb-a754-1c6478caeace]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5c4d21a3-c6a5-4555-918a-9267db1d0851 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:29:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 -p 172.16.0.220:3260 --logout {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 -p 172.16.0.220:3260 --logout" returned: 0 in 0.066s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:46 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ed4bdd52-e0a4-4606-9b4a-dffaf5f37584]: (4, ('Logging out of session [sid: 31, target: iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448, portal: 172.16.0.220,3260]\nLogout of [sid: 31, target: iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5c4d21a3-c6a5-4555-918a-9267db1d0851 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 31, target: iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448, portal: 172.16.0.220,3260]
Aug 09 02:29:46 user nova-compute[72351]: Logout of [sid: 31, target: iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448, portal: 172.16.0.220,3260] successful.
Aug 09 02:29:46 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:29:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 -p 172.16.0.220:3260 --op delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.024s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:46 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[5a8c8000-89fd-4d36-9156-b2eefbbfe5e2]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5c4d21a3-c6a5-4555-918a-9267db1d0851 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:29:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-5c4d21a3-c6a5-4555-918a-9267db1d0851 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.300s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:29:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5c4d21a3-c6a5-4555-918a-9267db1d0851 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] <== disconnect_volume: return (300ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:29:46 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-5c4d21a3-c6a5-4555-918a-9267db1d0851 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 02:29:46 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-5c4d21a3-c6a5-4555-918a-9267db1d0851 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Deleting instance files /opt/stack/data/nova/instances/2a3bcdea-be36-4a5b-93cb-53396efdfa8e_del
Aug 09 02:29:46 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-5c4d21a3-c6a5-4555-918a-9267db1d0851 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Deletion of /opt/stack/data/nova/instances/2a3bcdea-be36-4a5b-93cb-53396efdfa8e_del complete
Aug 09 02:29:46 user nova-compute[72351]: INFO nova.compute.manager [None req-5c4d21a3-c6a5-4555-918a-9267db1d0851 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Took 1.26 seconds to destroy the instance on the hypervisor.
Aug 09 02:29:46 user nova-compute[72351]: DEBUG oslo.service.loopingcall [None req-5c4d21a3-c6a5-4555-918a-9267db1d0851 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72351) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Aug 09 02:29:46 user nova-compute[72351]: DEBUG nova.compute.manager [-] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Deallocating network for instance {{(pid=72351) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2263}}
Aug 09 02:29:46 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] deallocate_for_instance() {{(pid=72351) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1795}}
Aug 09 02:29:47 user nova-compute[72351]: DEBUG nova.compute.manager [req-c420abc4-b579-4130-b640-56df64755d47 req-56e94202-b283-43e6-a0d8-477560c46c7a service nova] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Received event network-vif-plugged-4d36d764-e86b-40ee-81ed-a9f3adfadc48 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:29:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c420abc4-b579-4130-b640-56df64755d47 req-56e94202-b283-43e6-a0d8-477560c46c7a service nova] Acquiring lock "2a3bcdea-be36-4a5b-93cb-53396efdfa8e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:29:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c420abc4-b579-4130-b640-56df64755d47 req-56e94202-b283-43e6-a0d8-477560c46c7a service nova] Lock "2a3bcdea-be36-4a5b-93cb-53396efdfa8e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:29:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c420abc4-b579-4130-b640-56df64755d47 req-56e94202-b283-43e6-a0d8-477560c46c7a service nova] Lock "2a3bcdea-be36-4a5b-93cb-53396efdfa8e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:29:47 user nova-compute[72351]: DEBUG nova.compute.manager [req-c420abc4-b579-4130-b640-56df64755d47 req-56e94202-b283-43e6-a0d8-477560c46c7a service nova] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] No waiting events found dispatching network-vif-plugged-4d36d764-e86b-40ee-81ed-a9f3adfadc48 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:29:47 user nova-compute[72351]: WARNING nova.compute.manager [req-c420abc4-b579-4130-b640-56df64755d47 req-56e94202-b283-43e6-a0d8-477560c46c7a service nova] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Received unexpected event network-vif-plugged-4d36d764-e86b-40ee-81ed-a9f3adfadc48 for instance with vm_state active and task_state deleting.
Aug 09 02:29:47 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:29:47 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Took 1.31 seconds to deallocate network for instance.
Aug 09 02:29:48 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:29:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Acquiring lock "38a2b8e3-337d-416a-8a8c-da4921123db6" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:29:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:29:48 user nova-compute[72351]: DEBUG nova.compute.manager [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Starting instance... {{(pid=72351) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2406}}
Aug 09 02:29:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:29:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:29:48 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72351) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Aug 09 02:29:48 user nova-compute[72351]: INFO nova.compute.claims [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Claim successful on node user
Aug 09 02:29:48 user nova-compute[72351]: INFO nova.compute.manager [None req-5c4d21a3-c6a5-4555-918a-9267db1d0851 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Took 0.76 seconds to detach 1 volumes for instance.
Aug 09 02:29:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5c4d21a3-c6a5-4555-918a-9267db1d0851 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:29:48 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:29:48 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:29:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.437s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:29:48 user nova-compute[72351]: DEBUG nova.compute.manager [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Start building networks asynchronously for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2803}}
Aug 09 02:29:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5c4d21a3-c6a5-4555-918a-9267db1d0851 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.088s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:29:48 user nova-compute[72351]: DEBUG nova.compute.manager [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Allocating IP information in the background. {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1956}}
Aug 09 02:29:48 user nova-compute[72351]: DEBUG nova.network.neutron [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] allocate_for_instance() {{(pid=72351) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Aug 09 02:29:48 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Aug 09 02:29:48 user nova-compute[72351]: DEBUG nova.compute.manager [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Start building block device mappings for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2838}}
Aug 09 02:29:48 user nova-compute[72351]: INFO nova.virt.block_device [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Booting with volume 9d91e65d-30f6-4d64-a186-cbd8f80000b3 at /dev/vda
Aug 09 02:29:49 user nova-compute[72351]: DEBUG os_brick.utils [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:29:49 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:49 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:49 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[c3a77f91-8aae-47d4-81f9-ec183f5867b6]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:a52269602ff4\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:49 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:49 user nova-compute[72351]: DEBUG nova.policy [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '76924d471e204a5ebee3c99280e62c91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb5ff9ee85084536b6479ed16b24293d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72351) authorize /opt/stack/nova/nova/policy.py:203}}
Aug 09 02:29:49 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.015s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:49 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[200bf04a-2b7b-4369-92a9-577cca236b73]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:49 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:49 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:49 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[c4b04e63-4cc6-4219-8a3f-76aa0ab0f206]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:49 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[12655fd0-23e9-42c9-914d-8f2f3687a926]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:49 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Running cmd (subprocess): nvme version {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:49 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] 'nvme version' failed. Not Retrying. {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:29:49 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.nvmeof [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] nvme not present on system {{(pid=72351) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Aug 09 02:29:49 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:49 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-5c4d21a3-c6a5-4555-918a-9267db1d0851 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:29:49 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:29:49 user nova-compute[72351]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Aug 09 02:29:49 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[dc43f98c-bab4-4c0b-8990-c1ca4d96b170]: (4, '') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:49 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72351) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Aug 09 02:29:49 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Aug 09 02:29:49 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] LIGHTOS: no hostnqn found. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Aug 09 02:29:49 user nova-compute[72351]: DEBUG os_brick.utils [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] <== get_connector_properties: return (132ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:a52269602ff4', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:29:49 user nova-compute[72351]: DEBUG nova.virt.block_device [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Updating existing volume attachment record: 33c7551c-71dd-4dd9-8a06-8fa840185786 {{(pid=72351) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Aug 09 02:29:49 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-5c4d21a3-c6a5-4555-918a-9267db1d0851 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:29:49 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5c4d21a3-c6a5-4555-918a-9267db1d0851 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.427s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:29:49 user nova-compute[72351]: INFO nova.scheduler.client.report [None req-5c4d21a3-c6a5-4555-918a-9267db1d0851 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Deleted allocations for instance 2a3bcdea-be36-4a5b-93cb-53396efdfa8e
Aug 09 02:29:49 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5c4d21a3-c6a5-4555-918a-9267db1d0851 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "2a3bcdea-be36-4a5b-93cb-53396efdfa8e" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 4.019s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:29:49 user nova-compute[72351]: DEBUG nova.compute.manager [req-4c3ae305-37e9-47dc-8793-94ca0b123db5 req-f6dbf764-d32c-48b3-b292-8ceda95eb80b service nova] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Received event network-vif-deleted-4d36d764-e86b-40ee-81ed-a9f3adfadc48 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:29:49 user nova-compute[72351]: DEBUG nova.network.neutron [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Successfully created port: 0ebd83d0-5519-4945-8637-90c2ce024ac1 {{(pid=72351) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Aug 09 02:29:50 user nova-compute[72351]: DEBUG nova.network.neutron [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Successfully created port: ccb930f6-9625-450b-b01b-c5009e610ae1 {{(pid=72351) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Aug 09 02:29:51 user nova-compute[72351]: DEBUG nova.network.neutron [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Successfully created port: d083d9f2-dfe4-4a73-a624-3336f2f02b0d {{(pid=72351) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Aug 09 02:29:51 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:29:51 user nova-compute[72351]: DEBUG nova.network.neutron [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Successfully created port: a88b872b-9340-4597-8ab4-ea6173dc4e4c {{(pid=72351) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Aug 09 02:29:52 user nova-compute[72351]: DEBUG nova.network.neutron [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Successfully created port: 48dfbf61-0142-4478-9ec1-585d5fbe11da {{(pid=72351) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Aug 09 02:29:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-2eada4f9-4e0e-4880-baef-fe12293d89b3 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Acquiring lock "75bbbead-8628-448b-9484-19bf4ab922ff" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:29:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-2eada4f9-4e0e-4880-baef-fe12293d89b3 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Lock "75bbbead-8628-448b-9484-19bf4ab922ff" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:29:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-2eada4f9-4e0e-4880-baef-fe12293d89b3 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Acquiring lock "75bbbead-8628-448b-9484-19bf4ab922ff-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:29:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-2eada4f9-4e0e-4880-baef-fe12293d89b3 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Lock "75bbbead-8628-448b-9484-19bf4ab922ff-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:29:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-2eada4f9-4e0e-4880-baef-fe12293d89b3 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Lock "75bbbead-8628-448b-9484-19bf4ab922ff-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:29:52 user nova-compute[72351]: INFO nova.compute.manager [None req-2eada4f9-4e0e-4880-baef-fe12293d89b3 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Terminating instance
Aug 09 02:29:52 user nova-compute[72351]: DEBUG nova.compute.manager [None req-2eada4f9-4e0e-4880-baef-fe12293d89b3 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Start destroying the instance on the hypervisor. {{(pid=72351) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3124}}
Aug 09 02:29:52 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:29:52 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:29:52 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:29:52 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:29:52 user nova-compute[72351]: DEBUG nova.compute.manager [req-16c00699-29f1-4fdb-ad9e-435d58fe75ad req-f0dea93e-8121-4bac-9507-bac894a3ce7b service nova] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Received event network-vif-unplugged-be4ec483-8532-45f7-bb9f-0b50952ce94b {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:29:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-16c00699-29f1-4fdb-ad9e-435d58fe75ad req-f0dea93e-8121-4bac-9507-bac894a3ce7b service nova] Acquiring lock "75bbbead-8628-448b-9484-19bf4ab922ff-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:29:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-16c00699-29f1-4fdb-ad9e-435d58fe75ad req-f0dea93e-8121-4bac-9507-bac894a3ce7b service nova] Lock "75bbbead-8628-448b-9484-19bf4ab922ff-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:29:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-16c00699-29f1-4fdb-ad9e-435d58fe75ad req-f0dea93e-8121-4bac-9507-bac894a3ce7b service nova] Lock "75bbbead-8628-448b-9484-19bf4ab922ff-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:29:52 user nova-compute[72351]: DEBUG nova.compute.manager [req-16c00699-29f1-4fdb-ad9e-435d58fe75ad req-f0dea93e-8121-4bac-9507-bac894a3ce7b service nova] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] No waiting events found dispatching network-vif-unplugged-be4ec483-8532-45f7-bb9f-0b50952ce94b {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:29:52 user nova-compute[72351]: DEBUG nova.compute.manager [req-16c00699-29f1-4fdb-ad9e-435d58fe75ad req-f0dea93e-8121-4bac-9507-bac894a3ce7b service nova] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Received event network-vif-unplugged-be4ec483-8532-45f7-bb9f-0b50952ce94b for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG nova.network.neutron [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Successfully updated port: 0ebd83d0-5519-4945-8637-90c2ce024ac1 {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG nova.compute.manager [req-d7bd4c84-302a-44bc-ab35-1fc8fafd453c req-c2b091d1-2b31-48d2-af18-76829716db8d service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received event network-changed-0ebd83d0-5519-4945-8637-90c2ce024ac1 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG nova.compute.manager [req-d7bd4c84-302a-44bc-ab35-1fc8fafd453c req-c2b091d1-2b31-48d2-af18-76829716db8d service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Refreshing instance network info cache due to event network-changed-0ebd83d0-5519-4945-8637-90c2ce024ac1. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d7bd4c84-302a-44bc-ab35-1fc8fafd453c req-c2b091d1-2b31-48d2-af18-76829716db8d service nova] Acquiring lock "refresh_cache-38a2b8e3-337d-416a-8a8c-da4921123db6" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d7bd4c84-302a-44bc-ab35-1fc8fafd453c req-c2b091d1-2b31-48d2-af18-76829716db8d service nova] Acquired lock "refresh_cache-38a2b8e3-337d-416a-8a8c-da4921123db6" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG nova.network.neutron [req-d7bd4c84-302a-44bc-ab35-1fc8fafd453c req-c2b091d1-2b31-48d2-af18-76829716db8d service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Refreshing network info cache for port 0ebd83d0-5519-4945-8637-90c2ce024ac1 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG nova.network.neutron [req-d7bd4c84-302a-44bc-ab35-1fc8fafd453c req-c2b091d1-2b31-48d2-af18-76829716db8d service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:29:53 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Instance destroyed successfully.
Aug 09 02:29:53 user nova-compute[72351]: DEBUG nova.objects.instance [None req-2eada4f9-4e0e-4880-baef-fe12293d89b3 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Lazy-loading 'resources' on Instance uuid 75bbbead-8628-448b-9484-19bf4ab922ff {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-2eada4f9-4e0e-4880-baef-fe12293d89b3 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:21:09Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestOtherB-server-634605118',display_name='tempest-ServerActionsTestOtherB-server-634605118',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serveractionstestotherb-server-634605118',id=15,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLdtpWQKmHav2tGxZemUVMEaT9TXyD7Duhy5QV84PPI+5KrgLVkbxVzWgryocScOebEprbcJ80bjBPnYhLtQJYSe5MXXz6GOQHLb+tJCZajhrDoMbn41/bPAnB5B3oFwCw==',key_name='tempest-keypair-644333596',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:27:20Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='fc90e94bbcb54fb6b86a9ac45edb596a',ramdisk_id='',reservation_id='r-ozz8gdfj',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerActionsTestOtherB-1726991791',owner_user_name='tempest-ServerActionsTestOtherB-1726991791-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:27:21Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='19668b0c790d49dd830c39e3c2be29bf',uuid=75bbbead-8628-448b-9484-19bf4ab922ff,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "address": "fa:16:3e:8b:b5:2e", "network": {"id": "8231f2c2-237e-41c6-8892-720b950c1453", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherB-1917380761-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.30", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fc90e94bbcb54fb6b86a9ac45edb596a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbe4ec483-85", "ovs_interfaceid": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-2eada4f9-4e0e-4880-baef-fe12293d89b3 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Converting VIF {"id": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "address": "fa:16:3e:8b:b5:2e", "network": {"id": "8231f2c2-237e-41c6-8892-720b950c1453", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherB-1917380761-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.30", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fc90e94bbcb54fb6b86a9ac45edb596a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbe4ec483-85", "ovs_interfaceid": "be4ec483-8532-45f7-bb9f-0b50952ce94b", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-2eada4f9-4e0e-4880-baef-fe12293d89b3 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:8b:b5:2e,bridge_name='br-int',has_traffic_filtering=True,id=be4ec483-8532-45f7-bb9f-0b50952ce94b,network=Network(8231f2c2-237e-41c6-8892-720b950c1453),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapbe4ec483-85') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG os_vif [None req-2eada4f9-4e0e-4880-baef-fe12293d89b3 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:8b:b5:2e,bridge_name='br-int',has_traffic_filtering=True,id=be4ec483-8532-45f7-bb9f-0b50952ce94b,network=Network(8231f2c2-237e-41c6-8892-720b950c1453),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapbe4ec483-85') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG nova.network.neutron [req-d7bd4c84-302a-44bc-ab35-1fc8fafd453c req-c2b091d1-2b31-48d2-af18-76829716db8d service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapbe4ec483-85, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:29:53 user nova-compute[72351]: INFO os_vif [None req-2eada4f9-4e0e-4880-baef-fe12293d89b3 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:8b:b5:2e,bridge_name='br-int',has_traffic_filtering=True,id=be4ec483-8532-45f7-bb9f-0b50952ce94b,network=Network(8231f2c2-237e-41c6-8892-720b950c1453),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapbe4ec483-85')
Aug 09 02:29:53 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d7bd4c84-302a-44bc-ab35-1fc8fafd453c req-c2b091d1-2b31-48d2-af18-76829716db8d service nova] Releasing lock "refresh_cache-38a2b8e3-337d-416a-8a8c-da4921123db6" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-2eada4f9-4e0e-4880-baef-fe12293d89b3 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-2eada4f9-4e0e-4880-baef-fe12293d89b3 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'oTnweUhv', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb'}, None), 'kwargs': True}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:29:53 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-2eada4f9-4e0e-4880-baef-fe12293d89b3 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:29:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-2eada4f9-4e0e-4880-baef-fe12293d89b3 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-2eada4f9-4e0e-4880-baef-fe12293d89b3 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-2eada4f9-4e0e-4880-baef-fe12293d89b3 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[c0ee89e5-c2d8-4ba1-bbb6-9d4c0ae24ae0]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[0b290dff-9e29-46ea-98c9-a859b4c740cd]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-2eada4f9-4e0e-4880-baef-fe12293d89b3 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:29:53 user nova-compute[72351]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:29:53 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-2eada4f9-4e0e-4880-baef-fe12293d89b3 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:29:53 user nova-compute[72351]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd (non-flash)
Aug 09 02:29:53 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-2eada4f9-4e0e-4880-baef-fe12293d89b3 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fcfdc4ce9e0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd'): ({'sdb'}, set())}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-2eada4f9-4e0e-4880-baef-fe12293d89b3 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Removing single pathed devices sdb {{(pid=72351) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[53b5e83f-999b-46cd-b0bd-fc469f533cb6]: (4, ('path checker states:\nup                  1\npending             1\n\npaths: 0\nbusy: False\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[5656e238-cbca-492b-9ab7-ac6e72c4f962]: (4, ('ok\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-2eada4f9-4e0e-4880-baef-fe12293d89b3 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Flushing IO for device /dev/sdb {{(pid=72351) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.015s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[4dd3fa90-2dab-4587-9705-6721a58bfa18]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-2eada4f9-4e0e-4880-baef-fe12293d89b3 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=72351) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.035s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[3c94b173-dfb9-4b7c-abdb-c0f60cb9a9ca]: (4, ('1', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-2eada4f9-4e0e-4880-baef-fe12293d89b3 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-2eada4f9-4e0e-4880-baef-fe12293d89b3 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] SCSI volumes sdb have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-2eada4f9-4e0e-4880-baef-fe12293d89b3 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd')] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[becb18d0-c651-4449-b7b1-280e9d7449c2]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-2eada4f9-4e0e-4880-baef-fe12293d89b3 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 --logout {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 --logout" returned: 0 in 0.060s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[4392fe71-1c80-4872-aa8a-70db03dc1f79]: (4, ('Logging out of session [sid: 28, target: iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd, portal: 172.16.0.220,3260]\nLogout of [sid: 28, target: iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-2eada4f9-4e0e-4880-baef-fe12293d89b3 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 28, target: iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd, portal: 172.16.0.220,3260]
Aug 09 02:29:53 user nova-compute[72351]: Logout of [sid: 28, target: iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd, portal: 172.16.0.220,3260] successful.
Aug 09 02:29:53 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 --op delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:2db51b4f-d032-4367-bb6f-8964563eb8fd -p 172.16.0.220:3260 --op delete" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[61f6d88d-1449-4e81-854b-d30f0d802b61]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-2eada4f9-4e0e-4880-baef-fe12293d89b3 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-2eada4f9-4e0e-4880-baef-fe12293d89b3 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.272s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-2eada4f9-4e0e-4880-baef-fe12293d89b3 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] <== disconnect_volume: return (272ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-2eada4f9-4e0e-4880-baef-fe12293d89b3 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 02:29:53 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-2eada4f9-4e0e-4880-baef-fe12293d89b3 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Deleting instance files /opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff_del
Aug 09 02:29:53 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-2eada4f9-4e0e-4880-baef-fe12293d89b3 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Deletion of /opt/stack/data/nova/instances/75bbbead-8628-448b-9484-19bf4ab922ff_del complete
Aug 09 02:29:53 user nova-compute[72351]: INFO nova.compute.manager [None req-2eada4f9-4e0e-4880-baef-fe12293d89b3 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Took 1.19 seconds to destroy the instance on the hypervisor.
Aug 09 02:29:53 user nova-compute[72351]: DEBUG oslo.service.loopingcall [None req-2eada4f9-4e0e-4880-baef-fe12293d89b3 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72351) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG nova.compute.manager [-] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Deallocating network for instance {{(pid=72351) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2263}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] deallocate_for_instance() {{(pid=72351) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1795}}
Aug 09 02:29:53 user nova-compute[72351]: DEBUG nova.network.neutron [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Successfully updated port: 8b1245ae-85e9-49c3-9631-a288a172e4c1 {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:29:54 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:29:54 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Took 0.76 seconds to deallocate network for instance.
Aug 09 02:29:54 user nova-compute[72351]: DEBUG nova.network.neutron [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Successfully updated port: 1be27a59-07a4-4b6a-b25a-ce6d7efac4a3 {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:29:55 user nova-compute[72351]: DEBUG nova.compute.manager [req-34a76c69-2026-4706-ad11-9219e5d14f0c req-6d90088a-10c1-4446-bbf0-40e96da0d056 service nova] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Received event network-vif-plugged-be4ec483-8532-45f7-bb9f-0b50952ce94b {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:29:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-34a76c69-2026-4706-ad11-9219e5d14f0c req-6d90088a-10c1-4446-bbf0-40e96da0d056 service nova] Acquiring lock "75bbbead-8628-448b-9484-19bf4ab922ff-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:29:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-34a76c69-2026-4706-ad11-9219e5d14f0c req-6d90088a-10c1-4446-bbf0-40e96da0d056 service nova] Lock "75bbbead-8628-448b-9484-19bf4ab922ff-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:29:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-34a76c69-2026-4706-ad11-9219e5d14f0c req-6d90088a-10c1-4446-bbf0-40e96da0d056 service nova] Lock "75bbbead-8628-448b-9484-19bf4ab922ff-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:29:55 user nova-compute[72351]: DEBUG nova.compute.manager [req-34a76c69-2026-4706-ad11-9219e5d14f0c req-6d90088a-10c1-4446-bbf0-40e96da0d056 service nova] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] No waiting events found dispatching network-vif-plugged-be4ec483-8532-45f7-bb9f-0b50952ce94b {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:29:55 user nova-compute[72351]: WARNING nova.compute.manager [req-34a76c69-2026-4706-ad11-9219e5d14f0c req-6d90088a-10c1-4446-bbf0-40e96da0d056 service nova] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Received unexpected event network-vif-plugged-be4ec483-8532-45f7-bb9f-0b50952ce94b for instance with vm_state active and task_state deleting.
Aug 09 02:29:55 user nova-compute[72351]: DEBUG nova.compute.manager [req-34a76c69-2026-4706-ad11-9219e5d14f0c req-6d90088a-10c1-4446-bbf0-40e96da0d056 service nova] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Received event network-vif-deleted-be4ec483-8532-45f7-bb9f-0b50952ce94b {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:29:55 user nova-compute[72351]: DEBUG nova.compute.manager [req-6dbeefe8-8b61-4734-9f1f-a4f19e976769 req-4d7ca66e-1447-41e8-910c-015b4198cf25 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received event network-changed-8b1245ae-85e9-49c3-9631-a288a172e4c1 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:29:55 user nova-compute[72351]: DEBUG nova.compute.manager [req-6dbeefe8-8b61-4734-9f1f-a4f19e976769 req-4d7ca66e-1447-41e8-910c-015b4198cf25 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Refreshing instance network info cache due to event network-changed-8b1245ae-85e9-49c3-9631-a288a172e4c1. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:29:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-6dbeefe8-8b61-4734-9f1f-a4f19e976769 req-4d7ca66e-1447-41e8-910c-015b4198cf25 service nova] Acquiring lock "refresh_cache-38a2b8e3-337d-416a-8a8c-da4921123db6" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:29:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-6dbeefe8-8b61-4734-9f1f-a4f19e976769 req-4d7ca66e-1447-41e8-910c-015b4198cf25 service nova] Acquired lock "refresh_cache-38a2b8e3-337d-416a-8a8c-da4921123db6" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:29:55 user nova-compute[72351]: DEBUG nova.network.neutron [req-6dbeefe8-8b61-4734-9f1f-a4f19e976769 req-4d7ca66e-1447-41e8-910c-015b4198cf25 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Refreshing network info cache for port 8b1245ae-85e9-49c3-9631-a288a172e4c1 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:29:55 user nova-compute[72351]: DEBUG nova.network.neutron [req-6dbeefe8-8b61-4734-9f1f-a4f19e976769 req-4d7ca66e-1447-41e8-910c-015b4198cf25 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:29:55 user nova-compute[72351]: INFO nova.compute.manager [None req-2eada4f9-4e0e-4880-baef-fe12293d89b3 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Took 0.73 seconds to detach 1 volumes for instance.
Aug 09 02:29:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-2eada4f9-4e0e-4880-baef-fe12293d89b3 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:29:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-2eada4f9-4e0e-4880-baef-fe12293d89b3 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:29:55 user nova-compute[72351]: DEBUG nova.network.neutron [req-6dbeefe8-8b61-4734-9f1f-a4f19e976769 req-4d7ca66e-1447-41e8-910c-015b4198cf25 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:29:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-6dbeefe8-8b61-4734-9f1f-a4f19e976769 req-4d7ca66e-1447-41e8-910c-015b4198cf25 service nova] Releasing lock "refresh_cache-38a2b8e3-337d-416a-8a8c-da4921123db6" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:29:55 user nova-compute[72351]: DEBUG nova.compute.manager [req-6dbeefe8-8b61-4734-9f1f-a4f19e976769 req-4d7ca66e-1447-41e8-910c-015b4198cf25 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received event network-changed-1be27a59-07a4-4b6a-b25a-ce6d7efac4a3 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:29:55 user nova-compute[72351]: DEBUG nova.compute.manager [req-6dbeefe8-8b61-4734-9f1f-a4f19e976769 req-4d7ca66e-1447-41e8-910c-015b4198cf25 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Refreshing instance network info cache due to event network-changed-1be27a59-07a4-4b6a-b25a-ce6d7efac4a3. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:29:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-6dbeefe8-8b61-4734-9f1f-a4f19e976769 req-4d7ca66e-1447-41e8-910c-015b4198cf25 service nova] Acquiring lock "refresh_cache-38a2b8e3-337d-416a-8a8c-da4921123db6" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:29:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-6dbeefe8-8b61-4734-9f1f-a4f19e976769 req-4d7ca66e-1447-41e8-910c-015b4198cf25 service nova] Acquired lock "refresh_cache-38a2b8e3-337d-416a-8a8c-da4921123db6" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:29:55 user nova-compute[72351]: DEBUG nova.network.neutron [req-6dbeefe8-8b61-4734-9f1f-a4f19e976769 req-4d7ca66e-1447-41e8-910c-015b4198cf25 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Refreshing network info cache for port 1be27a59-07a4-4b6a-b25a-ce6d7efac4a3 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:29:55 user nova-compute[72351]: DEBUG nova.network.neutron [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Successfully updated port: ccb930f6-9625-450b-b01b-c5009e610ae1 {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:29:55 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-2eada4f9-4e0e-4880-baef-fe12293d89b3 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:29:55 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-2eada4f9-4e0e-4880-baef-fe12293d89b3 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:29:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-2eada4f9-4e0e-4880-baef-fe12293d89b3 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.552s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:29:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-cf985c63-02a4-488a-80ca-b04fc6f71d3d tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Acquiring lock "16c1e558-baa1-4c2d-b13b-50585f74d587" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:29:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-cf985c63-02a4-488a-80ca-b04fc6f71d3d tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Lock "16c1e558-baa1-4c2d-b13b-50585f74d587" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:29:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-cf985c63-02a4-488a-80ca-b04fc6f71d3d tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Acquiring lock "16c1e558-baa1-4c2d-b13b-50585f74d587-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:29:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-cf985c63-02a4-488a-80ca-b04fc6f71d3d tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Lock "16c1e558-baa1-4c2d-b13b-50585f74d587-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:29:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-cf985c63-02a4-488a-80ca-b04fc6f71d3d tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Lock "16c1e558-baa1-4c2d-b13b-50585f74d587-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:29:55 user nova-compute[72351]: INFO nova.compute.manager [None req-cf985c63-02a4-488a-80ca-b04fc6f71d3d tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Terminating instance
Aug 09 02:29:55 user nova-compute[72351]: DEBUG nova.compute.manager [None req-cf985c63-02a4-488a-80ca-b04fc6f71d3d tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Start destroying the instance on the hypervisor. {{(pid=72351) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3124}}
Aug 09 02:29:55 user nova-compute[72351]: INFO nova.scheduler.client.report [None req-2eada4f9-4e0e-4880-baef-fe12293d89b3 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Deleted allocations for instance 75bbbead-8628-448b-9484-19bf4ab922ff
Aug 09 02:29:55 user nova-compute[72351]: DEBUG nova.network.neutron [req-6dbeefe8-8b61-4734-9f1f-a4f19e976769 req-4d7ca66e-1447-41e8-910c-015b4198cf25 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:29:56 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:29:56 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:29:56 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-2eada4f9-4e0e-4880-baef-fe12293d89b3 tempest-ServerActionsTestOtherB-1726991791 tempest-ServerActionsTestOtherB-1726991791-project-member] Lock "75bbbead-8628-448b-9484-19bf4ab922ff" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 3.457s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:29:56 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:29:56 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:29:56 user nova-compute[72351]: DEBUG nova.network.neutron [req-6dbeefe8-8b61-4734-9f1f-a4f19e976769 req-4d7ca66e-1447-41e8-910c-015b4198cf25 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:29:56 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-6dbeefe8-8b61-4734-9f1f-a4f19e976769 req-4d7ca66e-1447-41e8-910c-015b4198cf25 service nova] Releasing lock "refresh_cache-38a2b8e3-337d-416a-8a8c-da4921123db6" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:29:56 user nova-compute[72351]: DEBUG nova.network.neutron [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Successfully updated port: d083d9f2-dfe4-4a73-a624-3336f2f02b0d {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:29:56 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Instance destroyed successfully.
Aug 09 02:29:56 user nova-compute[72351]: DEBUG nova.objects.instance [None req-cf985c63-02a4-488a-80ca-b04fc6f71d3d tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Lazy-loading 'resources' on Instance uuid 16c1e558-baa1-4c2d-b13b-50585f74d587 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:29:56 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-cf985c63-02a4-488a-80ca-b04fc6f71d3d tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:25:54Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestServerVolumeAttachmentScenario-server-360751817',display_name='tempest-TestServerVolumeAttachmentScenario-server-360751817',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testservervolumeattachmentscenario-server-360751817',id=27,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBO5LKDWUPul31VlDBgVZo5WIbldUiPko7qY2o+u8Xw9ekIjzjjzrKBVPG8SI+m7Xh+/4dSwSjsie+3WOYIYW29h/ArzglDh0AzoC+h3SECFUWbPsf8al1tugMF9vK3vduA==',key_name='tempest-keypair-148936292',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:26:02Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='de4a7655f2604c8a9c7964cfd298be37',ramdisk_id='',reservation_id='r-s9d31yuy',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TestServerVolumeAttachmentScenario-1098806376',owner_user_name='tempest-TestServerVolumeAttachmentScenario-1098806376-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:26:02Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='fd71797160d14dfbba2c65eca2e2f694',uuid=16c1e558-baa1-4c2d-b13b-50585f74d587,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "fb1b25f9-3a24-43df-805c-9fffe7cd74f7", "address": "fa:16:3e:66:3c:d2", "network": {"id": "39ddd1d4-5757-425b-92e8-a6a5d8ff29db", "bridge": "br-int", "label": "tempest-TestServerVolumeAttachmentScenario-2122827433-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.175", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "de4a7655f2604c8a9c7964cfd298be37", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfb1b25f9-3a", "ovs_interfaceid": "fb1b25f9-3a24-43df-805c-9fffe7cd74f7", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:29:56 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-cf985c63-02a4-488a-80ca-b04fc6f71d3d tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Converting VIF {"id": "fb1b25f9-3a24-43df-805c-9fffe7cd74f7", "address": "fa:16:3e:66:3c:d2", "network": {"id": "39ddd1d4-5757-425b-92e8-a6a5d8ff29db", "bridge": "br-int", "label": "tempest-TestServerVolumeAttachmentScenario-2122827433-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.175", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "de4a7655f2604c8a9c7964cfd298be37", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfb1b25f9-3a", "ovs_interfaceid": "fb1b25f9-3a24-43df-805c-9fffe7cd74f7", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:29:56 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-cf985c63-02a4-488a-80ca-b04fc6f71d3d tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:66:3c:d2,bridge_name='br-int',has_traffic_filtering=True,id=fb1b25f9-3a24-43df-805c-9fffe7cd74f7,network=Network(39ddd1d4-5757-425b-92e8-a6a5d8ff29db),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfb1b25f9-3a') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:29:56 user nova-compute[72351]: DEBUG os_vif [None req-cf985c63-02a4-488a-80ca-b04fc6f71d3d tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:66:3c:d2,bridge_name='br-int',has_traffic_filtering=True,id=fb1b25f9-3a24-43df-805c-9fffe7cd74f7,network=Network(39ddd1d4-5757-425b-92e8-a6a5d8ff29db),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfb1b25f9-3a') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:29:56 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:29:56 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapfb1b25f9-3a, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:29:56 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:29:56 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:29:56 user nova-compute[72351]: INFO os_vif [None req-cf985c63-02a4-488a-80ca-b04fc6f71d3d tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:66:3c:d2,bridge_name='br-int',has_traffic_filtering=True,id=fb1b25f9-3a24-43df-805c-9fffe7cd74f7,network=Network(39ddd1d4-5757-425b-92e8-a6a5d8ff29db),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfb1b25f9-3a')
Aug 09 02:29:56 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-cf985c63-02a4-488a-80ca-b04fc6f71d3d tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Deleting instance files /opt/stack/data/nova/instances/16c1e558-baa1-4c2d-b13b-50585f74d587_del
Aug 09 02:29:56 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-cf985c63-02a4-488a-80ca-b04fc6f71d3d tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Deletion of /opt/stack/data/nova/instances/16c1e558-baa1-4c2d-b13b-50585f74d587_del complete
Aug 09 02:29:56 user nova-compute[72351]: INFO nova.compute.manager [None req-cf985c63-02a4-488a-80ca-b04fc6f71d3d tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Took 0.91 seconds to destroy the instance on the hypervisor.
Aug 09 02:29:56 user nova-compute[72351]: DEBUG oslo.service.loopingcall [None req-cf985c63-02a4-488a-80ca-b04fc6f71d3d tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72351) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Aug 09 02:29:56 user nova-compute[72351]: DEBUG nova.compute.manager [-] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Deallocating network for instance {{(pid=72351) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2263}}
Aug 09 02:29:56 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] deallocate_for_instance() {{(pid=72351) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1795}}
Aug 09 02:29:57 user nova-compute[72351]: DEBUG nova.compute.manager [req-80e29139-f109-4bca-bc0c-3595f18f1593 req-4c296737-d4e4-4d8c-8ac3-98281f59c267 service nova] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Received event network-vif-unplugged-fb1b25f9-3a24-43df-805c-9fffe7cd74f7 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:29:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-80e29139-f109-4bca-bc0c-3595f18f1593 req-4c296737-d4e4-4d8c-8ac3-98281f59c267 service nova] Acquiring lock "16c1e558-baa1-4c2d-b13b-50585f74d587-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:29:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-80e29139-f109-4bca-bc0c-3595f18f1593 req-4c296737-d4e4-4d8c-8ac3-98281f59c267 service nova] Lock "16c1e558-baa1-4c2d-b13b-50585f74d587-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:29:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-80e29139-f109-4bca-bc0c-3595f18f1593 req-4c296737-d4e4-4d8c-8ac3-98281f59c267 service nova] Lock "16c1e558-baa1-4c2d-b13b-50585f74d587-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:29:57 user nova-compute[72351]: DEBUG nova.compute.manager [req-80e29139-f109-4bca-bc0c-3595f18f1593 req-4c296737-d4e4-4d8c-8ac3-98281f59c267 service nova] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] No waiting events found dispatching network-vif-unplugged-fb1b25f9-3a24-43df-805c-9fffe7cd74f7 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:29:57 user nova-compute[72351]: DEBUG nova.compute.manager [req-80e29139-f109-4bca-bc0c-3595f18f1593 req-4c296737-d4e4-4d8c-8ac3-98281f59c267 service nova] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Received event network-vif-unplugged-fb1b25f9-3a24-43df-805c-9fffe7cd74f7 for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:29:57 user nova-compute[72351]: DEBUG nova.compute.manager [req-80e29139-f109-4bca-bc0c-3595f18f1593 req-4c296737-d4e4-4d8c-8ac3-98281f59c267 service nova] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Received event network-vif-plugged-fb1b25f9-3a24-43df-805c-9fffe7cd74f7 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:29:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-80e29139-f109-4bca-bc0c-3595f18f1593 req-4c296737-d4e4-4d8c-8ac3-98281f59c267 service nova] Acquiring lock "16c1e558-baa1-4c2d-b13b-50585f74d587-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:29:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-80e29139-f109-4bca-bc0c-3595f18f1593 req-4c296737-d4e4-4d8c-8ac3-98281f59c267 service nova] Lock "16c1e558-baa1-4c2d-b13b-50585f74d587-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:29:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-80e29139-f109-4bca-bc0c-3595f18f1593 req-4c296737-d4e4-4d8c-8ac3-98281f59c267 service nova] Lock "16c1e558-baa1-4c2d-b13b-50585f74d587-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:29:57 user nova-compute[72351]: DEBUG nova.compute.manager [req-80e29139-f109-4bca-bc0c-3595f18f1593 req-4c296737-d4e4-4d8c-8ac3-98281f59c267 service nova] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] No waiting events found dispatching network-vif-plugged-fb1b25f9-3a24-43df-805c-9fffe7cd74f7 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:29:57 user nova-compute[72351]: WARNING nova.compute.manager [req-80e29139-f109-4bca-bc0c-3595f18f1593 req-4c296737-d4e4-4d8c-8ac3-98281f59c267 service nova] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Received unexpected event network-vif-plugged-fb1b25f9-3a24-43df-805c-9fffe7cd74f7 for instance with vm_state active and task_state deleting.
Aug 09 02:29:57 user nova-compute[72351]: DEBUG nova.network.neutron [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Successfully updated port: a88b872b-9340-4597-8ab4-ea6173dc4e4c {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:29:57 user nova-compute[72351]: DEBUG nova.compute.manager [req-c1a2ecd1-d65d-4673-9a56-18c0c08ea73e req-28233c11-2f41-4f55-bb33-0e269f87c833 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received event network-changed-ccb930f6-9625-450b-b01b-c5009e610ae1 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:29:57 user nova-compute[72351]: DEBUG nova.compute.manager [req-c1a2ecd1-d65d-4673-9a56-18c0c08ea73e req-28233c11-2f41-4f55-bb33-0e269f87c833 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Refreshing instance network info cache due to event network-changed-ccb930f6-9625-450b-b01b-c5009e610ae1. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:29:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c1a2ecd1-d65d-4673-9a56-18c0c08ea73e req-28233c11-2f41-4f55-bb33-0e269f87c833 service nova] Acquiring lock "refresh_cache-38a2b8e3-337d-416a-8a8c-da4921123db6" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:29:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c1a2ecd1-d65d-4673-9a56-18c0c08ea73e req-28233c11-2f41-4f55-bb33-0e269f87c833 service nova] Acquired lock "refresh_cache-38a2b8e3-337d-416a-8a8c-da4921123db6" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:29:57 user nova-compute[72351]: DEBUG nova.network.neutron [req-c1a2ecd1-d65d-4673-9a56-18c0c08ea73e req-28233c11-2f41-4f55-bb33-0e269f87c833 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Refreshing network info cache for port ccb930f6-9625-450b-b01b-c5009e610ae1 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:29:57 user nova-compute[72351]: DEBUG nova.network.neutron [req-c1a2ecd1-d65d-4673-9a56-18c0c08ea73e req-28233c11-2f41-4f55-bb33-0e269f87c833 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:29:57 user nova-compute[72351]: INFO nova.virt.block_device [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Booting with volume aa951fc7-aa57-4fc9-9433-de522d77537e at /dev/vdb
Aug 09 02:29:57 user nova-compute[72351]: DEBUG nova.network.neutron [req-c1a2ecd1-d65d-4673-9a56-18c0c08ea73e req-28233c11-2f41-4f55-bb33-0e269f87c833 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:29:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c1a2ecd1-d65d-4673-9a56-18c0c08ea73e req-28233c11-2f41-4f55-bb33-0e269f87c833 service nova] Releasing lock "refresh_cache-38a2b8e3-337d-416a-8a8c-da4921123db6" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:29:57 user nova-compute[72351]: DEBUG nova.compute.manager [req-c1a2ecd1-d65d-4673-9a56-18c0c08ea73e req-28233c11-2f41-4f55-bb33-0e269f87c833 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received event network-changed-d083d9f2-dfe4-4a73-a624-3336f2f02b0d {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:29:57 user nova-compute[72351]: DEBUG nova.compute.manager [req-c1a2ecd1-d65d-4673-9a56-18c0c08ea73e req-28233c11-2f41-4f55-bb33-0e269f87c833 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Refreshing instance network info cache due to event network-changed-d083d9f2-dfe4-4a73-a624-3336f2f02b0d. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:29:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c1a2ecd1-d65d-4673-9a56-18c0c08ea73e req-28233c11-2f41-4f55-bb33-0e269f87c833 service nova] Acquiring lock "refresh_cache-38a2b8e3-337d-416a-8a8c-da4921123db6" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:29:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c1a2ecd1-d65d-4673-9a56-18c0c08ea73e req-28233c11-2f41-4f55-bb33-0e269f87c833 service nova] Acquired lock "refresh_cache-38a2b8e3-337d-416a-8a8c-da4921123db6" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:29:57 user nova-compute[72351]: DEBUG nova.network.neutron [req-c1a2ecd1-d65d-4673-9a56-18c0c08ea73e req-28233c11-2f41-4f55-bb33-0e269f87c833 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Refreshing network info cache for port d083d9f2-dfe4-4a73-a624-3336f2f02b0d {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:29:57 user nova-compute[72351]: DEBUG os_brick.utils [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:29:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.013s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:57 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e93e1b78-6180-4327-8fb4-c96c1242fb1e]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:a52269602ff4\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.016s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:57 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a0596443-5c73-4543-ba2d-d593c7688bc8]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:57 user nova-compute[72351]: DEBUG nova.network.neutron [req-c1a2ecd1-d65d-4673-9a56-18c0c08ea73e req-28233c11-2f41-4f55-bb33-0e269f87c833 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:29:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:29:57 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d8af83fb-bb96-4fc9-bf82-8a16e61f34d9]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:57 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[87681b8d-0e7f-421e-9728-6fe33eeda0b8]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Running cmd (subprocess): nvme version {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] 'nvme version' failed. Not Retrying. {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:29:57 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.nvmeof [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] nvme not present on system {{(pid=72351) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Aug 09 02:29:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:29:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:29:57 user nova-compute[72351]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Aug 09 02:29:57 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e0a97480-25b8-4b3a-8557-60e7371bf040]: (4, '') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:29:57 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72351) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Aug 09 02:29:57 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Aug 09 02:29:57 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] LIGHTOS: no hostnqn found. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Aug 09 02:29:57 user nova-compute[72351]: DEBUG os_brick.utils [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] <== get_connector_properties: return (131ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:a52269602ff4', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:29:57 user nova-compute[72351]: DEBUG nova.virt.block_device [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Updating existing volume attachment record: 29eb6e27-816c-4661-999d-90058aeb0ea8 {{(pid=72351) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Aug 09 02:29:58 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:29:58 user nova-compute[72351]: DEBUG nova.network.neutron [req-c1a2ecd1-d65d-4673-9a56-18c0c08ea73e req-28233c11-2f41-4f55-bb33-0e269f87c833 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:29:58 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Took 1.27 seconds to deallocate network for instance.
Aug 09 02:29:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c1a2ecd1-d65d-4673-9a56-18c0c08ea73e req-28233c11-2f41-4f55-bb33-0e269f87c833 service nova] Releasing lock "refresh_cache-38a2b8e3-337d-416a-8a8c-da4921123db6" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:29:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-cf985c63-02a4-488a-80ca-b04fc6f71d3d tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:29:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-cf985c63-02a4-488a-80ca-b04fc6f71d3d tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:29:58 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-cf985c63-02a4-488a-80ca-b04fc6f71d3d tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:29:58 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-cf985c63-02a4-488a-80ca-b04fc6f71d3d tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:29:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-cf985c63-02a4-488a-80ca-b04fc6f71d3d tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.338s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:29:58 user nova-compute[72351]: DEBUG nova.network.neutron [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Successfully updated port: 48dfbf61-0142-4478-9ec1-585d5fbe11da {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:29:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Acquiring lock "refresh_cache-38a2b8e3-337d-416a-8a8c-da4921123db6" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:29:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Acquired lock "refresh_cache-38a2b8e3-337d-416a-8a8c-da4921123db6" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:29:58 user nova-compute[72351]: DEBUG nova.network.neutron [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:29:58 user nova-compute[72351]: INFO nova.scheduler.client.report [None req-cf985c63-02a4-488a-80ca-b04fc6f71d3d tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Deleted allocations for instance 16c1e558-baa1-4c2d-b13b-50585f74d587
Aug 09 02:29:58 user nova-compute[72351]: DEBUG nova.network.neutron [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:29:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-cf985c63-02a4-488a-80ca-b04fc6f71d3d tempest-TestServerVolumeAttachmentScenario-1098806376 tempest-TestServerVolumeAttachmentScenario-1098806376-project-member] Lock "16c1e558-baa1-4c2d-b13b-50585f74d587" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.693s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:29:59 user nova-compute[72351]: DEBUG nova.compute.manager [req-ec14565f-1d40-47a0-a08a-20a802f2bc08 req-e1fa3fe5-050c-40f3-8d4f-77f36cad9142 service nova] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Received event network-vif-deleted-fb1b25f9-3a24-43df-805c-9fffe7cd74f7 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:29:59 user nova-compute[72351]: DEBUG nova.compute.manager [req-42637536-2658-4adf-a94a-10bf03f845ab req-af14874c-0b0a-4ff3-b192-687716df8eab service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received event network-changed-a88b872b-9340-4597-8ab4-ea6173dc4e4c {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:29:59 user nova-compute[72351]: DEBUG nova.compute.manager [req-42637536-2658-4adf-a94a-10bf03f845ab req-af14874c-0b0a-4ff3-b192-687716df8eab service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Refreshing instance network info cache due to event network-changed-a88b872b-9340-4597-8ab4-ea6173dc4e4c. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:29:59 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-42637536-2658-4adf-a94a-10bf03f845ab req-af14874c-0b0a-4ff3-b192-687716df8eab service nova] Acquiring lock "refresh_cache-38a2b8e3-337d-416a-8a8c-da4921123db6" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:30:00 user nova-compute[72351]: DEBUG nova.network.neutron [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Updating instance_info_cache with network_info: [{"id": "0ebd83d0-5519-4945-8637-90c2ce024ac1", "address": "fa:16:3e:81:a2:62", "network": {"id": "c45d6719-95f9-4856-9b34-9d89c22ff899", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-927662772-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0ebd83d0-55", "ovs_interfaceid": "0ebd83d0-5519-4945-8637-90c2ce024ac1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "8b1245ae-85e9-49c3-9631-a288a172e4c1", "address": "fa:16:3e:a8:7b:e4", "network": {"id": "4617a3d0-dcb5-42a4-85d3-358087aeac69", "bridge": "br-int", "label": "tempest-device-tagging-net1-1844430962", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.34", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8b1245ae-85", "ovs_interfaceid": "8b1245ae-85e9-49c3-9631-a288a172e4c1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "1be27a59-07a4-4b6a-b25a-ce6d7efac4a3", "address": "fa:16:3e:38:62:75", "network": {"id": "4617a3d0-dcb5-42a4-85d3-358087aeac69", "bridge": "br-int", "label": "tempest-device-tagging-net1-1844430962", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.226", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1be27a59-07", "ovs_interfaceid": "1be27a59-07a4-4b6a-b25a-ce6d7efac4a3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "ccb930f6-9625-450b-b01b-c5009e610ae1", "address": "fa:16:3e:a9:0c:63", "network": {"id": "4617a3d0-dcb5-42a4-85d3-358087aeac69", "bridge": "br-int", "label": "tempest-device-tagging-net1-1844430962", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.237", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapccb930f6-96", "ovs_interfaceid": "ccb930f6-9625-450b-b01b-c5009e610ae1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "d083d9f2-dfe4-4a73-a624-3336f2f02b0d", "address": "fa:16:3e:a0:b1:db", "network": {"id": "4617a3d0-dcb5-42a4-85d3-358087aeac69", "bridge": "br-int", "label": "tempest-device-tagging-net1-1844430962", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.185", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd083d9f2-df", "ovs_interfaceid": "d083d9f2-dfe4-4a73-a624-3336f2f02b0d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "a88b872b-9340-4597-8ab4-ea6173dc4e4c", "address": "fa:16:3e:cf:3d:24", "network": {"id": "2984e5a6-dc0d-40ee-8064-f78a7686663e", "bridge": "br-int", "label": "tempest-device-tagging-net2-1630262622", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa88b872b-93", "ovs_interfaceid": "a88b872b-9340-4597-8ab4-ea6173dc4e4c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "48dfbf61-0142-4478-9ec1-585d5fbe11da", "address": "fa:16:3e:5b:d8:08", "network": {"id": "2984e5a6-dc0d-40ee-8064-f78a7686663e", "bridge": "br-int", "label": "tempest-device-tagging-net2-1630262622", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap48dfbf61-01", "ovs_interfaceid": "48dfbf61-0142-4478-9ec1-585d5fbe11da", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:30:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Releasing lock "refresh_cache-38a2b8e3-337d-416a-8a8c-da4921123db6" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:30:00 user nova-compute[72351]: DEBUG nova.compute.manager [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Instance network_info: |[{"id": "0ebd83d0-5519-4945-8637-90c2ce024ac1", "address": "fa:16:3e:81:a2:62", "network": {"id": "c45d6719-95f9-4856-9b34-9d89c22ff899", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-927662772-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0ebd83d0-55", "ovs_interfaceid": "0ebd83d0-5519-4945-8637-90c2ce024ac1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "8b1245ae-85e9-49c3-9631-a288a172e4c1", "address": "fa:16:3e:a8:7b:e4", "network": {"id": "4617a3d0-dcb5-42a4-85d3-358087aeac69", "bridge": "br-int", "label": "tempest-device-tagging-net1-1844430962", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.34", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8b1245ae-85", "ovs_interfaceid": "8b1245ae-85e9-49c3-9631-a288a172e4c1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "1be27a59-07a4-4b6a-b25a-ce6d7efac4a3", "address": "fa:16:3e:38:62:75", "network": {"id": "4617a3d0-dcb5-42a4-85d3-358087aeac69", "bridge": "br-int", "label": "tempest-device-tagging-net1-1844430962", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.226", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1be27a59-07", "ovs_interfaceid": "1be27a59-07a4-4b6a-b25a-ce6d7efac4a3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "ccb930f6-9625-450b-b01b-c5009e610ae1", "address": "fa:16:3e:a9:0c:63", "network": {"id": "4617a3d0-dcb5-42a4-85d3-358087aeac69", "bridge": "br-int", "label": "tempest-device-tagging-net1-1844430962", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.237", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapccb930f6-96", "ovs_interfaceid": "ccb930f6-9625-450b-b01b-c5009e610ae1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "d083d9f2-dfe4-4a73-a624-3336f2f02b0d", "address": "fa:16:3e:a0:b1:db", "network": {"id": "4617a3d0-dcb5-42a4-85d3-358087aeac69", "bridge": "br-int", "label": "tempest-device-tagging-net1-1844430962", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.185", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd083d9f2-df", "ovs_interfaceid": "d083d9f2-dfe4-4a73-a624-3336f2f02b0d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "a88b872b-9340-4597-8ab4-ea6173dc4e4c", "address": "fa:16:3e:cf:3d:24", "network": {"id": "2984e5a6-dc0d-40ee-8064-f78a7686663e", "bridge": "br-int", "label": "tempest-device-tagging-net2-1630262622", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa88b872b-93", "ovs_interfaceid": "a88b872b-9340-4597-8ab4-ea6173dc4e4c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "48dfbf61-0142-4478-9ec1-585d5fbe11da", "address": "fa:16:3e:5b:d8:08", "network": {"id": "2984e5a6-dc0d-40ee-8064-f78a7686663e", "bridge": "br-int", "label": "tempest-device-tagging-net2-1630262622", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap48dfbf61-01", "ovs_interfaceid": "48dfbf61-0142-4478-9ec1-585d5fbe11da", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1971}}
Aug 09 02:30:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-42637536-2658-4adf-a94a-10bf03f845ab req-af14874c-0b0a-4ff3-b192-687716df8eab service nova] Acquired lock "refresh_cache-38a2b8e3-337d-416a-8a8c-da4921123db6" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:30:00 user nova-compute[72351]: DEBUG nova.network.neutron [req-42637536-2658-4adf-a94a-10bf03f845ab req-af14874c-0b0a-4ff3-b192-687716df8eab service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Refreshing network info cache for port a88b872b-9340-4597-8ab4-ea6173dc4e4c {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:30:00 user nova-compute[72351]: DEBUG nova.network.neutron [req-42637536-2658-4adf-a94a-10bf03f845ab req-af14874c-0b0a-4ff3-b192-687716df8eab service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Updated VIF entry in instance network info cache for port a88b872b-9340-4597-8ab4-ea6173dc4e4c. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:30:00 user nova-compute[72351]: DEBUG nova.network.neutron [req-42637536-2658-4adf-a94a-10bf03f845ab req-af14874c-0b0a-4ff3-b192-687716df8eab service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Updating instance_info_cache with network_info: [{"id": "0ebd83d0-5519-4945-8637-90c2ce024ac1", "address": "fa:16:3e:81:a2:62", "network": {"id": "c45d6719-95f9-4856-9b34-9d89c22ff899", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-927662772-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0ebd83d0-55", "ovs_interfaceid": "0ebd83d0-5519-4945-8637-90c2ce024ac1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "8b1245ae-85e9-49c3-9631-a288a172e4c1", "address": "fa:16:3e:a8:7b:e4", "network": {"id": "4617a3d0-dcb5-42a4-85d3-358087aeac69", "bridge": "br-int", "label": "tempest-device-tagging-net1-1844430962", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.34", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8b1245ae-85", "ovs_interfaceid": "8b1245ae-85e9-49c3-9631-a288a172e4c1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "1be27a59-07a4-4b6a-b25a-ce6d7efac4a3", "address": "fa:16:3e:38:62:75", "network": {"id": "4617a3d0-dcb5-42a4-85d3-358087aeac69", "bridge": "br-int", "label": "tempest-device-tagging-net1-1844430962", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.226", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1be27a59-07", "ovs_interfaceid": "1be27a59-07a4-4b6a-b25a-ce6d7efac4a3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "ccb930f6-9625-450b-b01b-c5009e610ae1", "address": "fa:16:3e:a9:0c:63", "network": {"id": "4617a3d0-dcb5-42a4-85d3-358087aeac69", "bridge": "br-int", "label": "tempest-device-tagging-net1-1844430962", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.237", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapccb930f6-96", "ovs_interfaceid": "ccb930f6-9625-450b-b01b-c5009e610ae1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "d083d9f2-dfe4-4a73-a624-3336f2f02b0d", "address": "fa:16:3e:a0:b1:db", "network": {"id": "4617a3d0-dcb5-42a4-85d3-358087aeac69", "bridge": "br-int", "label": "tempest-device-tagging-net1-1844430962", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.185", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd083d9f2-df", "ovs_interfaceid": "d083d9f2-dfe4-4a73-a624-3336f2f02b0d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "a88b872b-9340-4597-8ab4-ea6173dc4e4c", "address": "fa:16:3e:cf:3d:24", "network": {"id": "2984e5a6-dc0d-40ee-8064-f78a7686663e", "bridge": "br-int", "label": "tempest-device-tagging-net2-1630262622", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa88b872b-93", "ovs_interfaceid": "a88b872b-9340-4597-8ab4-ea6173dc4e4c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "48dfbf61-0142-4478-9ec1-585d5fbe11da", "address": "fa:16:3e:5b:d8:08", "network": {"id": "2984e5a6-dc0d-40ee-8064-f78a7686663e", "bridge": "br-int", "label": "tempest-device-tagging-net2-1630262622", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap48dfbf61-01", "ovs_interfaceid": "48dfbf61-0142-4478-9ec1-585d5fbe11da", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:30:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-42637536-2658-4adf-a94a-10bf03f845ab req-af14874c-0b0a-4ff3-b192-687716df8eab service nova] Releasing lock "refresh_cache-38a2b8e3-337d-416a-8a8c-da4921123db6" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:30:00 user nova-compute[72351]: DEBUG nova.compute.manager [req-42637536-2658-4adf-a94a-10bf03f845ab req-af14874c-0b0a-4ff3-b192-687716df8eab service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received event network-changed-48dfbf61-0142-4478-9ec1-585d5fbe11da {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:30:00 user nova-compute[72351]: DEBUG nova.compute.manager [req-42637536-2658-4adf-a94a-10bf03f845ab req-af14874c-0b0a-4ff3-b192-687716df8eab service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Refreshing instance network info cache due to event network-changed-48dfbf61-0142-4478-9ec1-585d5fbe11da. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:30:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-42637536-2658-4adf-a94a-10bf03f845ab req-af14874c-0b0a-4ff3-b192-687716df8eab service nova] Acquiring lock "refresh_cache-38a2b8e3-337d-416a-8a8c-da4921123db6" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:30:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-42637536-2658-4adf-a94a-10bf03f845ab req-af14874c-0b0a-4ff3-b192-687716df8eab service nova] Acquired lock "refresh_cache-38a2b8e3-337d-416a-8a8c-da4921123db6" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:30:00 user nova-compute[72351]: DEBUG nova.network.neutron [req-42637536-2658-4adf-a94a-10bf03f845ab req-af14874c-0b0a-4ff3-b192-687716df8eab service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Refreshing network info cache for port 48dfbf61-0142-4478-9ec1-585d5fbe11da {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:30:00 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:30:01 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691548186.0380924, 2a3bcdea-be36-4a5b-93cb-53396efdfa8e => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:30:01 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] VM Stopped (Lifecycle Event)
Aug 09 02:30:01 user nova-compute[72351]: DEBUG nova.compute.manager [None req-1855abad-610c-4db2-9ed6-3532b19e3e40 None None] [instance: 2a3bcdea-be36-4a5b-93cb-53396efdfa8e] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:30:01 user nova-compute[72351]: DEBUG nova.network.neutron [req-42637536-2658-4adf-a94a-10bf03f845ab req-af14874c-0b0a-4ff3-b192-687716df8eab service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Updated VIF entry in instance network info cache for port 48dfbf61-0142-4478-9ec1-585d5fbe11da. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:30:01 user nova-compute[72351]: DEBUG nova.network.neutron [req-42637536-2658-4adf-a94a-10bf03f845ab req-af14874c-0b0a-4ff3-b192-687716df8eab service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Updating instance_info_cache with network_info: [{"id": "0ebd83d0-5519-4945-8637-90c2ce024ac1", "address": "fa:16:3e:81:a2:62", "network": {"id": "c45d6719-95f9-4856-9b34-9d89c22ff899", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-927662772-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0ebd83d0-55", "ovs_interfaceid": "0ebd83d0-5519-4945-8637-90c2ce024ac1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "8b1245ae-85e9-49c3-9631-a288a172e4c1", "address": "fa:16:3e:a8:7b:e4", "network": {"id": "4617a3d0-dcb5-42a4-85d3-358087aeac69", "bridge": "br-int", "label": "tempest-device-tagging-net1-1844430962", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.34", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8b1245ae-85", "ovs_interfaceid": "8b1245ae-85e9-49c3-9631-a288a172e4c1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "1be27a59-07a4-4b6a-b25a-ce6d7efac4a3", "address": "fa:16:3e:38:62:75", "network": {"id": "4617a3d0-dcb5-42a4-85d3-358087aeac69", "bridge": "br-int", "label": "tempest-device-tagging-net1-1844430962", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.226", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1be27a59-07", "ovs_interfaceid": "1be27a59-07a4-4b6a-b25a-ce6d7efac4a3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "ccb930f6-9625-450b-b01b-c5009e610ae1", "address": "fa:16:3e:a9:0c:63", "network": {"id": "4617a3d0-dcb5-42a4-85d3-358087aeac69", "bridge": "br-int", "label": "tempest-device-tagging-net1-1844430962", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.237", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapccb930f6-96", "ovs_interfaceid": "ccb930f6-9625-450b-b01b-c5009e610ae1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "d083d9f2-dfe4-4a73-a624-3336f2f02b0d", "address": "fa:16:3e:a0:b1:db", "network": {"id": "4617a3d0-dcb5-42a4-85d3-358087aeac69", "bridge": "br-int", "label": "tempest-device-tagging-net1-1844430962", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.185", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd083d9f2-df", "ovs_interfaceid": "d083d9f2-dfe4-4a73-a624-3336f2f02b0d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "a88b872b-9340-4597-8ab4-ea6173dc4e4c", "address": "fa:16:3e:cf:3d:24", "network": {"id": "2984e5a6-dc0d-40ee-8064-f78a7686663e", "bridge": "br-int", "label": "tempest-device-tagging-net2-1630262622", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa88b872b-93", "ovs_interfaceid": "a88b872b-9340-4597-8ab4-ea6173dc4e4c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "48dfbf61-0142-4478-9ec1-585d5fbe11da", "address": "fa:16:3e:5b:d8:08", "network": {"id": "2984e5a6-dc0d-40ee-8064-f78a7686663e", "bridge": "br-int", "label": "tempest-device-tagging-net2-1630262622", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap48dfbf61-01", "ovs_interfaceid": "48dfbf61-0142-4478-9ec1-585d5fbe11da", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:30:01 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-42637536-2658-4adf-a94a-10bf03f845ab req-af14874c-0b0a-4ff3-b192-687716df8eab service nova] Releasing lock "refresh_cache-38a2b8e3-337d-416a-8a8c-da4921123db6" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:30:01 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:01 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:30:01 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:30:01 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:30:01 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:30:01 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Auditing locally available compute resources for user (node: user) {{(pid=72351) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Aug 09 02:30:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json" returned: 0 in 0.142s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json" returned: 0 in 0.132s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:02 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdg (vda) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:30:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/957c1510-1320-4492-9052-a58ff540319d/disk.rescue --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/957c1510-1320-4492-9052-a58ff540319d/disk.rescue --force-share --output=json" returned: 0 in 0.178s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/957c1510-1320-4492-9052-a58ff540319d/disk.rescue --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/957c1510-1320-4492-9052-a58ff540319d/disk.rescue --force-share --output=json" returned: 0 in 0.136s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/47afce95-e5f7-4703-a99f-7ef8b28c4bd2/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/47afce95-e5f7-4703-a99f-7ef8b28c4bd2/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/47afce95-e5f7-4703-a99f-7ef8b28c4bd2/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/47afce95-e5f7-4703-a99f-7ef8b28c4bd2/disk --force-share --output=json" returned: 0 in 0.133s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.rescue --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.rescue --force-share --output=json" returned: 0 in 0.136s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.rescue --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.rescue --force-share --output=json" returned: 0 in 0.133s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json" returned: 0 in 0.146s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json" returned: 0 in 0.130s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json" returned: 0 in 0.130s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json" returned: 0 in 0.131s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a03fbded-23ad-440b-97e7-bea07425e873/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a03fbded-23ad-440b-97e7-bea07425e873/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a03fbded-23ad-440b-97e7-bea07425e873/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a03fbded-23ad-440b-97e7-bea07425e873/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9df16e46-d097-4553-b44c-732ea24a4169/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9df16e46-d097-4553-b44c-732ea24a4169/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9df16e46-d097-4553-b44c-732ea24a4169/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9df16e46-d097-4553-b44c-732ea24a4169/disk --force-share --output=json" returned: 0 in 0.128s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:05 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:30:05 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:30:05 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Hypervisor/Node resource view: name=user free_ram=7448MB free_disk=25.28199005126953GB free_vcpus=4 pci_devices=[{"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72351) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Aug 09 02:30:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:30:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:30:05 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:30:05 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance eb86cb49-2018-42c2-bd97-c662b829775d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:30:05 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:30:05 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance a7f3d076-864b-4b68-9207-214ce2c21444 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:30:05 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 47afce95-e5f7-4703-a99f-7ef8b28c4bd2 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:30:05 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 957c1510-1320-4492-9052-a58ff540319d actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:30:05 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance a03fbded-23ad-440b-97e7-bea07425e873 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:30:05 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 9df16e46-d097-4553-b44c-732ea24a4169 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:30:05 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 38a2b8e3-337d-416a-8a8c-da4921123db6 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:30:05 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Total usable vcpus: 12, total allocated vcpus: 9 {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Aug 09 02:30:05 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Final resource view: name=user phys_ram=16011MB used_ram=5120MB phys_disk=40GB used_disk=7GB total_vcpus=12 used_vcpus=9 pci_stats=[] {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Aug 09 02:30:05 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:30:05 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:30:05 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Compute_service record updated for user:user {{(pid=72351) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Aug 09 02:30:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.430s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:30:06 user nova-compute[72351]: INFO nova.virt.block_device [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Booting with volume 5aa01207-8de0-4c53-9662-6771d109fe70 at /dev/vdc
Aug 09 02:30:06 user nova-compute[72351]: DEBUG os_brick.utils [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:30:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.013s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:06 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[b5371ce4-5335-40a3-9e36-3519152976ca]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:a52269602ff4\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.016s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:06 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[f2ea22ae-ff1f-4e38-b70b-a13d7cced610]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.016s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:06 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[3c01a73a-73a1-4f72-82cf-b7ba62007414]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:06 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[2b587dcc-9fbc-4759-8d36-3fcf5f4bc00a]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Running cmd (subprocess): nvme version {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] 'nvme version' failed. Not Retrying. {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:30:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.nvmeof [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] nvme not present on system {{(pid=72351) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Aug 09 02:30:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:30:06 user nova-compute[72351]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Aug 09 02:30:06 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[085d7553-b0df-4f0d-b59f-3f9859bed366]: (4, '') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72351) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Aug 09 02:30:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Aug 09 02:30:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] LIGHTOS: no hostnqn found. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Aug 09 02:30:06 user nova-compute[72351]: DEBUG os_brick.utils [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] <== get_connector_properties: return (120ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:a52269602ff4', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:30:06 user nova-compute[72351]: DEBUG nova.virt.block_device [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Updating existing volume attachment record: 545efb74-b5df-452f-b28e-17f3b3bdde89 {{(pid=72351) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Aug 09 02:30:06 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:30:06 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Starting heal instance info cache {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9842}}
Aug 09 02:30:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "refresh_cache-e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:30:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquired lock "refresh_cache-e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:30:06 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Forcefully refreshing network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1996}}
Aug 09 02:30:06 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:07 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Updating instance_info_cache with network_info: [{"id": "9481e457-9c23-4c45-b0bd-1faf63f8b7b8", "address": "fa:16:3e:69:03:30", "network": {"id": "49e8751c-e119-430f-9110-3dc6069b1589", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-899275135-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3aea1e66805146f7a8cd1d97e64a24af", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9481e457-9c", "ovs_interfaceid": "9481e457-9c23-4c45-b0bd-1faf63f8b7b8", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:30:07 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Releasing lock "refresh_cache-e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:30:07 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Updated the network info_cache for instance {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9913}}
Aug 09 02:30:07 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:30:07 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:30:07 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72351) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10461}}
Aug 09 02:30:07 user nova-compute[72351]: INFO nova.compute.manager [None req-5dcb163a-4a97-469e-99af-b42e44247955 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 957c1510-1320-4492-9052-a58ff540319d] Unrescuing
Aug 09 02:30:07 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5dcb163a-4a97-469e-99af-b42e44247955 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Acquiring lock "refresh_cache-957c1510-1320-4492-9052-a58ff540319d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:30:07 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5dcb163a-4a97-469e-99af-b42e44247955 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Acquired lock "refresh_cache-957c1510-1320-4492-9052-a58ff540319d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:30:07 user nova-compute[72351]: DEBUG nova.network.neutron [None req-5dcb163a-4a97-469e-99af-b42e44247955 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 957c1510-1320-4492-9052-a58ff540319d] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:30:07 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:30:07 user nova-compute[72351]: DEBUG nova.network.neutron [None req-5dcb163a-4a97-469e-99af-b42e44247955 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 957c1510-1320-4492-9052-a58ff540319d] Updating instance_info_cache with network_info: [{"id": "e679cfd9-123f-4b88-8fd2-397e4c8616cc", "address": "fa:16:3e:7f:d6:1b", "network": {"id": "9a452414-d61f-4045-a6cc-9be93c892c4c", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "321111f935dd4e8aab4a083206ea9a7e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape679cfd9-12", "ovs_interfaceid": "e679cfd9-123f-4b88-8fd2-397e4c8616cc", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:30:07 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5dcb163a-4a97-469e-99af-b42e44247955 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Releasing lock "refresh_cache-957c1510-1320-4492-9052-a58ff540319d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:30:07 user nova-compute[72351]: DEBUG nova.objects.instance [None req-5dcb163a-4a97-469e-99af-b42e44247955 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lazy-loading 'flavor' on Instance uuid 957c1510-1320-4492-9052-a58ff540319d {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:30:07 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:30:07 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:30:07 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:30:07 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:07 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:08 user nova-compute[72351]: DEBUG nova.compute.manager [req-9a783b32-6f38-4033-a44e-a3767fe2bf58 req-7999d8cd-5667-4188-8ef5-06a0b25d4d11 service nova] [instance: 957c1510-1320-4492-9052-a58ff540319d] Received event network-vif-unplugged-e679cfd9-123f-4b88-8fd2-397e4c8616cc {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:30:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-9a783b32-6f38-4033-a44e-a3767fe2bf58 req-7999d8cd-5667-4188-8ef5-06a0b25d4d11 service nova] Acquiring lock "957c1510-1320-4492-9052-a58ff540319d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:30:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-9a783b32-6f38-4033-a44e-a3767fe2bf58 req-7999d8cd-5667-4188-8ef5-06a0b25d4d11 service nova] Lock "957c1510-1320-4492-9052-a58ff540319d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:30:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-9a783b32-6f38-4033-a44e-a3767fe2bf58 req-7999d8cd-5667-4188-8ef5-06a0b25d4d11 service nova] Lock "957c1510-1320-4492-9052-a58ff540319d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:30:08 user nova-compute[72351]: DEBUG nova.compute.manager [req-9a783b32-6f38-4033-a44e-a3767fe2bf58 req-7999d8cd-5667-4188-8ef5-06a0b25d4d11 service nova] [instance: 957c1510-1320-4492-9052-a58ff540319d] No waiting events found dispatching network-vif-unplugged-e679cfd9-123f-4b88-8fd2-397e4c8616cc {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:30:08 user nova-compute[72351]: WARNING nova.compute.manager [req-9a783b32-6f38-4033-a44e-a3767fe2bf58 req-7999d8cd-5667-4188-8ef5-06a0b25d4d11 service nova] [instance: 957c1510-1320-4492-9052-a58ff540319d] Received unexpected event network-vif-unplugged-e679cfd9-123f-4b88-8fd2-397e4c8616cc for instance with vm_state rescued and task_state unrescuing.
Aug 09 02:30:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:08 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691548193.3794587, 75bbbead-8628-448b-9484-19bf4ab922ff => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:30:08 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] VM Stopped (Lifecycle Event)
Aug 09 02:30:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:08 user nova-compute[72351]: DEBUG nova.compute.manager [None req-018c2009-775a-441a-92c8-7e0295bceee5 None None] [instance: 75bbbead-8628-448b-9484-19bf4ab922ff] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:30:08 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 957c1510-1320-4492-9052-a58ff540319d] Instance destroyed successfully.
Aug 09 02:30:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:09 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Acquiring lock "7b6f5d60-5f23-4ef1-b972-9ba7086f56cb" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:30:09 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "7b6f5d60-5f23-4ef1-b972-9ba7086f56cb" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:30:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:09 user nova-compute[72351]: DEBUG nova.compute.manager [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Starting instance... {{(pid=72351) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2406}}
Aug 09 02:30:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:30:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:30:10 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72351) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Aug 09 02:30:10 user nova-compute[72351]: INFO nova.compute.claims [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Claim successful on node user
Aug 09 02:30:10 user nova-compute[72351]: DEBUG nova.compute.manager [req-d2ee577f-d2f0-4d7c-b040-40b5ac36212b req-d82388e8-fafa-4efa-8c39-4002ee469f28 service nova] [instance: 957c1510-1320-4492-9052-a58ff540319d] Received event network-vif-plugged-e679cfd9-123f-4b88-8fd2-397e4c8616cc {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:30:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d2ee577f-d2f0-4d7c-b040-40b5ac36212b req-d82388e8-fafa-4efa-8c39-4002ee469f28 service nova] Acquiring lock "957c1510-1320-4492-9052-a58ff540319d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:30:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d2ee577f-d2f0-4d7c-b040-40b5ac36212b req-d82388e8-fafa-4efa-8c39-4002ee469f28 service nova] Lock "957c1510-1320-4492-9052-a58ff540319d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:30:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d2ee577f-d2f0-4d7c-b040-40b5ac36212b req-d82388e8-fafa-4efa-8c39-4002ee469f28 service nova] Lock "957c1510-1320-4492-9052-a58ff540319d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:30:10 user nova-compute[72351]: DEBUG nova.compute.manager [req-d2ee577f-d2f0-4d7c-b040-40b5ac36212b req-d82388e8-fafa-4efa-8c39-4002ee469f28 service nova] [instance: 957c1510-1320-4492-9052-a58ff540319d] No waiting events found dispatching network-vif-plugged-e679cfd9-123f-4b88-8fd2-397e4c8616cc {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:30:10 user nova-compute[72351]: WARNING nova.compute.manager [req-d2ee577f-d2f0-4d7c-b040-40b5ac36212b req-d82388e8-fafa-4efa-8c39-4002ee469f28 service nova] [instance: 957c1510-1320-4492-9052-a58ff540319d] Received unexpected event network-vif-plugged-e679cfd9-123f-4b88-8fd2-397e4c8616cc for instance with vm_state rescued and task_state unrescuing.
Aug 09 02:30:10 user nova-compute[72351]: DEBUG nova.compute.manager [req-d2ee577f-d2f0-4d7c-b040-40b5ac36212b req-d82388e8-fafa-4efa-8c39-4002ee469f28 service nova] [instance: 957c1510-1320-4492-9052-a58ff540319d] Received event network-vif-plugged-e679cfd9-123f-4b88-8fd2-397e4c8616cc {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:30:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d2ee577f-d2f0-4d7c-b040-40b5ac36212b req-d82388e8-fafa-4efa-8c39-4002ee469f28 service nova] Acquiring lock "957c1510-1320-4492-9052-a58ff540319d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:30:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d2ee577f-d2f0-4d7c-b040-40b5ac36212b req-d82388e8-fafa-4efa-8c39-4002ee469f28 service nova] Lock "957c1510-1320-4492-9052-a58ff540319d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:30:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d2ee577f-d2f0-4d7c-b040-40b5ac36212b req-d82388e8-fafa-4efa-8c39-4002ee469f28 service nova] Lock "957c1510-1320-4492-9052-a58ff540319d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:30:10 user nova-compute[72351]: DEBUG nova.compute.manager [req-d2ee577f-d2f0-4d7c-b040-40b5ac36212b req-d82388e8-fafa-4efa-8c39-4002ee469f28 service nova] [instance: 957c1510-1320-4492-9052-a58ff540319d] No waiting events found dispatching network-vif-plugged-e679cfd9-123f-4b88-8fd2-397e4c8616cc {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:30:10 user nova-compute[72351]: WARNING nova.compute.manager [req-d2ee577f-d2f0-4d7c-b040-40b5ac36212b req-d82388e8-fafa-4efa-8c39-4002ee469f28 service nova] [instance: 957c1510-1320-4492-9052-a58ff540319d] Received unexpected event network-vif-plugged-e679cfd9-123f-4b88-8fd2-397e4c8616cc for instance with vm_state rescued and task_state unrescuing.
Aug 09 02:30:10 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:10 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:10 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:30:10 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:30:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.483s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:30:10 user nova-compute[72351]: DEBUG nova.compute.manager [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Start building networks asynchronously for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2803}}
Aug 09 02:30:10 user nova-compute[72351]: DEBUG nova.compute.manager [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Allocating IP information in the background. {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1956}}
Aug 09 02:30:10 user nova-compute[72351]: DEBUG nova.network.neutron [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] allocate_for_instance() {{(pid=72351) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Aug 09 02:30:10 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Aug 09 02:30:10 user nova-compute[72351]: DEBUG nova.compute.manager [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Start building block device mappings for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2838}}
Aug 09 02:30:10 user nova-compute[72351]: DEBUG nova.policy [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '419b28d5a46c481db435715a30ab2614', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3aea1e66805146f7a8cd1d97e64a24af', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72351) authorize /opt/stack/nova/nova/policy.py:203}}
Aug 09 02:30:10 user nova-compute[72351]: DEBUG nova.compute.manager [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Start spawning the instance on the hypervisor. {{(pid=72351) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2612}}
Aug 09 02:30:10 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Creating instance directory {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4717}}
Aug 09 02:30:10 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Creating image(s)
Aug 09 02:30:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Acquiring lock "/opt/stack/data/nova/instances/7b6f5d60-5f23-4ef1-b972-9ba7086f56cb/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:30:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "/opt/stack/data/nova/instances/7b6f5d60-5f23-4ef1-b972-9ba7086f56cb/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:30:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "/opt/stack/data/nova/instances/7b6f5d60-5f23-4ef1-b972-9ba7086f56cb/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.005s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:30:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.138s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Acquiring lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:30:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:30:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.138s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/7b6f5d60-5f23-4ef1-b972-9ba7086f56cb/disk 1073741824 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/7b6f5d60-5f23-4ef1-b972-9ba7086f56cb/disk 1073741824" returned: 0 in 0.053s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.197s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:30:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.178s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:11 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Checking if we can resize image /opt/stack/data/nova/instances/7b6f5d60-5f23-4ef1-b972-9ba7086f56cb/disk. size=1073741824 {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Aug 09 02:30:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7b6f5d60-5f23-4ef1-b972-9ba7086f56cb/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:11 user nova-compute[72351]: DEBUG nova.network.neutron [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Successfully created port: c4a56310-1841-46ae-9a97-0421861e62c0 {{(pid=72351) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Aug 09 02:30:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7b6f5d60-5f23-4ef1-b972-9ba7086f56cb/disk --force-share --output=json" returned: 0 in 0.143s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:11 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Cannot resize image /opt/stack/data/nova/instances/7b6f5d60-5f23-4ef1-b972-9ba7086f56cb/disk to a smaller size. {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Aug 09 02:30:11 user nova-compute[72351]: DEBUG nova.objects.instance [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lazy-loading 'migration_context' on Instance uuid 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:30:11 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Created local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4851}}
Aug 09 02:30:11 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Ensure instance console log exists: /opt/stack/data/nova/instances/7b6f5d60-5f23-4ef1-b972-9ba7086f56cb/console.log {{(pid=72351) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4603}}
Aug 09 02:30:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:30:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:30:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:30:11 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691548196.7527208, 16c1e558-baa1-4c2d-b13b-50585f74d587 => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:30:11 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] VM Stopped (Lifecycle Event)
Aug 09 02:30:11 user nova-compute[72351]: DEBUG nova.compute.manager [None req-1c41a54a-30b5-491c-885a-594f49e4c744 None None] [instance: 16c1e558-baa1-4c2d-b13b-50585f74d587] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:30:11 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:12 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Removed pending event for 957c1510-1320-4492-9052-a58ff540319d due to event {{(pid=72351) _event_emit_delayed /opt/stack/nova/nova/virt/libvirt/host.py:438}}
Aug 09 02:30:12 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548212.1067975, 957c1510-1320-4492-9052-a58ff540319d => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:30:12 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 957c1510-1320-4492-9052-a58ff540319d] VM Resumed (Lifecycle Event)
Aug 09 02:30:12 user nova-compute[72351]: DEBUG nova.compute.manager [None req-5dcb163a-4a97-469e-99af-b42e44247955 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 957c1510-1320-4492-9052-a58ff540319d] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:30:12 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 957c1510-1320-4492-9052-a58ff540319d] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:30:12 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 957c1510-1320-4492-9052-a58ff540319d] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: rescued, current task_state: unrescuing, current DB power_state: 1, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:30:12 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 957c1510-1320-4492-9052-a58ff540319d] During sync_power_state the instance has a pending task (unrescuing). Skip.
Aug 09 02:30:12 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548212.1075518, 957c1510-1320-4492-9052-a58ff540319d => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:30:12 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 957c1510-1320-4492-9052-a58ff540319d] VM Started (Lifecycle Event)
Aug 09 02:30:12 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 957c1510-1320-4492-9052-a58ff540319d] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:30:12 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 957c1510-1320-4492-9052-a58ff540319d] Synchronizing instance power state after lifecycle event "Started"; current vm_state: active, current task_state: None, current DB power_state: 1, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:30:12 user nova-compute[72351]: DEBUG nova.compute.manager [req-08b3c4fe-057d-4e7a-8e7a-737ddcfabf4b req-85652fa6-6c8f-4b5f-a00c-cdbf470077e3 service nova] [instance: 957c1510-1320-4492-9052-a58ff540319d] Received event network-vif-plugged-e679cfd9-123f-4b88-8fd2-397e4c8616cc {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:30:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-08b3c4fe-057d-4e7a-8e7a-737ddcfabf4b req-85652fa6-6c8f-4b5f-a00c-cdbf470077e3 service nova] Acquiring lock "957c1510-1320-4492-9052-a58ff540319d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:30:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-08b3c4fe-057d-4e7a-8e7a-737ddcfabf4b req-85652fa6-6c8f-4b5f-a00c-cdbf470077e3 service nova] Lock "957c1510-1320-4492-9052-a58ff540319d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:30:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-08b3c4fe-057d-4e7a-8e7a-737ddcfabf4b req-85652fa6-6c8f-4b5f-a00c-cdbf470077e3 service nova] Lock "957c1510-1320-4492-9052-a58ff540319d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:30:12 user nova-compute[72351]: DEBUG nova.compute.manager [req-08b3c4fe-057d-4e7a-8e7a-737ddcfabf4b req-85652fa6-6c8f-4b5f-a00c-cdbf470077e3 service nova] [instance: 957c1510-1320-4492-9052-a58ff540319d] No waiting events found dispatching network-vif-plugged-e679cfd9-123f-4b88-8fd2-397e4c8616cc {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:30:12 user nova-compute[72351]: WARNING nova.compute.manager [req-08b3c4fe-057d-4e7a-8e7a-737ddcfabf4b req-85652fa6-6c8f-4b5f-a00c-cdbf470077e3 service nova] [instance: 957c1510-1320-4492-9052-a58ff540319d] Received unexpected event network-vif-plugged-e679cfd9-123f-4b88-8fd2-397e4c8616cc for instance with vm_state active and task_state None.
Aug 09 02:30:12 user nova-compute[72351]: DEBUG nova.network.neutron [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Successfully updated port: c4a56310-1841-46ae-9a97-0421861e62c0 {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:30:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Acquiring lock "refresh_cache-7b6f5d60-5f23-4ef1-b972-9ba7086f56cb" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:30:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Acquired lock "refresh_cache-7b6f5d60-5f23-4ef1-b972-9ba7086f56cb" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:30:12 user nova-compute[72351]: DEBUG nova.network.neutron [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:30:12 user nova-compute[72351]: DEBUG nova.network.neutron [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:30:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:13 user nova-compute[72351]: DEBUG nova.network.neutron [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Updating instance_info_cache with network_info: [{"id": "c4a56310-1841-46ae-9a97-0421861e62c0", "address": "fa:16:3e:75:cf:ba", "network": {"id": "49e8751c-e119-430f-9110-3dc6069b1589", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-899275135-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3aea1e66805146f7a8cd1d97e64a24af", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc4a56310-18", "ovs_interfaceid": "c4a56310-1841-46ae-9a97-0421861e62c0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:30:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Releasing lock "refresh_cache-7b6f5d60-5f23-4ef1-b972-9ba7086f56cb" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:30:13 user nova-compute[72351]: DEBUG nova.compute.manager [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Instance network_info: |[{"id": "c4a56310-1841-46ae-9a97-0421861e62c0", "address": "fa:16:3e:75:cf:ba", "network": {"id": "49e8751c-e119-430f-9110-3dc6069b1589", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-899275135-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3aea1e66805146f7a8cd1d97e64a24af", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc4a56310-18", "ovs_interfaceid": "c4a56310-1841-46ae-9a97-0421861e62c0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1971}}
Aug 09 02:30:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Start _get_guest_xml network_info=[{"id": "c4a56310-1841-46ae-9a97-0421861e62c0", "address": "fa:16:3e:75:cf:ba", "network": {"id": "49e8751c-e119-430f-9110-3dc6069b1589", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-899275135-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3aea1e66805146f7a8cd1d97e64a24af", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc4a56310-18", "ovs_interfaceid": "c4a56310-1841-46ae-9a97-0421861e62c0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'size': 0, 'device_type': 'disk', 'encryption_secret_uuid': None, 'guest_format': None, 'encryption_options': None, 'disk_bus': 'virtio', 'encrypted': False, 'encryption_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'image_id': '18ccda7f-31d3-4978-ab9d-4ed2f74eff94'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:30:13 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:30:13 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:30:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:30:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:30:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:30:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:30:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:30:13 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:30:13 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:30:13 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:30:13 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:30:13 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:30:13 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:30:13 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:30:13 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:30:13 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:30:13 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:30:13 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:30:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:30:09Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerRescueNegativeTestJSON-server-2047297087',display_name='tempest-ServerRescueNegativeTestJSON-server-2047297087',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serverrescuenegativetestjson-server-2047297087',id=34,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBPLsNIutXj2fb507aP4WngpUN1jl7pet8vFZkz28Sl/BaBQhyC41kBOXIZAx0v3md9QJVdNaIzkTSV1yroT5SZJzTuTaLLebnfImCGnkD5mtQK+8liTnNCRPuxSfIElfWA==',key_name='tempest-keypair-694850304',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='3aea1e66805146f7a8cd1d97e64a24af',ramdisk_id='',reservation_id='r-02psy90j',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerRescueNegativeTestJSON-289364438',owner_user_name='tempest-ServerRescueNegativeTestJSON-289364438-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:30:11Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='419b28d5a46c481db435715a30ab2614',uuid=7b6f5d60-5f23-4ef1-b972-9ba7086f56cb,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "c4a56310-1841-46ae-9a97-0421861e62c0", "address": "fa:16:3e:75:cf:ba", "network": {"id": "49e8751c-e119-430f-9110-3dc6069b1589", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-899275135-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3aea1e66805146f7a8cd1d97e64a24af", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc4a56310-18", "ovs_interfaceid": "c4a56310-1841-46ae-9a97-0421861e62c0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:30:13 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Converting VIF {"id": "c4a56310-1841-46ae-9a97-0421861e62c0", "address": "fa:16:3e:75:cf:ba", "network": {"id": "49e8751c-e119-430f-9110-3dc6069b1589", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-899275135-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3aea1e66805146f7a8cd1d97e64a24af", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc4a56310-18", "ovs_interfaceid": "c4a56310-1841-46ae-9a97-0421861e62c0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:30:13 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:75:cf:ba,bridge_name='br-int',has_traffic_filtering=True,id=c4a56310-1841-46ae-9a97-0421861e62c0,network=Network(49e8751c-e119-430f-9110-3dc6069b1589),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc4a56310-18') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:30:13 user nova-compute[72351]: DEBUG nova.objects.instance [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lazy-loading 'pci_devices' on Instance uuid 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:30:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:30:13 user nova-compute[72351]:   <uuid>7b6f5d60-5f23-4ef1-b972-9ba7086f56cb</uuid>
Aug 09 02:30:13 user nova-compute[72351]:   <name>instance-00000022</name>
Aug 09 02:30:13 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:30:13 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:30:13 user nova-compute[72351]:   <metadata>
Aug 09 02:30:13 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:30:13 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:30:13 user nova-compute[72351]:       <nova:name>tempest-ServerRescueNegativeTestJSON-server-2047297087</nova:name>
Aug 09 02:30:13 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:30:13</nova:creationTime>
Aug 09 02:30:13 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:30:13 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:30:13 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:30:13 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:30:13 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:30:13 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:30:13 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:30:13 user nova-compute[72351]:       <nova:owner>
Aug 09 02:30:13 user nova-compute[72351]:         <nova:user uuid="419b28d5a46c481db435715a30ab2614">tempest-ServerRescueNegativeTestJSON-289364438-project-member</nova:user>
Aug 09 02:30:13 user nova-compute[72351]:         <nova:project uuid="3aea1e66805146f7a8cd1d97e64a24af">tempest-ServerRescueNegativeTestJSON-289364438</nova:project>
Aug 09 02:30:13 user nova-compute[72351]:       </nova:owner>
Aug 09 02:30:13 user nova-compute[72351]:       <nova:root type="image" uuid="18ccda7f-31d3-4978-ab9d-4ed2f74eff94"/>
Aug 09 02:30:13 user nova-compute[72351]:       <nova:ports>
Aug 09 02:30:13 user nova-compute[72351]:         <nova:port uuid="c4a56310-1841-46ae-9a97-0421861e62c0">
Aug 09 02:30:13 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.7" ipVersion="4"/>
Aug 09 02:30:13 user nova-compute[72351]:         </nova:port>
Aug 09 02:30:13 user nova-compute[72351]:       </nova:ports>
Aug 09 02:30:13 user nova-compute[72351]:     </nova:instance>
Aug 09 02:30:13 user nova-compute[72351]:   </metadata>
Aug 09 02:30:13 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:30:13 user nova-compute[72351]:     <system>
Aug 09 02:30:13 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:30:13 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:30:13 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:30:13 user nova-compute[72351]:       <entry name="serial">7b6f5d60-5f23-4ef1-b972-9ba7086f56cb</entry>
Aug 09 02:30:13 user nova-compute[72351]:       <entry name="uuid">7b6f5d60-5f23-4ef1-b972-9ba7086f56cb</entry>
Aug 09 02:30:13 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:30:13 user nova-compute[72351]:     </system>
Aug 09 02:30:13 user nova-compute[72351]:   </sysinfo>
Aug 09 02:30:13 user nova-compute[72351]:   <os>
Aug 09 02:30:13 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:30:13 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:30:13 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:30:13 user nova-compute[72351]:   </os>
Aug 09 02:30:13 user nova-compute[72351]:   <features>
Aug 09 02:30:13 user nova-compute[72351]:     <acpi/>
Aug 09 02:30:13 user nova-compute[72351]:     <apic/>
Aug 09 02:30:13 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:30:13 user nova-compute[72351]:   </features>
Aug 09 02:30:13 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:30:13 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:30:13 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:30:13 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:30:13 user nova-compute[72351]:   </clock>
Aug 09 02:30:13 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:30:13 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:30:13 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:30:13 user nova-compute[72351]:   </cpu>
Aug 09 02:30:13 user nova-compute[72351]:   <devices>
Aug 09 02:30:13 user nova-compute[72351]:     <disk type="file" device="disk">
Aug 09 02:30:13 user nova-compute[72351]:       <driver name="qemu" type="qcow2" cache="none"/>
Aug 09 02:30:13 user nova-compute[72351]:       <source file="/opt/stack/data/nova/instances/7b6f5d60-5f23-4ef1-b972-9ba7086f56cb/disk"/>
Aug 09 02:30:13 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:30:13 user nova-compute[72351]:     </disk>
Aug 09 02:30:13 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:30:13 user nova-compute[72351]:       <mac address="fa:16:3e:75:cf:ba"/>
Aug 09 02:30:13 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:30:13 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:30:13 user nova-compute[72351]:       <target dev="tapc4a56310-18"/>
Aug 09 02:30:13 user nova-compute[72351]:     </interface>
Aug 09 02:30:13 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:30:13 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/7b6f5d60-5f23-4ef1-b972-9ba7086f56cb/console.log" append="off"/>
Aug 09 02:30:13 user nova-compute[72351]:     </serial>
Aug 09 02:30:13 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:30:13 user nova-compute[72351]:     <video>
Aug 09 02:30:13 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:30:13 user nova-compute[72351]:     </video>
Aug 09 02:30:13 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:30:13 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:30:13 user nova-compute[72351]:     </rng>
Aug 09 02:30:13 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:30:13 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:30:13 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:30:13 user nova-compute[72351]:     </memballoon>
Aug 09 02:30:13 user nova-compute[72351]:   </devices>
Aug 09 02:30:13 user nova-compute[72351]: </domain>
Aug 09 02:30:13 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:30:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:30:09Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerRescueNegativeTestJSON-server-2047297087',display_name='tempest-ServerRescueNegativeTestJSON-server-2047297087',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serverrescuenegativetestjson-server-2047297087',id=34,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBPLsNIutXj2fb507aP4WngpUN1jl7pet8vFZkz28Sl/BaBQhyC41kBOXIZAx0v3md9QJVdNaIzkTSV1yroT5SZJzTuTaLLebnfImCGnkD5mtQK+8liTnNCRPuxSfIElfWA==',key_name='tempest-keypair-694850304',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='3aea1e66805146f7a8cd1d97e64a24af',ramdisk_id='',reservation_id='r-02psy90j',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerRescueNegativeTestJSON-289364438',owner_user_name='tempest-ServerRescueNegativeTestJSON-289364438-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:30:11Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='419b28d5a46c481db435715a30ab2614',uuid=7b6f5d60-5f23-4ef1-b972-9ba7086f56cb,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "c4a56310-1841-46ae-9a97-0421861e62c0", "address": "fa:16:3e:75:cf:ba", "network": {"id": "49e8751c-e119-430f-9110-3dc6069b1589", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-899275135-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3aea1e66805146f7a8cd1d97e64a24af", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc4a56310-18", "ovs_interfaceid": "c4a56310-1841-46ae-9a97-0421861e62c0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:30:13 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Converting VIF {"id": "c4a56310-1841-46ae-9a97-0421861e62c0", "address": "fa:16:3e:75:cf:ba", "network": {"id": "49e8751c-e119-430f-9110-3dc6069b1589", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-899275135-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3aea1e66805146f7a8cd1d97e64a24af", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc4a56310-18", "ovs_interfaceid": "c4a56310-1841-46ae-9a97-0421861e62c0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:30:13 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:75:cf:ba,bridge_name='br-int',has_traffic_filtering=True,id=c4a56310-1841-46ae-9a97-0421861e62c0,network=Network(49e8751c-e119-430f-9110-3dc6069b1589),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc4a56310-18') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:30:13 user nova-compute[72351]: DEBUG os_vif [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:75:cf:ba,bridge_name='br-int',has_traffic_filtering=True,id=c4a56310-1841-46ae-9a97-0421861e62c0,network=Network(49e8751c-e119-430f-9110-3dc6069b1589),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc4a56310-18') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:30:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:30:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:30:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapc4a56310-18, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:30:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapc4a56310-18, col_values=(('external_ids', {'iface-id': 'c4a56310-1841-46ae-9a97-0421861e62c0', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:75:cf:ba', 'vm-uuid': '7b6f5d60-5f23-4ef1-b972-9ba7086f56cb'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:30:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:30:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:13 user nova-compute[72351]: INFO os_vif [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:75:cf:ba,bridge_name='br-int',has_traffic_filtering=True,id=c4a56310-1841-46ae-9a97-0421861e62c0,network=Network(49e8751c-e119-430f-9110-3dc6069b1589),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc4a56310-18')
Aug 09 02:30:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:30:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] No VIF found with MAC fa:16:3e:75:cf:ba, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:30:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-dab285c2-8b0d-46ad-8db3-440348d0fd27 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Acquiring lock "9df16e46-d097-4553-b44c-732ea24a4169" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:30:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-dab285c2-8b0d-46ad-8db3-440348d0fd27 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lock "9df16e46-d097-4553-b44c-732ea24a4169" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:30:13 user nova-compute[72351]: DEBUG nova.objects.instance [None req-dab285c2-8b0d-46ad-8db3-440348d0fd27 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lazy-loading 'flavor' on Instance uuid 9df16e46-d097-4553-b44c-732ea24a4169 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:30:13 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-dab285c2-8b0d-46ad-8db3-440348d0fd27 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Ignoring supplied device name: /dev/vdb
Aug 09 02:30:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-dab285c2-8b0d-46ad-8db3-440348d0fd27 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lock "9df16e46-d097-4553-b44c-732ea24a4169" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.057s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:30:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-dab285c2-8b0d-46ad-8db3-440348d0fd27 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Acquiring lock "9df16e46-d097-4553-b44c-732ea24a4169" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:30:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-dab285c2-8b0d-46ad-8db3-440348d0fd27 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lock "9df16e46-d097-4553-b44c-732ea24a4169" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:30:13 user nova-compute[72351]: INFO nova.compute.manager [None req-dab285c2-8b0d-46ad-8db3-440348d0fd27 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Attaching volume 3cb561e0-d189-4a63-abe1-3a68ffbe4616 to /dev/vdb
Aug 09 02:30:13 user nova-compute[72351]: DEBUG os_brick.utils [None req-dab285c2-8b0d-46ad-8db3-440348d0fd27 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:30:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.015s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:13 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[adec534c-a2c3-454b-8593-b558dc68b7f8]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:a52269602ff4\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:14 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[c980ddc2-bc4f-4685-bf20-f74134a0e144]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:14 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[1119d1fe-46ec-4890-b801-3aa1ada65493]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:14 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[497009f7-34be-436e-8f9c-cc3e61b2ee7e]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-dab285c2-8b0d-46ad-8db3-440348d0fd27 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Running cmd (subprocess): nvme version {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-dab285c2-8b0d-46ad-8db3-440348d0fd27 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] 'nvme version' failed. Not Retrying. {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:30:14 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.nvmeof [None req-dab285c2-8b0d-46ad-8db3-440348d0fd27 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] nvme not present on system {{(pid=72351) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Aug 09 02:30:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:30:14 user nova-compute[72351]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Aug 09 02:30:14 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e285228f-f6bc-485d-a899-a75c4e40cf3c]: (4, '') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:14 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-dab285c2-8b0d-46ad-8db3-440348d0fd27 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72351) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Aug 09 02:30:14 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-dab285c2-8b0d-46ad-8db3-440348d0fd27 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Aug 09 02:30:14 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-dab285c2-8b0d-46ad-8db3-440348d0fd27 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] LIGHTOS: no hostnqn found. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Aug 09 02:30:14 user nova-compute[72351]: DEBUG os_brick.utils [None req-dab285c2-8b0d-46ad-8db3-440348d0fd27 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] <== get_connector_properties: return (125ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:a52269602ff4', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:30:14 user nova-compute[72351]: DEBUG nova.virt.block_device [None req-dab285c2-8b0d-46ad-8db3-440348d0fd27 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Updating existing volume attachment record: f35a4e78-cb7a-4727-9763-80a042d9174d {{(pid=72351) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Aug 09 02:30:14 user nova-compute[72351]: DEBUG nova.compute.manager [req-b6514bdb-1ea7-4710-99d6-3e628e19e7b6 req-c6c45502-b8bc-451e-8470-53ba5d748f08 service nova] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Received event network-changed-c4a56310-1841-46ae-9a97-0421861e62c0 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:30:14 user nova-compute[72351]: DEBUG nova.compute.manager [req-b6514bdb-1ea7-4710-99d6-3e628e19e7b6 req-c6c45502-b8bc-451e-8470-53ba5d748f08 service nova] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Refreshing instance network info cache due to event network-changed-c4a56310-1841-46ae-9a97-0421861e62c0. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:30:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-b6514bdb-1ea7-4710-99d6-3e628e19e7b6 req-c6c45502-b8bc-451e-8470-53ba5d748f08 service nova] Acquiring lock "refresh_cache-7b6f5d60-5f23-4ef1-b972-9ba7086f56cb" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:30:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-b6514bdb-1ea7-4710-99d6-3e628e19e7b6 req-c6c45502-b8bc-451e-8470-53ba5d748f08 service nova] Acquired lock "refresh_cache-7b6f5d60-5f23-4ef1-b972-9ba7086f56cb" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:30:14 user nova-compute[72351]: DEBUG nova.network.neutron [req-b6514bdb-1ea7-4710-99d6-3e628e19e7b6 req-c6c45502-b8bc-451e-8470-53ba5d748f08 service nova] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Refreshing network info cache for port c4a56310-1841-46ae-9a97-0421861e62c0 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:30:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:15 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:15 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:15 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:15 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:15 user nova-compute[72351]: DEBUG nova.network.neutron [req-b6514bdb-1ea7-4710-99d6-3e628e19e7b6 req-c6c45502-b8bc-451e-8470-53ba5d748f08 service nova] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Updated VIF entry in instance network info cache for port c4a56310-1841-46ae-9a97-0421861e62c0. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:30:15 user nova-compute[72351]: DEBUG nova.network.neutron [req-b6514bdb-1ea7-4710-99d6-3e628e19e7b6 req-c6c45502-b8bc-451e-8470-53ba5d748f08 service nova] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Updating instance_info_cache with network_info: [{"id": "c4a56310-1841-46ae-9a97-0421861e62c0", "address": "fa:16:3e:75:cf:ba", "network": {"id": "49e8751c-e119-430f-9110-3dc6069b1589", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-899275135-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3aea1e66805146f7a8cd1d97e64a24af", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc4a56310-18", "ovs_interfaceid": "c4a56310-1841-46ae-9a97-0421861e62c0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:30:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-b6514bdb-1ea7-4710-99d6-3e628e19e7b6 req-c6c45502-b8bc-451e-8470-53ba5d748f08 service nova] Releasing lock "refresh_cache-7b6f5d60-5f23-4ef1-b972-9ba7086f56cb" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:30:15 user nova-compute[72351]: DEBUG nova.compute.manager [req-b6514bdb-1ea7-4710-99d6-3e628e19e7b6 req-c6c45502-b8bc-451e-8470-53ba5d748f08 service nova] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Received event network-changed-e8c25a1c-a605-4d7a-bb03-395b3d844571 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:30:15 user nova-compute[72351]: DEBUG nova.compute.manager [req-b6514bdb-1ea7-4710-99d6-3e628e19e7b6 req-c6c45502-b8bc-451e-8470-53ba5d748f08 service nova] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Refreshing instance network info cache due to event network-changed-e8c25a1c-a605-4d7a-bb03-395b3d844571. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:30:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-b6514bdb-1ea7-4710-99d6-3e628e19e7b6 req-c6c45502-b8bc-451e-8470-53ba5d748f08 service nova] Acquiring lock "refresh_cache-9df16e46-d097-4553-b44c-732ea24a4169" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:30:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-b6514bdb-1ea7-4710-99d6-3e628e19e7b6 req-c6c45502-b8bc-451e-8470-53ba5d748f08 service nova] Acquired lock "refresh_cache-9df16e46-d097-4553-b44c-732ea24a4169" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:30:15 user nova-compute[72351]: DEBUG nova.network.neutron [req-b6514bdb-1ea7-4710-99d6-3e628e19e7b6 req-c6c45502-b8bc-451e-8470-53ba5d748f08 service nova] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Refreshing network info cache for port e8c25a1c-a605-4d7a-bb03-395b3d844571 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:30:15 user nova-compute[72351]: DEBUG nova.compute.manager [req-2783843e-7599-4054-b688-36edcdd53559 req-b6fdbdb0-d005-48aa-a037-3a4b6b6e853c service nova] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Received event network-vif-plugged-c4a56310-1841-46ae-9a97-0421861e62c0 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:30:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-2783843e-7599-4054-b688-36edcdd53559 req-b6fdbdb0-d005-48aa-a037-3a4b6b6e853c service nova] Acquiring lock "7b6f5d60-5f23-4ef1-b972-9ba7086f56cb-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:30:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-2783843e-7599-4054-b688-36edcdd53559 req-b6fdbdb0-d005-48aa-a037-3a4b6b6e853c service nova] Lock "7b6f5d60-5f23-4ef1-b972-9ba7086f56cb-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:30:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-2783843e-7599-4054-b688-36edcdd53559 req-b6fdbdb0-d005-48aa-a037-3a4b6b6e853c service nova] Lock "7b6f5d60-5f23-4ef1-b972-9ba7086f56cb-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:30:15 user nova-compute[72351]: DEBUG nova.compute.manager [req-2783843e-7599-4054-b688-36edcdd53559 req-b6fdbdb0-d005-48aa-a037-3a4b6b6e853c service nova] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] No waiting events found dispatching network-vif-plugged-c4a56310-1841-46ae-9a97-0421861e62c0 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:30:15 user nova-compute[72351]: WARNING nova.compute.manager [req-2783843e-7599-4054-b688-36edcdd53559 req-b6fdbdb0-d005-48aa-a037-3a4b6b6e853c service nova] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Received unexpected event network-vif-plugged-c4a56310-1841-46ae-9a97-0421861e62c0 for instance with vm_state building and task_state spawning.
Aug 09 02:30:15 user nova-compute[72351]: DEBUG nova.compute.manager [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Start spawning the instance on the hypervisor. {{(pid=72351) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2612}}
Aug 09 02:30:15 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Creating instance directory {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4717}}
Aug 09 02:30:15 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Creating image(s)
Aug 09 02:30:15 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Did not create local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4853}}
Aug 09 02:30:15 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Ensure instance console log exists: /opt/stack/data/nova/instances/38a2b8e3-337d-416a-8a8c-da4921123db6/console.log {{(pid=72351) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4603}}
Aug 09 02:30:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:30:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:30:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:30:15 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Start _get_guest_xml network_info=[{"id": "0ebd83d0-5519-4945-8637-90c2ce024ac1", "address": "fa:16:3e:81:a2:62", "network": {"id": "c45d6719-95f9-4856-9b34-9d89c22ff899", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-927662772-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0ebd83d0-55", "ovs_interfaceid": "0ebd83d0-5519-4945-8637-90c2ce024ac1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "8b1245ae-85e9-49c3-9631-a288a172e4c1", "address": "fa:16:3e:a8:7b:e4", "network": {"id": "4617a3d0-dcb5-42a4-85d3-358087aeac69", "bridge": "br-int", "label": "tempest-device-tagging-net1-1844430962", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.34", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8b1245ae-85", "ovs_interfaceid": "8b1245ae-85e9-49c3-9631-a288a172e4c1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "1be27a59-07a4-4b6a-b25a-ce6d7efac4a3", "address": "fa:16:3e:38:62:75", "network": {"id": "4617a3d0-dcb5-42a4-85d3-358087aeac69", "bridge": "br-int", "label": "tempest-device-tagging-net1-1844430962", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.226", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1be27a59-07", "ovs_interfaceid": "1be27a59-07a4-4b6a-b25a-ce6d7efac4a3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "ccb930f6-9625-450b-b01b-c5009e610ae1", "address": "fa:16:3e:a9:0c:63", "network": {"id": "4617a3d0-dcb5-42a4-85d3-358087aeac69", "bridge": "br-int", "label": "tempest-device-tagging-net1-1844430962", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.237", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapccb930f6-96", "ovs_interfaceid": "ccb930f6-9625-450b-b01b-c5009e610ae1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "d083d9f2-dfe4-4a73-a624-3336f2f02b0d", "address": "fa:16:3e:a0:b1:db", "network": {"id": "4617a3d0-dcb5-42a4-85d3-358087aeac69", "bridge": "br-int", "label": "tempest-device-tagging-net1-1844430962", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.185", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd083d9f2-df", "ovs_interfaceid": "d083d9f2-dfe4-4a73-a624-3336f2f02b0d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "a88b872b-9340-4597-8ab4-ea6173dc4e4c", "address": "fa:16:3e:cf:3d:24", "network": {"id": "2984e5a6-dc0d-40ee-8064-f78a7686663e", "bridge": "br-int", "label": "tempest-device-tagging-net2-1630262622", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa88b872b-93", "ovs_interfaceid": "a88b872b-9340-4597-8ab4-ea6173dc4e4c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "48dfbf61-0142-4478-9ec1-585d5fbe11da", "address": "fa:16:3e:5b:d8:08", "network": {"id": "2984e5a6-dc0d-40ee-8064-f78a7686663e", "bridge": "br-int", "label": "tempest-device-tagging-net2-1630262622", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap48dfbf61-01", "ovs_interfaceid": "48dfbf61-0142-4478-9ec1-585d5fbe11da", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vdb': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk', 'boot_index': '2'}, '/dev/vdc': {'bus': 'virtio', 'dev': 'vdc', 'type': 'disk', 'boot_index': '3'}, 'disk.config': {'bus': 'ide', 'dev': 'hda', 'type': 'cdrom'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'device_type': 'disk', 'guest_format': None, 'disk_bus': 'virtio', 'mount_device': '/dev/vda', 'boot_index': 0, 'attachment_id': '33c7551c-71dd-4dd9-8a06-8fa840185786', 'delete_on_termination': False, 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqns': ['iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'NEfWCzIA', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '38a2b8e3-337d-416a-8a8c-da4921123db6', 'attached_at': '', 'detached_at': '', 'volume_id': '9d91e65d-30f6-4d64-a186-cbd8f80000b3', 'serial': '9d91e65d-30f6-4d64-a186-cbd8f80000b3'}, 'volume_type': None}, {'device_type': 'disk', 'guest_format': None, 'disk_bus': 'virtio', 'mount_device': '/dev/vdb', 'boot_index': 1, 'attachment_id': '29eb6e27-816c-4661-999d-90058aeb0ea8', 'delete_on_termination': False, 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqns': ['iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'pefgySaV', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '38a2b8e3-337d-416a-8a8c-da4921123db6', 'attached_at': '', 'detached_at': '', 'volume_id': 'aa951fc7-aa57-4fc9-9433-de522d77537e', 'serial': 'aa951fc7-aa57-4fc9-9433-de522d77537e'}, 'volume_type': None}, {'device_type': 'disk', 'guest_format': None, 'disk_bus': 'virtio', 'mount_device': '/dev/vdc', 'boot_index': 2, 'attachment_id': '545efb74-b5df-452f-b28e-17f3b3bdde89', 'delete_on_termination': False, 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqns': ['iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'UYXtqndm', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '38a2b8e3-337d-416a-8a8c-da4921123db6', 'attached_at': '', 'detached_at': '', 'volume_id': '5aa01207-8de0-4c53-9662-6771d109fe70', 'serial': '5aa01207-8de0-4c53-9662-6771d109fe70'}, 'volume_type': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:30:15 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:30:15 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:30:15 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:30:15 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:30:15 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:30:15 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:30:15 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:30:15 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:30:15 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:30:15 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:30:15 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:30:15 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:30:15 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:30:15 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:30:15 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:30:15 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:30:15 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:30:15 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:30:15 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 02:30:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'NEfWCzIA', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:30:15 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:30:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:30:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:30:15 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:30:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:30:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:30:15 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 02:30:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 -p 172.16.0.220:3260" returned: 21 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:15 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[eca2ec15-1bb1-4fe0-80f1-7581baf1d088]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Aug 09 02:30:15 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:30:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 -p 172.16.0.220:3260 --interface default --op new {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:15 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[1d03ecb6-f7fa-4566-85fb-3ae1077b9472]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3] added\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3] added
Aug 09 02:30:15 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:30:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.024s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:15 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[fb588205-1dfb-4a67-b751-17aee4232a7f]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:30:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:15 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[cbe1d5a1-91f5-4992-b973-ad1eaecc02e0]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:30:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v NEfWCzIA {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:15 user nova-compute[72351]: DEBUG nova.network.neutron [req-b6514bdb-1ea7-4710-99d6-3e628e19e7b6 req-c6c45502-b8bc-451e-8470-53ba5d748f08 service nova] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Updated VIF entry in instance network info cache for port e8c25a1c-a605-4d7a-bb03-395b3d844571. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:30:16 user nova-compute[72351]: DEBUG nova.network.neutron [req-b6514bdb-1ea7-4710-99d6-3e628e19e7b6 req-c6c45502-b8bc-451e-8470-53ba5d748f08 service nova] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Updating instance_info_cache with network_info: [{"id": "e8c25a1c-a605-4d7a-bb03-395b3d844571", "address": "fa:16:3e:69:6a:fa", "network": {"id": "96018a29-3b2e-44de-b2c4-67abc92e3a30", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-924549970-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.167", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "1c8ff52319444079b754f79843c67b2b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape8c25a1c-a6", "ovs_interfaceid": "e8c25a1c-a605-4d7a-bb03-395b3d844571", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:30:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v NEfWCzIA" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:16 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[03be4e94-22cb-4a6f-af39-e433ca155393]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:16 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'NEfWCzIA'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:30:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:16 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-b6514bdb-1ea7-4710-99d6-3e628e19e7b6 req-c6c45502-b8bc-451e-8470-53ba5d748f08 service nova] Releasing lock "refresh_cache-9df16e46-d097-4553-b44c-732ea24a4169" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:30:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:16 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[665827bf-978c-43eb-bc87-d241527b5e2b]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:16 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:30:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:16 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[40391974-7429-4e42-8d53-bab630bc2cb3]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:16 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:30:16 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:30:16 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:30:16 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:30:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 -p 172.16.0.220:3260 --login {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 -p 172.16.0.220:3260 --login" returned: 0 in 0.062s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:16 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[af20eecd-9295-46fd-a42b-a65feaff2feb]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:16 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3, portal: 172.16.0.220,3260]
Aug 09 02:30:16 user nova-compute[72351]: Login to [iface: default, target: iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3, portal: 172.16.0.220,3260] successful.
Aug 09 02:30:16 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:30:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:16 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ccb3301e-0a8c-4a70-842f-211d1e53538e]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:16 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:30:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:16 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[5584f7bb-eaa1-4814-a889-0f0814ab1937]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:16 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:30:16 user nova-compute[72351]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 (non-flash)
Aug 09 02:30:16 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:30:16 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:30:16 user nova-compute[72351]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 (non-flash)
Aug 09 02:30:16 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:30:16 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.286s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:30:16 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 02:30:16 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] HCTL ('33', '-', '-', 0) found on session 33 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 02:30:16 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 02:30:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:16 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[f64531a2-bf88-4bb6-a53c-604e0860e66f]: (4, ('- - 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:16 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Searching for a device in session 33 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:30:16 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Connected to sdb using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'NEfWCzIA', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.319s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] <== connect_volume: return (1321ms) {'type': 'block', 'scsi_wwn': '26565326230356231', 'path': '/dev/sdb'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26565326230356231', 'path': '/dev/sdb'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'pefgySaV', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:30:17 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:30:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:30:17 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:30:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:30:17 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 02:30:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e -p 172.16.0.220:3260" returned: 21 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[1fe72c54-73d9-4a6d-aa16-4f80a88c9f3e]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Aug 09 02:30:17 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e -p 172.16.0.220:3260 --interface default --op new {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[4b3fe1a1-0399-45bb-93b3-297982f6c722]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e] added\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e] added
Aug 09 02:30:17 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[27d04e2e-e573-4a71-937c-41515b286608]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548217.3003876, 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:30:17 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] VM Resumed (Lifecycle Event)
Aug 09 02:30:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG nova.compute.manager [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Guest created on hypervisor {{(pid=72351) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4411}}
Aug 09 02:30:17 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Instance spawned successfully.
Aug 09 02:30:17 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:894}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[008c9d47-0d7e-43a2-b4bd-98ebfa84b227]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e -p 172.16.0.220:3260 --op update -n node.session.auth.username -v pefgySaV {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Found default for hw_cdrom_bus of ide {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Found default for hw_disk_bus of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Found default for hw_input_bus of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Found default for hw_pointer_model of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Found default for hw_video_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Found default for hw_vif_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e -p 172.16.0.220:3260 --op update -n node.session.auth.username -v pefgySaV" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[4276234c-f06b-463e-8098-778a2ef0d6bd]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'pefgySaV'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:17 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:30:17 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548217.3017232, 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:30:17 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] VM Started (Lifecycle Event)
Aug 09 02:30:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[64ad0d03-821b-4896-9389-4a52423605bb]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[dfa1101b-d1bc-4eea-a2ca-7e2a466b9585]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:30:17 user nova-compute[72351]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 (non-flash)
Aug 09 02:30:17 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:30:17 user nova-compute[72351]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 (non-flash)
Aug 09 02:30:17 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e -p 172.16.0.220:3260 --login {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:17 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:30:17 user nova-compute[72351]: INFO nova.compute.manager [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Took 6.65 seconds to spawn the instance on the hypervisor.
Aug 09 02:30:17 user nova-compute[72351]: DEBUG nova.compute.manager [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e -p 172.16.0.220:3260 --login" returned: 0 in 0.063s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[2d1f8cb1-0b07-4a6d-9a26-62e06183f1bf]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e, portal: 172.16.0.220,3260]
Aug 09 02:30:17 user nova-compute[72351]: Login to [iface: default, target: iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e, portal: 172.16.0.220,3260] successful.
Aug 09 02:30:17 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[02d8a72a-eb14-4060-9785-d0c0e1979b0c]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG nova.compute.manager [req-bec5c85c-a12a-4f2a-b2d6-81b4895b2841 req-2483c637-0e8b-43ef-950e-32b3041ed734 service nova] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Received event network-vif-plugged-c4a56310-1841-46ae-9a97-0421861e62c0 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-bec5c85c-a12a-4f2a-b2d6-81b4895b2841 req-2483c637-0e8b-43ef-950e-32b3041ed734 service nova] Acquiring lock "7b6f5d60-5f23-4ef1-b972-9ba7086f56cb-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-bec5c85c-a12a-4f2a-b2d6-81b4895b2841 req-2483c637-0e8b-43ef-950e-32b3041ed734 service nova] Lock "7b6f5d60-5f23-4ef1-b972-9ba7086f56cb-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-bec5c85c-a12a-4f2a-b2d6-81b4895b2841 req-2483c637-0e8b-43ef-950e-32b3041ed734 service nova] Lock "7b6f5d60-5f23-4ef1-b972-9ba7086f56cb-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG nova.compute.manager [req-bec5c85c-a12a-4f2a-b2d6-81b4895b2841 req-2483c637-0e8b-43ef-950e-32b3041ed734 service nova] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] No waiting events found dispatching network-vif-plugged-c4a56310-1841-46ae-9a97-0421861e62c0 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:30:17 user nova-compute[72351]: WARNING nova.compute.manager [req-bec5c85c-a12a-4f2a-b2d6-81b4895b2841 req-2483c637-0e8b-43ef-950e-32b3041ed734 service nova] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Received unexpected event network-vif-plugged-c4a56310-1841-46ae-9a97-0421861e62c0 for instance with vm_state building and task_state spawning.
Aug 09 02:30:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ff4035c9-d3cd-4b2b-8e83-1d80cb6777a7]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:30:17 user nova-compute[72351]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 (non-flash)
Aug 09 02:30:17 user nova-compute[72351]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e (non-flash)
Aug 09 02:30:17 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:30:17 user nova-compute[72351]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 (non-flash)
Aug 09 02:30:17 user nova-compute[72351]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e (non-flash)
Aug 09 02:30:17 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.286s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] HCTL ('34', '-', '-', 0) found on session 34 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Scanning host 34 c: -, t: -, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:17 user nova-compute[72351]: INFO nova.compute.manager [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Took 7.50 seconds to build instance.
Aug 09 02:30:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.025s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[4afe4fc0-babc-4f89-ab8b-066773d561cb]: (4, ('- - 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Searching for a device in session 34 and hctl ['34', '*', '*', 0] yield: sdc {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Connected to sdc using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'pefgySaV', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 02:30:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ab9401db-599f-49be-a3be-1c69f8a4d076 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "7b6f5d60-5f23-4ef1-b972-9ba7086f56cb" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 7.631s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:30:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.324s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:30:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] <== connect_volume: return (1325ms) {'type': 'block', 'scsi_wwn': '26566656661613334', 'path': '/dev/sdc'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:30:18 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26566656661613334', 'path': '/dev/sdc'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 02:30:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:18 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 02:30:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'UYXtqndm', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:30:18 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:30:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:30:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:30:18 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:30:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:30:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:30:18 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 02:30:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70 -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70 -p 172.16.0.220:3260" returned: 21 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:18 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[dc451c34-0bd1-4213-b2ca-add3fed875f4]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Aug 09 02:30:18 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:30:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70 -p 172.16.0.220:3260 --interface default --op new {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:18 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[918d66e8-5897-43e9-8444-6e950c6c5694]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70] added\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70] added
Aug 09 02:30:18 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:30:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:18 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[87daf583-6a0e-457e-a86f-48290dfeec21]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:30:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:18 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ec31da9f-3fbb-4a23-9e91-bf5aeb1159ac]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:30:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v UYXtqndm {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v UYXtqndm" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:18 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[69411309-4824-44a9-8b63-2e6b78b884aa]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'UYXtqndm'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:30:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:18 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d343b663-ce8a-4414-90bc-a7073aff58a8]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:30:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:18 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d02292eb-6e98-4b00-baa8-be904e9b5aeb]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:30:18 user nova-compute[72351]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 (non-flash)
Aug 09 02:30:18 user nova-compute[72351]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e (non-flash)
Aug 09 02:30:18 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:30:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:30:18 user nova-compute[72351]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 (non-flash)
Aug 09 02:30:18 user nova-compute[72351]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e (non-flash)
Aug 09 02:30:18 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:30:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70 -p 172.16.0.220:3260 --login {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70 -p 172.16.0.220:3260 --login" returned: 0 in 0.068s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:18 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[3e3428d4-4f70-4bbd-af38-5f8140545074]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70, portal: 172.16.0.220,3260]
Aug 09 02:30:18 user nova-compute[72351]: Login to [iface: default, target: iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70, portal: 172.16.0.220,3260] successful.
Aug 09 02:30:18 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:30:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:18 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a1cfa990-1a99-400d-8e25-e3a16a1907a1]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:30:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:18 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[37ab0a09-f96d-411a-9366-9f57fb4a5acb]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e (non-flash)\ntcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:30:18 user nova-compute[72351]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 (non-flash)
Aug 09 02:30:18 user nova-compute[72351]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e (non-flash)
Aug 09 02:30:18 user nova-compute[72351]: tcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70 (non-flash)
Aug 09 02:30:18 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:30:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:30:18 user nova-compute[72351]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 (non-flash)
Aug 09 02:30:18 user nova-compute[72351]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e (non-flash)
Aug 09 02:30:18 user nova-compute[72351]: tcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70 (non-flash)
Aug 09 02:30:18 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:30:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.289s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:30:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 02:30:18 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] HCTL ('35', '-', '-', 0) found on session 35 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 02:30:18 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Scanning host 35 c: -, t: -, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 02:30:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host35/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host35/scan" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:18 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[b34469c3-4561-46aa-b1d0-cb5cb80cae16]: (4, ('- - 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:18 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Searching for a device in session 35 and hctl ['35', '*', '*', 0] yield: None {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:30:19 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Searching for a device in session 35 and hctl ['35', '*', '*', 0] yield: sdd {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:30:19 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Connected to sdd using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'UYXtqndm', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 02:30:19 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.319s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:30:19 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] <== connect_volume: return (1320ms) {'type': 'block', 'scsi_wwn': '26439663137613961', 'path': '/dev/sdd'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:30:19 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26439663137613961', 'path': '/dev/sdd'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 02:30:19 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-08-09T02:29:47Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-973345123',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-973345123',id=33,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBC5HMwUPTdKeiDcsJ4TkVOxRLOG1g/yzvRBKkBQKOQuDjCRYdbTKK7gH9+Y0CpEbv5BvHxw4RfGS5kTqYSc0L/jaTiuzJMVzMoXsxa5nneD0wyJcR62H0YaKv6Pgz9BCcQ==',key_name='tempest-keypair-2018857894',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='fb5ff9ee85084536b6479ed16b24293d',ramdisk_id='',reservation_id='r-oubsycpu',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-942602465',owner_user_name='tempest-TaggedBootDevicesTest_v242-942602465-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:29:49Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='76924d471e204a5ebee3c99280e62c91',uuid=38a2b8e3-337d-416a-8a8c-da4921123db6,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "0ebd83d0-5519-4945-8637-90c2ce024ac1", "address": "fa:16:3e:81:a2:62", "network": {"id": "c45d6719-95f9-4856-9b34-9d89c22ff899", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-927662772-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0ebd83d0-55", "ovs_interfaceid": "0ebd83d0-5519-4945-8637-90c2ce024ac1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:30:19 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Converting VIF {"id": "0ebd83d0-5519-4945-8637-90c2ce024ac1", "address": "fa:16:3e:81:a2:62", "network": {"id": "c45d6719-95f9-4856-9b34-9d89c22ff899", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-927662772-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0ebd83d0-55", "ovs_interfaceid": "0ebd83d0-5519-4945-8637-90c2ce024ac1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:30:19 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:81:a2:62,bridge_name='br-int',has_traffic_filtering=True,id=0ebd83d0-5519-4945-8637-90c2ce024ac1,network=Network(c45d6719-95f9-4856-9b34-9d89c22ff899),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap0ebd83d0-55') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:30:19 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-08-09T02:29:47Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-973345123',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-973345123',id=33,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBC5HMwUPTdKeiDcsJ4TkVOxRLOG1g/yzvRBKkBQKOQuDjCRYdbTKK7gH9+Y0CpEbv5BvHxw4RfGS5kTqYSc0L/jaTiuzJMVzMoXsxa5nneD0wyJcR62H0YaKv6Pgz9BCcQ==',key_name='tempest-keypair-2018857894',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='fb5ff9ee85084536b6479ed16b24293d',ramdisk_id='',reservation_id='r-oubsycpu',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-942602465',owner_user_name='tempest-TaggedBootDevicesTest_v242-942602465-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:29:49Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='76924d471e204a5ebee3c99280e62c91',uuid=38a2b8e3-337d-416a-8a8c-da4921123db6,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "8b1245ae-85e9-49c3-9631-a288a172e4c1", "address": "fa:16:3e:a8:7b:e4", "network": {"id": "4617a3d0-dcb5-42a4-85d3-358087aeac69", "bridge": "br-int", "label": "tempest-device-tagging-net1-1844430962", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.34", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8b1245ae-85", "ovs_interfaceid": "8b1245ae-85e9-49c3-9631-a288a172e4c1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:30:19 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Converting VIF {"id": "8b1245ae-85e9-49c3-9631-a288a172e4c1", "address": "fa:16:3e:a8:7b:e4", "network": {"id": "4617a3d0-dcb5-42a4-85d3-358087aeac69", "bridge": "br-int", "label": "tempest-device-tagging-net1-1844430962", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.34", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8b1245ae-85", "ovs_interfaceid": "8b1245ae-85e9-49c3-9631-a288a172e4c1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:30:19 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:a8:7b:e4,bridge_name='br-int',has_traffic_filtering=True,id=8b1245ae-85e9-49c3-9631-a288a172e4c1,network=Network(4617a3d0-dcb5-42a4-85d3-358087aeac69),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap8b1245ae-85') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:30:19 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-08-09T02:29:47Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-973345123',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-973345123',id=33,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBC5HMwUPTdKeiDcsJ4TkVOxRLOG1g/yzvRBKkBQKOQuDjCRYdbTKK7gH9+Y0CpEbv5BvHxw4RfGS5kTqYSc0L/jaTiuzJMVzMoXsxa5nneD0wyJcR62H0YaKv6Pgz9BCcQ==',key_name='tempest-keypair-2018857894',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='fb5ff9ee85084536b6479ed16b24293d',ramdisk_id='',reservation_id='r-oubsycpu',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-942602465',owner_user_name='tempest-TaggedBootDevicesTest_v242-942602465-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:29:49Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='76924d471e204a5ebee3c99280e62c91',uuid=38a2b8e3-337d-416a-8a8c-da4921123db6,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "1be27a59-07a4-4b6a-b25a-ce6d7efac4a3", "address": "fa:16:3e:38:62:75", "network": {"id": "4617a3d0-dcb5-42a4-85d3-358087aeac69", "bridge": "br-int", "label": "tempest-device-tagging-net1-1844430962", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.226", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1be27a59-07", "ovs_interfaceid": "1be27a59-07a4-4b6a-b25a-ce6d7efac4a3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:30:19 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Converting VIF {"id": "1be27a59-07a4-4b6a-b25a-ce6d7efac4a3", "address": "fa:16:3e:38:62:75", "network": {"id": "4617a3d0-dcb5-42a4-85d3-358087aeac69", "bridge": "br-int", "label": "tempest-device-tagging-net1-1844430962", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.226", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1be27a59-07", "ovs_interfaceid": "1be27a59-07a4-4b6a-b25a-ce6d7efac4a3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:30:19 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:38:62:75,bridge_name='br-int',has_traffic_filtering=True,id=1be27a59-07a4-4b6a-b25a-ce6d7efac4a3,network=Network(4617a3d0-dcb5-42a4-85d3-358087aeac69),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap1be27a59-07') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:30:19 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-08-09T02:29:47Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-973345123',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-973345123',id=33,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBC5HMwUPTdKeiDcsJ4TkVOxRLOG1g/yzvRBKkBQKOQuDjCRYdbTKK7gH9+Y0CpEbv5BvHxw4RfGS5kTqYSc0L/jaTiuzJMVzMoXsxa5nneD0wyJcR62H0YaKv6Pgz9BCcQ==',key_name='tempest-keypair-2018857894',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='fb5ff9ee85084536b6479ed16b24293d',ramdisk_id='',reservation_id='r-oubsycpu',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-942602465',owner_user_name='tempest-TaggedBootDevicesTest_v242-942602465-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:29:49Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='76924d471e204a5ebee3c99280e62c91',uuid=38a2b8e3-337d-416a-8a8c-da4921123db6,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "ccb930f6-9625-450b-b01b-c5009e610ae1", "address": "fa:16:3e:a9:0c:63", "network": {"id": "4617a3d0-dcb5-42a4-85d3-358087aeac69", "bridge": "br-int", "label": "tempest-device-tagging-net1-1844430962", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.237", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapccb930f6-96", "ovs_interfaceid": "ccb930f6-9625-450b-b01b-c5009e610ae1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:30:19 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Converting VIF {"id": "ccb930f6-9625-450b-b01b-c5009e610ae1", "address": "fa:16:3e:a9:0c:63", "network": {"id": "4617a3d0-dcb5-42a4-85d3-358087aeac69", "bridge": "br-int", "label": "tempest-device-tagging-net1-1844430962", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.237", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapccb930f6-96", "ovs_interfaceid": "ccb930f6-9625-450b-b01b-c5009e610ae1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:30:19 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:a9:0c:63,bridge_name='br-int',has_traffic_filtering=True,id=ccb930f6-9625-450b-b01b-c5009e610ae1,network=Network(4617a3d0-dcb5-42a4-85d3-358087aeac69),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapccb930f6-96') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:30:19 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-08-09T02:29:47Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-973345123',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-973345123',id=33,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBC5HMwUPTdKeiDcsJ4TkVOxRLOG1g/yzvRBKkBQKOQuDjCRYdbTKK7gH9+Y0CpEbv5BvHxw4RfGS5kTqYSc0L/jaTiuzJMVzMoXsxa5nneD0wyJcR62H0YaKv6Pgz9BCcQ==',key_name='tempest-keypair-2018857894',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='fb5ff9ee85084536b6479ed16b24293d',ramdisk_id='',reservation_id='r-oubsycpu',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-942602465',owner_user_name='tempest-TaggedBootDevicesTest_v242-942602465-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:29:49Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='76924d471e204a5ebee3c99280e62c91',uuid=38a2b8e3-337d-416a-8a8c-da4921123db6,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "d083d9f2-dfe4-4a73-a624-3336f2f02b0d", "address": "fa:16:3e:a0:b1:db", "network": {"id": "4617a3d0-dcb5-42a4-85d3-358087aeac69", "bridge": "br-int", "label": "tempest-device-tagging-net1-1844430962", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.185", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd083d9f2-df", "ovs_interfaceid": "d083d9f2-dfe4-4a73-a624-3336f2f02b0d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:30:19 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Converting VIF {"id": "d083d9f2-dfe4-4a73-a624-3336f2f02b0d", "address": "fa:16:3e:a0:b1:db", "network": {"id": "4617a3d0-dcb5-42a4-85d3-358087aeac69", "bridge": "br-int", "label": "tempest-device-tagging-net1-1844430962", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.185", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd083d9f2-df", "ovs_interfaceid": "d083d9f2-dfe4-4a73-a624-3336f2f02b0d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:30:19 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:a0:b1:db,bridge_name='br-int',has_traffic_filtering=True,id=d083d9f2-dfe4-4a73-a624-3336f2f02b0d,network=Network(4617a3d0-dcb5-42a4-85d3-358087aeac69),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd083d9f2-df') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:30:19 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-08-09T02:29:47Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-973345123',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-973345123',id=33,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBC5HMwUPTdKeiDcsJ4TkVOxRLOG1g/yzvRBKkBQKOQuDjCRYdbTKK7gH9+Y0CpEbv5BvHxw4RfGS5kTqYSc0L/jaTiuzJMVzMoXsxa5nneD0wyJcR62H0YaKv6Pgz9BCcQ==',key_name='tempest-keypair-2018857894',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='fb5ff9ee85084536b6479ed16b24293d',ramdisk_id='',reservation_id='r-oubsycpu',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-942602465',owner_user_name='tempest-TaggedBootDevicesTest_v242-942602465-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:29:49Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='76924d471e204a5ebee3c99280e62c91',uuid=38a2b8e3-337d-416a-8a8c-da4921123db6,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "a88b872b-9340-4597-8ab4-ea6173dc4e4c", "address": "fa:16:3e:cf:3d:24", "network": {"id": "2984e5a6-dc0d-40ee-8064-f78a7686663e", "bridge": "br-int", "label": "tempest-device-tagging-net2-1630262622", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa88b872b-93", "ovs_interfaceid": "a88b872b-9340-4597-8ab4-ea6173dc4e4c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:30:19 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Converting VIF {"id": "a88b872b-9340-4597-8ab4-ea6173dc4e4c", "address": "fa:16:3e:cf:3d:24", "network": {"id": "2984e5a6-dc0d-40ee-8064-f78a7686663e", "bridge": "br-int", "label": "tempest-device-tagging-net2-1630262622", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa88b872b-93", "ovs_interfaceid": "a88b872b-9340-4597-8ab4-ea6173dc4e4c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:30:19 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:cf:3d:24,bridge_name='br-int',has_traffic_filtering=True,id=a88b872b-9340-4597-8ab4-ea6173dc4e4c,network=Network(2984e5a6-dc0d-40ee-8064-f78a7686663e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa88b872b-93') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:30:19 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-08-09T02:29:47Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-973345123',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-973345123',id=33,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBC5HMwUPTdKeiDcsJ4TkVOxRLOG1g/yzvRBKkBQKOQuDjCRYdbTKK7gH9+Y0CpEbv5BvHxw4RfGS5kTqYSc0L/jaTiuzJMVzMoXsxa5nneD0wyJcR62H0YaKv6Pgz9BCcQ==',key_name='tempest-keypair-2018857894',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='fb5ff9ee85084536b6479ed16b24293d',ramdisk_id='',reservation_id='r-oubsycpu',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-942602465',owner_user_name='tempest-TaggedBootDevicesTest_v242-942602465-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:29:49Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='76924d471e204a5ebee3c99280e62c91',uuid=38a2b8e3-337d-416a-8a8c-da4921123db6,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "48dfbf61-0142-4478-9ec1-585d5fbe11da", "address": "fa:16:3e:5b:d8:08", "network": {"id": "2984e5a6-dc0d-40ee-8064-f78a7686663e", "bridge": "br-int", "label": "tempest-device-tagging-net2-1630262622", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap48dfbf61-01", "ovs_interfaceid": "48dfbf61-0142-4478-9ec1-585d5fbe11da", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:30:19 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Converting VIF {"id": "48dfbf61-0142-4478-9ec1-585d5fbe11da", "address": "fa:16:3e:5b:d8:08", "network": {"id": "2984e5a6-dc0d-40ee-8064-f78a7686663e", "bridge": "br-int", "label": "tempest-device-tagging-net2-1630262622", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap48dfbf61-01", "ovs_interfaceid": "48dfbf61-0142-4478-9ec1-585d5fbe11da", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:30:19 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:5b:d8:08,bridge_name='br-int',has_traffic_filtering=True,id=48dfbf61-0142-4478-9ec1-585d5fbe11da,network=Network(2984e5a6-dc0d-40ee-8064-f78a7686663e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap48dfbf61-01') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:30:19 user nova-compute[72351]: DEBUG nova.objects.instance [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Lazy-loading 'pci_devices' on Instance uuid 38a2b8e3-337d-416a-8a8c-da4921123db6 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:30:19 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:30:19 user nova-compute[72351]:   <uuid>38a2b8e3-337d-416a-8a8c-da4921123db6</uuid>
Aug 09 02:30:19 user nova-compute[72351]:   <name>instance-00000021</name>
Aug 09 02:30:19 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:30:19 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:30:19 user nova-compute[72351]:   <metadata>
Aug 09 02:30:19 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:30:19 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:30:19 user nova-compute[72351]:       <nova:name>tempest-device-tagging-server-973345123</nova:name>
Aug 09 02:30:19 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:30:15</nova:creationTime>
Aug 09 02:30:19 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:30:19 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:30:19 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:30:19 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:30:19 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:30:19 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:30:19 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:30:19 user nova-compute[72351]:       <nova:owner>
Aug 09 02:30:19 user nova-compute[72351]:         <nova:user uuid="76924d471e204a5ebee3c99280e62c91">tempest-TaggedBootDevicesTest_v242-942602465-project-member</nova:user>
Aug 09 02:30:19 user nova-compute[72351]:         <nova:project uuid="fb5ff9ee85084536b6479ed16b24293d">tempest-TaggedBootDevicesTest_v242-942602465</nova:project>
Aug 09 02:30:19 user nova-compute[72351]:       </nova:owner>
Aug 09 02:30:19 user nova-compute[72351]:       <nova:root type="image" uuid="18ccda7f-31d3-4978-ab9d-4ed2f74eff94"/>
Aug 09 02:30:19 user nova-compute[72351]:       <nova:ports>
Aug 09 02:30:19 user nova-compute[72351]:         <nova:port uuid="0ebd83d0-5519-4945-8637-90c2ce024ac1">
Aug 09 02:30:19 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.7" ipVersion="4"/>
Aug 09 02:30:19 user nova-compute[72351]:         </nova:port>
Aug 09 02:30:19 user nova-compute[72351]:         <nova:port uuid="8b1245ae-85e9-49c3-9631-a288a172e4c1">
Aug 09 02:30:19 user nova-compute[72351]:           <nova:ip type="fixed" address="10.1.1.34" ipVersion="4"/>
Aug 09 02:30:19 user nova-compute[72351]:         </nova:port>
Aug 09 02:30:19 user nova-compute[72351]:         <nova:port uuid="1be27a59-07a4-4b6a-b25a-ce6d7efac4a3">
Aug 09 02:30:19 user nova-compute[72351]:           <nova:ip type="fixed" address="10.1.1.226" ipVersion="4"/>
Aug 09 02:30:19 user nova-compute[72351]:         </nova:port>
Aug 09 02:30:19 user nova-compute[72351]:         <nova:port uuid="ccb930f6-9625-450b-b01b-c5009e610ae1">
Aug 09 02:30:19 user nova-compute[72351]:           <nova:ip type="fixed" address="10.1.1.237" ipVersion="4"/>
Aug 09 02:30:19 user nova-compute[72351]:         </nova:port>
Aug 09 02:30:19 user nova-compute[72351]:         <nova:port uuid="d083d9f2-dfe4-4a73-a624-3336f2f02b0d">
Aug 09 02:30:19 user nova-compute[72351]:           <nova:ip type="fixed" address="10.1.1.185" ipVersion="4"/>
Aug 09 02:30:19 user nova-compute[72351]:         </nova:port>
Aug 09 02:30:19 user nova-compute[72351]:         <nova:port uuid="a88b872b-9340-4597-8ab4-ea6173dc4e4c">
Aug 09 02:30:19 user nova-compute[72351]:           <nova:ip type="fixed" address="10.2.2.100" ipVersion="4"/>
Aug 09 02:30:19 user nova-compute[72351]:         </nova:port>
Aug 09 02:30:19 user nova-compute[72351]:         <nova:port uuid="48dfbf61-0142-4478-9ec1-585d5fbe11da">
Aug 09 02:30:19 user nova-compute[72351]:           <nova:ip type="fixed" address="10.2.2.200" ipVersion="4"/>
Aug 09 02:30:19 user nova-compute[72351]:         </nova:port>
Aug 09 02:30:19 user nova-compute[72351]:       </nova:ports>
Aug 09 02:30:19 user nova-compute[72351]:     </nova:instance>
Aug 09 02:30:19 user nova-compute[72351]:   </metadata>
Aug 09 02:30:19 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:30:19 user nova-compute[72351]:     <system>
Aug 09 02:30:19 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:30:19 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:30:19 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:30:19 user nova-compute[72351]:       <entry name="serial">38a2b8e3-337d-416a-8a8c-da4921123db6</entry>
Aug 09 02:30:19 user nova-compute[72351]:       <entry name="uuid">38a2b8e3-337d-416a-8a8c-da4921123db6</entry>
Aug 09 02:30:19 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:30:19 user nova-compute[72351]:     </system>
Aug 09 02:30:19 user nova-compute[72351]:   </sysinfo>
Aug 09 02:30:19 user nova-compute[72351]:   <os>
Aug 09 02:30:19 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:30:19 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:30:19 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:30:19 user nova-compute[72351]:   </os>
Aug 09 02:30:19 user nova-compute[72351]:   <features>
Aug 09 02:30:19 user nova-compute[72351]:     <acpi/>
Aug 09 02:30:19 user nova-compute[72351]:     <apic/>
Aug 09 02:30:19 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:30:19 user nova-compute[72351]:   </features>
Aug 09 02:30:19 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:30:19 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:30:19 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:30:19 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:30:19 user nova-compute[72351]:   </clock>
Aug 09 02:30:19 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:30:19 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:30:19 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:30:19 user nova-compute[72351]:   </cpu>
Aug 09 02:30:19 user nova-compute[72351]:   <devices>
Aug 09 02:30:19 user nova-compute[72351]:     <disk type="file" device="cdrom">
Aug 09 02:30:19 user nova-compute[72351]:       <driver name="qemu" type="raw" cache="none"/>
Aug 09 02:30:19 user nova-compute[72351]:       <source file="/opt/stack/data/nova/instances/38a2b8e3-337d-416a-8a8c-da4921123db6/disk.config"/>
Aug 09 02:30:19 user nova-compute[72351]:       <target dev="hda" bus="ide"/>
Aug 09 02:30:19 user nova-compute[72351]:     </disk>
Aug 09 02:30:19 user nova-compute[72351]:     <disk type="block" device="disk">
Aug 09 02:30:19 user nova-compute[72351]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:30:19 user nova-compute[72351]:       <source dev="/dev/sdb"/>
Aug 09 02:30:19 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:30:19 user nova-compute[72351]:       <serial>9d91e65d-30f6-4d64-a186-cbd8f80000b3</serial>
Aug 09 02:30:19 user nova-compute[72351]:     </disk>
Aug 09 02:30:19 user nova-compute[72351]:     <disk type="block" device="disk">
Aug 09 02:30:19 user nova-compute[72351]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:30:19 user nova-compute[72351]:       <source dev="/dev/sdc"/>
Aug 09 02:30:19 user nova-compute[72351]:       <target dev="vdb" bus="virtio"/>
Aug 09 02:30:19 user nova-compute[72351]:       <serial>aa951fc7-aa57-4fc9-9433-de522d77537e</serial>
Aug 09 02:30:19 user nova-compute[72351]:     </disk>
Aug 09 02:30:19 user nova-compute[72351]:     <disk type="block" device="disk">
Aug 09 02:30:19 user nova-compute[72351]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:30:19 user nova-compute[72351]:       <source dev="/dev/sdd"/>
Aug 09 02:30:19 user nova-compute[72351]:       <target dev="vdc" bus="virtio"/>
Aug 09 02:30:19 user nova-compute[72351]:       <serial>5aa01207-8de0-4c53-9662-6771d109fe70</serial>
Aug 09 02:30:19 user nova-compute[72351]:     </disk>
Aug 09 02:30:19 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:30:19 user nova-compute[72351]:       <mac address="fa:16:3e:81:a2:62"/>
Aug 09 02:30:19 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:30:19 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:30:19 user nova-compute[72351]:       <target dev="tap0ebd83d0-55"/>
Aug 09 02:30:19 user nova-compute[72351]:     </interface>
Aug 09 02:30:19 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:30:19 user nova-compute[72351]:       <mac address="fa:16:3e:a8:7b:e4"/>
Aug 09 02:30:19 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:30:19 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:30:19 user nova-compute[72351]:       <target dev="tap8b1245ae-85"/>
Aug 09 02:30:19 user nova-compute[72351]:     </interface>
Aug 09 02:30:19 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:30:19 user nova-compute[72351]:       <mac address="fa:16:3e:38:62:75"/>
Aug 09 02:30:19 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:30:19 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:30:19 user nova-compute[72351]:       <target dev="tap1be27a59-07"/>
Aug 09 02:30:19 user nova-compute[72351]:     </interface>
Aug 09 02:30:19 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:30:19 user nova-compute[72351]:       <mac address="fa:16:3e:a9:0c:63"/>
Aug 09 02:30:19 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:30:19 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:30:19 user nova-compute[72351]:       <target dev="tapccb930f6-96"/>
Aug 09 02:30:19 user nova-compute[72351]:     </interface>
Aug 09 02:30:19 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:30:19 user nova-compute[72351]:       <mac address="fa:16:3e:a0:b1:db"/>
Aug 09 02:30:19 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:30:19 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:30:19 user nova-compute[72351]:       <target dev="tapd083d9f2-df"/>
Aug 09 02:30:19 user nova-compute[72351]:     </interface>
Aug 09 02:30:19 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:30:19 user nova-compute[72351]:       <mac address="fa:16:3e:cf:3d:24"/>
Aug 09 02:30:19 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:30:19 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:30:19 user nova-compute[72351]:       <target dev="tapa88b872b-93"/>
Aug 09 02:30:19 user nova-compute[72351]:     </interface>
Aug 09 02:30:19 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:30:19 user nova-compute[72351]:       <mac address="fa:16:3e:5b:d8:08"/>
Aug 09 02:30:19 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:30:19 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:30:19 user nova-compute[72351]:       <target dev="tap48dfbf61-01"/>
Aug 09 02:30:19 user nova-compute[72351]:     </interface>
Aug 09 02:30:19 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:30:19 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/38a2b8e3-337d-416a-8a8c-da4921123db6/console.log" append="off"/>
Aug 09 02:30:19 user nova-compute[72351]:     </serial>
Aug 09 02:30:19 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:30:19 user nova-compute[72351]:     <video>
Aug 09 02:30:19 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:30:19 user nova-compute[72351]:     </video>
Aug 09 02:30:19 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:30:19 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:30:19 user nova-compute[72351]:     </rng>
Aug 09 02:30:19 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:30:19 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:30:19 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:30:19 user nova-compute[72351]:     </memballoon>
Aug 09 02:30:19 user nova-compute[72351]:   </devices>
Aug 09 02:30:19 user nova-compute[72351]: </domain>
Aug 09 02:30:19 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:30:19 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-08-09T02:29:47Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-973345123',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-973345123',id=33,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBC5HMwUPTdKeiDcsJ4TkVOxRLOG1g/yzvRBKkBQKOQuDjCRYdbTKK7gH9+Y0CpEbv5BvHxw4RfGS5kTqYSc0L/jaTiuzJMVzMoXsxa5nneD0wyJcR62H0YaKv6Pgz9BCcQ==',key_name='tempest-keypair-2018857894',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='fb5ff9ee85084536b6479ed16b24293d',ramdisk_id='',reservation_id='r-oubsycpu',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-942602465',owner_user_name='tempest-TaggedBootDevicesTest_v242-942602465-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:29:49Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='76924d471e204a5ebee3c99280e62c91',uuid=38a2b8e3-337d-416a-8a8c-da4921123db6,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "0ebd83d0-5519-4945-8637-90c2ce024ac1", "address": "fa:16:3e:81:a2:62", "network": {"id": "c45d6719-95f9-4856-9b34-9d89c22ff899", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-927662772-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0ebd83d0-55", "ovs_interfaceid": "0ebd83d0-5519-4945-8637-90c2ce024ac1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:30:19 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Converting VIF {"id": "0ebd83d0-5519-4945-8637-90c2ce024ac1", "address": "fa:16:3e:81:a2:62", "network": {"id": "c45d6719-95f9-4856-9b34-9d89c22ff899", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-927662772-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0ebd83d0-55", "ovs_interfaceid": "0ebd83d0-5519-4945-8637-90c2ce024ac1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:30:19 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:81:a2:62,bridge_name='br-int',has_traffic_filtering=True,id=0ebd83d0-5519-4945-8637-90c2ce024ac1,network=Network(c45d6719-95f9-4856-9b34-9d89c22ff899),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap0ebd83d0-55') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:30:19 user nova-compute[72351]: DEBUG os_vif [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:81:a2:62,bridge_name='br-int',has_traffic_filtering=True,id=0ebd83d0-5519-4945-8637-90c2ce024ac1,network=Network(c45d6719-95f9-4856-9b34-9d89c22ff899),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap0ebd83d0-55') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:30:19 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:19 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:30:19 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:30:19 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:19 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap0ebd83d0-55, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:30:19 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap0ebd83d0-55, col_values=(('external_ids', {'iface-id': '0ebd83d0-5519-4945-8637-90c2ce024ac1', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:81:a2:62', 'vm-uuid': '38a2b8e3-337d-416a-8a8c-da4921123db6'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:30:19 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:19 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:30:19 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:19 user nova-compute[72351]: INFO os_vif [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:81:a2:62,bridge_name='br-int',has_traffic_filtering=True,id=0ebd83d0-5519-4945-8637-90c2ce024ac1,network=Network(c45d6719-95f9-4856-9b34-9d89c22ff899),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap0ebd83d0-55')
Aug 09 02:30:19 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-08-09T02:29:47Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-973345123',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-973345123',id=33,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBC5HMwUPTdKeiDcsJ4TkVOxRLOG1g/yzvRBKkBQKOQuDjCRYdbTKK7gH9+Y0CpEbv5BvHxw4RfGS5kTqYSc0L/jaTiuzJMVzMoXsxa5nneD0wyJcR62H0YaKv6Pgz9BCcQ==',key_name='tempest-keypair-2018857894',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='fb5ff9ee85084536b6479ed16b24293d',ramdisk_id='',reservation_id='r-oubsycpu',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-942602465',owner_user_name='tempest-TaggedBootDevicesTest_v242-942602465-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:29:49Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='76924d471e204a5ebee3c99280e62c91',uuid=38a2b8e3-337d-416a-8a8c-da4921123db6,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "8b1245ae-85e9-49c3-9631-a288a172e4c1", "address": "fa:16:3e:a8:7b:e4", "network": {"id": "4617a3d0-dcb5-42a4-85d3-358087aeac69", "bridge": "br-int", "label": "tempest-device-tagging-net1-1844430962", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.34", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8b1245ae-85", "ovs_interfaceid": "8b1245ae-85e9-49c3-9631-a288a172e4c1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:30:19 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Converting VIF {"id": "8b1245ae-85e9-49c3-9631-a288a172e4c1", "address": "fa:16:3e:a8:7b:e4", "network": {"id": "4617a3d0-dcb5-42a4-85d3-358087aeac69", "bridge": "br-int", "label": "tempest-device-tagging-net1-1844430962", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.34", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8b1245ae-85", "ovs_interfaceid": "8b1245ae-85e9-49c3-9631-a288a172e4c1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:30:19 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:a8:7b:e4,bridge_name='br-int',has_traffic_filtering=True,id=8b1245ae-85e9-49c3-9631-a288a172e4c1,network=Network(4617a3d0-dcb5-42a4-85d3-358087aeac69),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap8b1245ae-85') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:30:19 user nova-compute[72351]: DEBUG os_vif [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:a8:7b:e4,bridge_name='br-int',has_traffic_filtering=True,id=8b1245ae-85e9-49c3-9631-a288a172e4c1,network=Network(4617a3d0-dcb5-42a4-85d3-358087aeac69),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap8b1245ae-85') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:30:19 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:19 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:30:19 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:30:19 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:19 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap8b1245ae-85, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:30:19 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap8b1245ae-85, col_values=(('external_ids', {'iface-id': '8b1245ae-85e9-49c3-9631-a288a172e4c1', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:a8:7b:e4', 'vm-uuid': '38a2b8e3-337d-416a-8a8c-da4921123db6'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:20 user nova-compute[72351]: INFO os_vif [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:a8:7b:e4,bridge_name='br-int',has_traffic_filtering=True,id=8b1245ae-85e9-49c3-9631-a288a172e4c1,network=Network(4617a3d0-dcb5-42a4-85d3-358087aeac69),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap8b1245ae-85')
Aug 09 02:30:20 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-08-09T02:29:47Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-973345123',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-973345123',id=33,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBC5HMwUPTdKeiDcsJ4TkVOxRLOG1g/yzvRBKkBQKOQuDjCRYdbTKK7gH9+Y0CpEbv5BvHxw4RfGS5kTqYSc0L/jaTiuzJMVzMoXsxa5nneD0wyJcR62H0YaKv6Pgz9BCcQ==',key_name='tempest-keypair-2018857894',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='fb5ff9ee85084536b6479ed16b24293d',ramdisk_id='',reservation_id='r-oubsycpu',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-942602465',owner_user_name='tempest-TaggedBootDevicesTest_v242-942602465-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:29:49Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='76924d471e204a5ebee3c99280e62c91',uuid=38a2b8e3-337d-416a-8a8c-da4921123db6,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "1be27a59-07a4-4b6a-b25a-ce6d7efac4a3", "address": "fa:16:3e:38:62:75", "network": {"id": "4617a3d0-dcb5-42a4-85d3-358087aeac69", "bridge": "br-int", "label": "tempest-device-tagging-net1-1844430962", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.226", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1be27a59-07", "ovs_interfaceid": "1be27a59-07a4-4b6a-b25a-ce6d7efac4a3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Converting VIF {"id": "1be27a59-07a4-4b6a-b25a-ce6d7efac4a3", "address": "fa:16:3e:38:62:75", "network": {"id": "4617a3d0-dcb5-42a4-85d3-358087aeac69", "bridge": "br-int", "label": "tempest-device-tagging-net1-1844430962", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.226", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1be27a59-07", "ovs_interfaceid": "1be27a59-07a4-4b6a-b25a-ce6d7efac4a3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:38:62:75,bridge_name='br-int',has_traffic_filtering=True,id=1be27a59-07a4-4b6a-b25a-ce6d7efac4a3,network=Network(4617a3d0-dcb5-42a4-85d3-358087aeac69),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap1be27a59-07') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG os_vif [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:38:62:75,bridge_name='br-int',has_traffic_filtering=True,id=1be27a59-07a4-4b6a-b25a-ce6d7efac4a3,network=Network(4617a3d0-dcb5-42a4-85d3-358087aeac69),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap1be27a59-07') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap1be27a59-07, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap1be27a59-07, col_values=(('external_ids', {'iface-id': '1be27a59-07a4-4b6a-b25a-ce6d7efac4a3', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:38:62:75', 'vm-uuid': '38a2b8e3-337d-416a-8a8c-da4921123db6'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:20 user nova-compute[72351]: INFO os_vif [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:38:62:75,bridge_name='br-int',has_traffic_filtering=True,id=1be27a59-07a4-4b6a-b25a-ce6d7efac4a3,network=Network(4617a3d0-dcb5-42a4-85d3-358087aeac69),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap1be27a59-07')
Aug 09 02:30:20 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-08-09T02:29:47Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-973345123',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-973345123',id=33,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBC5HMwUPTdKeiDcsJ4TkVOxRLOG1g/yzvRBKkBQKOQuDjCRYdbTKK7gH9+Y0CpEbv5BvHxw4RfGS5kTqYSc0L/jaTiuzJMVzMoXsxa5nneD0wyJcR62H0YaKv6Pgz9BCcQ==',key_name='tempest-keypair-2018857894',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='fb5ff9ee85084536b6479ed16b24293d',ramdisk_id='',reservation_id='r-oubsycpu',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-942602465',owner_user_name='tempest-TaggedBootDevicesTest_v242-942602465-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:29:49Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='76924d471e204a5ebee3c99280e62c91',uuid=38a2b8e3-337d-416a-8a8c-da4921123db6,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "ccb930f6-9625-450b-b01b-c5009e610ae1", "address": "fa:16:3e:a9:0c:63", "network": {"id": "4617a3d0-dcb5-42a4-85d3-358087aeac69", "bridge": "br-int", "label": "tempest-device-tagging-net1-1844430962", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.237", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapccb930f6-96", "ovs_interfaceid": "ccb930f6-9625-450b-b01b-c5009e610ae1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Converting VIF {"id": "ccb930f6-9625-450b-b01b-c5009e610ae1", "address": "fa:16:3e:a9:0c:63", "network": {"id": "4617a3d0-dcb5-42a4-85d3-358087aeac69", "bridge": "br-int", "label": "tempest-device-tagging-net1-1844430962", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.237", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapccb930f6-96", "ovs_interfaceid": "ccb930f6-9625-450b-b01b-c5009e610ae1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:a9:0c:63,bridge_name='br-int',has_traffic_filtering=True,id=ccb930f6-9625-450b-b01b-c5009e610ae1,network=Network(4617a3d0-dcb5-42a4-85d3-358087aeac69),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapccb930f6-96') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG os_vif [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:a9:0c:63,bridge_name='br-int',has_traffic_filtering=True,id=ccb930f6-9625-450b-b01b-c5009e610ae1,network=Network(4617a3d0-dcb5-42a4-85d3-358087aeac69),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapccb930f6-96') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapccb930f6-96, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapccb930f6-96, col_values=(('external_ids', {'iface-id': 'ccb930f6-9625-450b-b01b-c5009e610ae1', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:a9:0c:63', 'vm-uuid': '38a2b8e3-337d-416a-8a8c-da4921123db6'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:20 user nova-compute[72351]: INFO os_vif [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:a9:0c:63,bridge_name='br-int',has_traffic_filtering=True,id=ccb930f6-9625-450b-b01b-c5009e610ae1,network=Network(4617a3d0-dcb5-42a4-85d3-358087aeac69),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapccb930f6-96')
Aug 09 02:30:20 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-08-09T02:29:47Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-973345123',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-973345123',id=33,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBC5HMwUPTdKeiDcsJ4TkVOxRLOG1g/yzvRBKkBQKOQuDjCRYdbTKK7gH9+Y0CpEbv5BvHxw4RfGS5kTqYSc0L/jaTiuzJMVzMoXsxa5nneD0wyJcR62H0YaKv6Pgz9BCcQ==',key_name='tempest-keypair-2018857894',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='fb5ff9ee85084536b6479ed16b24293d',ramdisk_id='',reservation_id='r-oubsycpu',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-942602465',owner_user_name='tempest-TaggedBootDevicesTest_v242-942602465-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:29:49Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='76924d471e204a5ebee3c99280e62c91',uuid=38a2b8e3-337d-416a-8a8c-da4921123db6,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "d083d9f2-dfe4-4a73-a624-3336f2f02b0d", "address": "fa:16:3e:a0:b1:db", "network": {"id": "4617a3d0-dcb5-42a4-85d3-358087aeac69", "bridge": "br-int", "label": "tempest-device-tagging-net1-1844430962", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.185", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd083d9f2-df", "ovs_interfaceid": "d083d9f2-dfe4-4a73-a624-3336f2f02b0d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Converting VIF {"id": "d083d9f2-dfe4-4a73-a624-3336f2f02b0d", "address": "fa:16:3e:a0:b1:db", "network": {"id": "4617a3d0-dcb5-42a4-85d3-358087aeac69", "bridge": "br-int", "label": "tempest-device-tagging-net1-1844430962", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.185", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd083d9f2-df", "ovs_interfaceid": "d083d9f2-dfe4-4a73-a624-3336f2f02b0d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:a0:b1:db,bridge_name='br-int',has_traffic_filtering=True,id=d083d9f2-dfe4-4a73-a624-3336f2f02b0d,network=Network(4617a3d0-dcb5-42a4-85d3-358087aeac69),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd083d9f2-df') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG os_vif [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:a0:b1:db,bridge_name='br-int',has_traffic_filtering=True,id=d083d9f2-dfe4-4a73-a624-3336f2f02b0d,network=Network(4617a3d0-dcb5-42a4-85d3-358087aeac69),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd083d9f2-df') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapd083d9f2-df, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapd083d9f2-df, col_values=(('external_ids', {'iface-id': 'd083d9f2-dfe4-4a73-a624-3336f2f02b0d', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:a0:b1:db', 'vm-uuid': '38a2b8e3-337d-416a-8a8c-da4921123db6'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:20 user nova-compute[72351]: INFO os_vif [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:a0:b1:db,bridge_name='br-int',has_traffic_filtering=True,id=d083d9f2-dfe4-4a73-a624-3336f2f02b0d,network=Network(4617a3d0-dcb5-42a4-85d3-358087aeac69),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd083d9f2-df')
Aug 09 02:30:20 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-08-09T02:29:47Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-973345123',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-973345123',id=33,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBC5HMwUPTdKeiDcsJ4TkVOxRLOG1g/yzvRBKkBQKOQuDjCRYdbTKK7gH9+Y0CpEbv5BvHxw4RfGS5kTqYSc0L/jaTiuzJMVzMoXsxa5nneD0wyJcR62H0YaKv6Pgz9BCcQ==',key_name='tempest-keypair-2018857894',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='fb5ff9ee85084536b6479ed16b24293d',ramdisk_id='',reservation_id='r-oubsycpu',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-942602465',owner_user_name='tempest-TaggedBootDevicesTest_v242-942602465-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:29:49Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='76924d471e204a5ebee3c99280e62c91',uuid=38a2b8e3-337d-416a-8a8c-da4921123db6,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "a88b872b-9340-4597-8ab4-ea6173dc4e4c", "address": "fa:16:3e:cf:3d:24", "network": {"id": "2984e5a6-dc0d-40ee-8064-f78a7686663e", "bridge": "br-int", "label": "tempest-device-tagging-net2-1630262622", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa88b872b-93", "ovs_interfaceid": "a88b872b-9340-4597-8ab4-ea6173dc4e4c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Converting VIF {"id": "a88b872b-9340-4597-8ab4-ea6173dc4e4c", "address": "fa:16:3e:cf:3d:24", "network": {"id": "2984e5a6-dc0d-40ee-8064-f78a7686663e", "bridge": "br-int", "label": "tempest-device-tagging-net2-1630262622", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa88b872b-93", "ovs_interfaceid": "a88b872b-9340-4597-8ab4-ea6173dc4e4c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:cf:3d:24,bridge_name='br-int',has_traffic_filtering=True,id=a88b872b-9340-4597-8ab4-ea6173dc4e4c,network=Network(2984e5a6-dc0d-40ee-8064-f78a7686663e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa88b872b-93') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG os_vif [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:cf:3d:24,bridge_name='br-int',has_traffic_filtering=True,id=a88b872b-9340-4597-8ab4-ea6173dc4e4c,network=Network(2984e5a6-dc0d-40ee-8064-f78a7686663e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa88b872b-93') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapa88b872b-93, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapa88b872b-93, col_values=(('external_ids', {'iface-id': 'a88b872b-9340-4597-8ab4-ea6173dc4e4c', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:cf:3d:24', 'vm-uuid': '38a2b8e3-337d-416a-8a8c-da4921123db6'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:20 user nova-compute[72351]: INFO os_vif [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:cf:3d:24,bridge_name='br-int',has_traffic_filtering=True,id=a88b872b-9340-4597-8ab4-ea6173dc4e4c,network=Network(2984e5a6-dc0d-40ee-8064-f78a7686663e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa88b872b-93')
Aug 09 02:30:20 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-08-09T02:29:47Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-973345123',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-973345123',id=33,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBC5HMwUPTdKeiDcsJ4TkVOxRLOG1g/yzvRBKkBQKOQuDjCRYdbTKK7gH9+Y0CpEbv5BvHxw4RfGS5kTqYSc0L/jaTiuzJMVzMoXsxa5nneD0wyJcR62H0YaKv6Pgz9BCcQ==',key_name='tempest-keypair-2018857894',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='fb5ff9ee85084536b6479ed16b24293d',ramdisk_id='',reservation_id='r-oubsycpu',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-942602465',owner_user_name='tempest-TaggedBootDevicesTest_v242-942602465-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:29:49Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='76924d471e204a5ebee3c99280e62c91',uuid=38a2b8e3-337d-416a-8a8c-da4921123db6,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "48dfbf61-0142-4478-9ec1-585d5fbe11da", "address": "fa:16:3e:5b:d8:08", "network": {"id": "2984e5a6-dc0d-40ee-8064-f78a7686663e", "bridge": "br-int", "label": "tempest-device-tagging-net2-1630262622", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap48dfbf61-01", "ovs_interfaceid": "48dfbf61-0142-4478-9ec1-585d5fbe11da", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Converting VIF {"id": "48dfbf61-0142-4478-9ec1-585d5fbe11da", "address": "fa:16:3e:5b:d8:08", "network": {"id": "2984e5a6-dc0d-40ee-8064-f78a7686663e", "bridge": "br-int", "label": "tempest-device-tagging-net2-1630262622", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap48dfbf61-01", "ovs_interfaceid": "48dfbf61-0142-4478-9ec1-585d5fbe11da", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:5b:d8:08,bridge_name='br-int',has_traffic_filtering=True,id=48dfbf61-0142-4478-9ec1-585d5fbe11da,network=Network(2984e5a6-dc0d-40ee-8064-f78a7686663e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap48dfbf61-01') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG os_vif [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:5b:d8:08,bridge_name='br-int',has_traffic_filtering=True,id=48dfbf61-0142-4478-9ec1-585d5fbe11da,network=Network(2984e5a6-dc0d-40ee-8064-f78a7686663e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap48dfbf61-01') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap48dfbf61-01, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap48dfbf61-01, col_values=(('external_ids', {'iface-id': '48dfbf61-0142-4478-9ec1-585d5fbe11da', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:5b:d8:08', 'vm-uuid': '38a2b8e3-337d-416a-8a8c-da4921123db6'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:20 user nova-compute[72351]: INFO os_vif [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:5b:d8:08,bridge_name='br-int',has_traffic_filtering=True,id=48dfbf61-0142-4478-9ec1-585d5fbe11da,network=Network(2984e5a6-dc0d-40ee-8064-f78a7686663e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap48dfbf61-01')
Aug 09 02:30:20 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] No BDM found with device name hda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] No BDM found with device name vdc, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] No VIF found with MAC fa:16:3e:81:a2:62, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] No VIF found with MAC fa:16:3e:a0:b1:db, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:30:20 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Using config drive
Aug 09 02:30:20 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Creating config drive at /opt/stack/data/nova/instances/38a2b8e3-337d-416a-8a8c-da4921123db6/disk.config
Aug 09 02:30:20 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Running cmd (subprocess): genisoimage -o /opt/stack/data/nova/instances/38a2b8e3-337d-416a-8a8c-da4921123db6/disk.config -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 0.0.0 -quiet -J -r -V config-2 /tmp/tmpat0n3_v2 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:20 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] CMD "genisoimage -o /opt/stack/data/nova/instances/38a2b8e3-337d-416a-8a8c-da4921123db6/disk.config -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 0.0.0 -quiet -J -r -V config-2 /tmp/tmpat0n3_v2" returned: 0 in 0.057s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:22 user nova-compute[72351]: DEBUG nova.compute.manager [req-be6ca14f-fff3-4599-a4fb-d844bed7ba3e req-429f84b1-1282-4c35-8265-dd74743b6915 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received event network-vif-plugged-0ebd83d0-5519-4945-8637-90c2ce024ac1 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:30:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-be6ca14f-fff3-4599-a4fb-d844bed7ba3e req-429f84b1-1282-4c35-8265-dd74743b6915 service nova] Acquiring lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:30:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-be6ca14f-fff3-4599-a4fb-d844bed7ba3e req-429f84b1-1282-4c35-8265-dd74743b6915 service nova] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:30:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-be6ca14f-fff3-4599-a4fb-d844bed7ba3e req-429f84b1-1282-4c35-8265-dd74743b6915 service nova] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:30:22 user nova-compute[72351]: DEBUG nova.compute.manager [req-be6ca14f-fff3-4599-a4fb-d844bed7ba3e req-429f84b1-1282-4c35-8265-dd74743b6915 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] No waiting events found dispatching network-vif-plugged-0ebd83d0-5519-4945-8637-90c2ce024ac1 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:30:22 user nova-compute[72351]: WARNING nova.compute.manager [req-be6ca14f-fff3-4599-a4fb-d844bed7ba3e req-429f84b1-1282-4c35-8265-dd74743b6915 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received unexpected event network-vif-plugged-0ebd83d0-5519-4945-8637-90c2ce024ac1 for instance with vm_state building and task_state spawning.
Aug 09 02:30:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG nova.compute.manager [req-ede5fd70-6309-4561-b12e-313fd4b32d45 req-26d15b7f-7d4f-44fa-8fce-27d4f375a6b4 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received event network-vif-plugged-8b1245ae-85e9-49c3-9631-a288a172e4c1 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ede5fd70-6309-4561-b12e-313fd4b32d45 req-26d15b7f-7d4f-44fa-8fce-27d4f375a6b4 service nova] Acquiring lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ede5fd70-6309-4561-b12e-313fd4b32d45 req-26d15b7f-7d4f-44fa-8fce-27d4f375a6b4 service nova] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ede5fd70-6309-4561-b12e-313fd4b32d45 req-26d15b7f-7d4f-44fa-8fce-27d4f375a6b4 service nova] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG nova.compute.manager [req-ede5fd70-6309-4561-b12e-313fd4b32d45 req-26d15b7f-7d4f-44fa-8fce-27d4f375a6b4 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] No waiting events found dispatching network-vif-plugged-8b1245ae-85e9-49c3-9631-a288a172e4c1 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:30:23 user nova-compute[72351]: WARNING nova.compute.manager [req-ede5fd70-6309-4561-b12e-313fd4b32d45 req-26d15b7f-7d4f-44fa-8fce-27d4f375a6b4 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received unexpected event network-vif-plugged-8b1245ae-85e9-49c3-9631-a288a172e4c1 for instance with vm_state building and task_state spawning.
Aug 09 02:30:23 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-dab285c2-8b0d-46ad-8db3-440348d0fd27 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-dab285c2-8b0d-46ad-8db3-440348d0fd27 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:3cb561e0-d189-4a63-abe1-3a68ffbe4616'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'GKjCnTvV', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:30:23 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-dab285c2-8b0d-46ad-8db3-440348d0fd27 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:30:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-dab285c2-8b0d-46ad-8db3-440348d0fd27 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-dab285c2-8b0d-46ad-8db3-440348d0fd27 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:30:23 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-dab285c2-8b0d-46ad-8db3-440348d0fd27 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:30:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-dab285c2-8b0d-46ad-8db3-440348d0fd27 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:3cb561e0-d189-4a63-abe1-3a68ffbe4616" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-dab285c2-8b0d-46ad-8db3-440348d0fd27 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:3cb561e0-d189-4a63-abe1-3a68ffbe4616" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:30:23 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-dab285c2-8b0d-46ad-8db3-440348d0fd27 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 02:30:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3cb561e0-d189-4a63-abe1-3a68ffbe4616 -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3cb561e0-d189-4a63-abe1-3a68ffbe4616 -p 172.16.0.220:3260" returned: 21 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[1d75fb60-0c63-4f75-88a6-4dcef52d6b77]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-dab285c2-8b0d-46ad-8db3-440348d0fd27 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Aug 09 02:30:23 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3cb561e0-d189-4a63-abe1-3a68ffbe4616 -p 172.16.0.220:3260 --interface default --op new {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3cb561e0-d189-4a63-abe1-3a68ffbe4616 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[c2c55c33-c6fb-4b0e-93fe-3cbe40fb378c]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:3cb561e0-d189-4a63-abe1-3a68ffbe4616] added\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-dab285c2-8b0d-46ad-8db3-440348d0fd27 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:3cb561e0-d189-4a63-abe1-3a68ffbe4616] added
Aug 09 02:30:23 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3cb561e0-d189-4a63-abe1-3a68ffbe4616 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3cb561e0-d189-4a63-abe1-3a68ffbe4616 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[84cda713-eba9-41a7-b319-1a047e32c76a]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-dab285c2-8b0d-46ad-8db3-440348d0fd27 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3cb561e0-d189-4a63-abe1-3a68ffbe4616 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3cb561e0-d189-4a63-abe1-3a68ffbe4616 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.027s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[8e40a380-89b3-4d95-a22f-ece43d983a95]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-dab285c2-8b0d-46ad-8db3-440348d0fd27 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3cb561e0-d189-4a63-abe1-3a68ffbe4616 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v GKjCnTvV {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3cb561e0-d189-4a63-abe1-3a68ffbe4616 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v GKjCnTvV" returned: 0 in 0.032s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[b39c8023-a4e6-409b-b1d3-26b92387eca2]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-dab285c2-8b0d-46ad-8db3-440348d0fd27 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'GKjCnTvV'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3cb561e0-d189-4a63-abe1-3a68ffbe4616 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3cb561e0-d189-4a63-abe1-3a68ffbe4616 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.033s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[88189eb5-0285-4211-ba59-15a18d6dd73b]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-dab285c2-8b0d-46ad-8db3-440348d0fd27 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.029s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[0ed40b71-0eab-448b-a83a-132a957eb9a3]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e (non-flash)\ntcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-dab285c2-8b0d-46ad-8db3-440348d0fd27 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:30:23 user nova-compute[72351]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 (non-flash)
Aug 09 02:30:23 user nova-compute[72351]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e (non-flash)
Aug 09 02:30:23 user nova-compute[72351]: tcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70 (non-flash)
Aug 09 02:30:23 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-dab285c2-8b0d-46ad-8db3-440348d0fd27 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:30:23 user nova-compute[72351]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 (non-flash)
Aug 09 02:30:23 user nova-compute[72351]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e (non-flash)
Aug 09 02:30:23 user nova-compute[72351]: tcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70 (non-flash)
Aug 09 02:30:23 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3cb561e0-d189-4a63-abe1-3a68ffbe4616 -p 172.16.0.220:3260 --login {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3cb561e0-d189-4a63-abe1-3a68ffbe4616 -p 172.16.0.220:3260 --login" returned: 0 in 0.085s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e59307a9-faf7-4453-be0a-a848502f5d20]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:3cb561e0-d189-4a63-abe1-3a68ffbe4616, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:3cb561e0-d189-4a63-abe1-3a68ffbe4616, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-dab285c2-8b0d-46ad-8db3-440348d0fd27 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:3cb561e0-d189-4a63-abe1-3a68ffbe4616, portal: 172.16.0.220,3260]
Aug 09 02:30:23 user nova-compute[72351]: Login to [iface: default, target: iqn.2010-10.org.openstack:3cb561e0-d189-4a63-abe1-3a68ffbe4616, portal: 172.16.0.220,3260] successful.
Aug 09 02:30:23 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3cb561e0-d189-4a63-abe1-3a68ffbe4616 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3cb561e0-d189-4a63-abe1-3a68ffbe4616 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.034s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[14fd0995-39b2-449a-8d81-cef433aa1f8e]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-dab285c2-8b0d-46ad-8db3-440348d0fd27 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.028s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a4c388b2-2c95-4893-bdfa-d71eb0955b5c]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e (non-flash)\ntcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70 (non-flash)\ntcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3cb561e0-d189-4a63-abe1-3a68ffbe4616 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-dab285c2-8b0d-46ad-8db3-440348d0fd27 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:30:23 user nova-compute[72351]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 (non-flash)
Aug 09 02:30:23 user nova-compute[72351]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e (non-flash)
Aug 09 02:30:23 user nova-compute[72351]: tcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70 (non-flash)
Aug 09 02:30:23 user nova-compute[72351]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3cb561e0-d189-4a63-abe1-3a68ffbe4616 (non-flash)
Aug 09 02:30:23 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-dab285c2-8b0d-46ad-8db3-440348d0fd27 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:30:23 user nova-compute[72351]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 (non-flash)
Aug 09 02:30:23 user nova-compute[72351]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e (non-flash)
Aug 09 02:30:23 user nova-compute[72351]: tcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70 (non-flash)
Aug 09 02:30:23 user nova-compute[72351]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3cb561e0-d189-4a63-abe1-3a68ffbe4616 (non-flash)
Aug 09 02:30:23 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-dab285c2-8b0d-46ad-8db3-440348d0fd27 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:3cb561e0-d189-4a63-abe1-3a68ffbe4616" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.389s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-dab285c2-8b0d-46ad-8db3-440348d0fd27 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-dab285c2-8b0d-46ad-8db3-440348d0fd27 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] HCTL ('36', '-', '-', 0) found on session 36 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-dab285c2-8b0d-46ad-8db3-440348d0fd27 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Scanning host 36 c: -, t: -, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host36/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host36/scan" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[550569b3-368f-479b-8119-b0a004521a24]: (4, ('- - 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-dab285c2-8b0d-46ad-8db3-440348d0fd27 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Searching for a device in session 36 and hctl ['36', '*', '*', 0] yield: sde {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-dab285c2-8b0d-46ad-8db3-440348d0fd27 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Connected to sde using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'GKjCnTvV', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:23 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:24 user nova-compute[72351]: DEBUG nova.compute.manager [req-202bef3f-7fbe-49ab-9735-978829b8f31f req-040fd38d-956f-4543-b88b-047c50a80a1a service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received event network-vif-plugged-48dfbf61-0142-4478-9ec1-585d5fbe11da {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:30:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-202bef3f-7fbe-49ab-9735-978829b8f31f req-040fd38d-956f-4543-b88b-047c50a80a1a service nova] Acquiring lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:30:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-202bef3f-7fbe-49ab-9735-978829b8f31f req-040fd38d-956f-4543-b88b-047c50a80a1a service nova] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:30:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-202bef3f-7fbe-49ab-9735-978829b8f31f req-040fd38d-956f-4543-b88b-047c50a80a1a service nova] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:30:24 user nova-compute[72351]: DEBUG nova.compute.manager [req-202bef3f-7fbe-49ab-9735-978829b8f31f req-040fd38d-956f-4543-b88b-047c50a80a1a service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] No waiting events found dispatching network-vif-plugged-48dfbf61-0142-4478-9ec1-585d5fbe11da {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:30:24 user nova-compute[72351]: WARNING nova.compute.manager [req-202bef3f-7fbe-49ab-9735-978829b8f31f req-040fd38d-956f-4543-b88b-047c50a80a1a service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received unexpected event network-vif-plugged-48dfbf61-0142-4478-9ec1-585d5fbe11da for instance with vm_state building and task_state spawning.
Aug 09 02:30:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:24 user nova-compute[72351]: DEBUG nova.compute.manager [req-3a7cee9d-71d2-4093-94fb-f1caa4ba6653 req-225ac73c-0a50-483f-ba90-48145a858863 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received event network-vif-plugged-0ebd83d0-5519-4945-8637-90c2ce024ac1 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:30:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-3a7cee9d-71d2-4093-94fb-f1caa4ba6653 req-225ac73c-0a50-483f-ba90-48145a858863 service nova] Acquiring lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:30:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-3a7cee9d-71d2-4093-94fb-f1caa4ba6653 req-225ac73c-0a50-483f-ba90-48145a858863 service nova] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:30:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-3a7cee9d-71d2-4093-94fb-f1caa4ba6653 req-225ac73c-0a50-483f-ba90-48145a858863 service nova] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:30:24 user nova-compute[72351]: DEBUG nova.compute.manager [req-3a7cee9d-71d2-4093-94fb-f1caa4ba6653 req-225ac73c-0a50-483f-ba90-48145a858863 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] No waiting events found dispatching network-vif-plugged-0ebd83d0-5519-4945-8637-90c2ce024ac1 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:30:24 user nova-compute[72351]: WARNING nova.compute.manager [req-3a7cee9d-71d2-4093-94fb-f1caa4ba6653 req-225ac73c-0a50-483f-ba90-48145a858863 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received unexpected event network-vif-plugged-0ebd83d0-5519-4945-8637-90c2ce024ac1 for instance with vm_state building and task_state spawning.
Aug 09 02:30:24 user nova-compute[72351]: DEBUG nova.compute.manager [req-3a7cee9d-71d2-4093-94fb-f1caa4ba6653 req-225ac73c-0a50-483f-ba90-48145a858863 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received event network-vif-plugged-1be27a59-07a4-4b6a-b25a-ce6d7efac4a3 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:30:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-3a7cee9d-71d2-4093-94fb-f1caa4ba6653 req-225ac73c-0a50-483f-ba90-48145a858863 service nova] Acquiring lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:30:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-3a7cee9d-71d2-4093-94fb-f1caa4ba6653 req-225ac73c-0a50-483f-ba90-48145a858863 service nova] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:30:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-3a7cee9d-71d2-4093-94fb-f1caa4ba6653 req-225ac73c-0a50-483f-ba90-48145a858863 service nova] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:30:24 user nova-compute[72351]: DEBUG nova.compute.manager [req-3a7cee9d-71d2-4093-94fb-f1caa4ba6653 req-225ac73c-0a50-483f-ba90-48145a858863 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] No waiting events found dispatching network-vif-plugged-1be27a59-07a4-4b6a-b25a-ce6d7efac4a3 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:30:24 user nova-compute[72351]: WARNING nova.compute.manager [req-3a7cee9d-71d2-4093-94fb-f1caa4ba6653 req-225ac73c-0a50-483f-ba90-48145a858863 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received unexpected event network-vif-plugged-1be27a59-07a4-4b6a-b25a-ce6d7efac4a3 for instance with vm_state building and task_state spawning.
Aug 09 02:30:24 user nova-compute[72351]: DEBUG nova.compute.manager [req-3a7cee9d-71d2-4093-94fb-f1caa4ba6653 req-225ac73c-0a50-483f-ba90-48145a858863 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received event network-vif-plugged-1be27a59-07a4-4b6a-b25a-ce6d7efac4a3 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:30:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-3a7cee9d-71d2-4093-94fb-f1caa4ba6653 req-225ac73c-0a50-483f-ba90-48145a858863 service nova] Acquiring lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:30:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-3a7cee9d-71d2-4093-94fb-f1caa4ba6653 req-225ac73c-0a50-483f-ba90-48145a858863 service nova] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:30:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-3a7cee9d-71d2-4093-94fb-f1caa4ba6653 req-225ac73c-0a50-483f-ba90-48145a858863 service nova] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.003s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:30:24 user nova-compute[72351]: DEBUG nova.compute.manager [req-3a7cee9d-71d2-4093-94fb-f1caa4ba6653 req-225ac73c-0a50-483f-ba90-48145a858863 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] No waiting events found dispatching network-vif-plugged-1be27a59-07a4-4b6a-b25a-ce6d7efac4a3 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:30:24 user nova-compute[72351]: WARNING nova.compute.manager [req-3a7cee9d-71d2-4093-94fb-f1caa4ba6653 req-225ac73c-0a50-483f-ba90-48145a858863 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received unexpected event network-vif-plugged-1be27a59-07a4-4b6a-b25a-ce6d7efac4a3 for instance with vm_state building and task_state spawning.
Aug 09 02:30:24 user nova-compute[72351]: DEBUG nova.compute.manager [req-3a7cee9d-71d2-4093-94fb-f1caa4ba6653 req-225ac73c-0a50-483f-ba90-48145a858863 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received event network-vif-plugged-ccb930f6-9625-450b-b01b-c5009e610ae1 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:30:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-3a7cee9d-71d2-4093-94fb-f1caa4ba6653 req-225ac73c-0a50-483f-ba90-48145a858863 service nova] Acquiring lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:30:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-3a7cee9d-71d2-4093-94fb-f1caa4ba6653 req-225ac73c-0a50-483f-ba90-48145a858863 service nova] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:30:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-3a7cee9d-71d2-4093-94fb-f1caa4ba6653 req-225ac73c-0a50-483f-ba90-48145a858863 service nova] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:30:24 user nova-compute[72351]: DEBUG nova.compute.manager [req-3a7cee9d-71d2-4093-94fb-f1caa4ba6653 req-225ac73c-0a50-483f-ba90-48145a858863 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] No waiting events found dispatching network-vif-plugged-ccb930f6-9625-450b-b01b-c5009e610ae1 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:30:24 user nova-compute[72351]: WARNING nova.compute.manager [req-3a7cee9d-71d2-4093-94fb-f1caa4ba6653 req-225ac73c-0a50-483f-ba90-48145a858863 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received unexpected event network-vif-plugged-ccb930f6-9625-450b-b01b-c5009e610ae1 for instance with vm_state building and task_state spawning.
Aug 09 02:30:24 user nova-compute[72351]: DEBUG nova.compute.manager [req-3a7cee9d-71d2-4093-94fb-f1caa4ba6653 req-225ac73c-0a50-483f-ba90-48145a858863 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received event network-vif-plugged-ccb930f6-9625-450b-b01b-c5009e610ae1 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:30:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-3a7cee9d-71d2-4093-94fb-f1caa4ba6653 req-225ac73c-0a50-483f-ba90-48145a858863 service nova] Acquiring lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:30:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-3a7cee9d-71d2-4093-94fb-f1caa4ba6653 req-225ac73c-0a50-483f-ba90-48145a858863 service nova] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:30:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-3a7cee9d-71d2-4093-94fb-f1caa4ba6653 req-225ac73c-0a50-483f-ba90-48145a858863 service nova] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:30:24 user nova-compute[72351]: DEBUG nova.compute.manager [req-3a7cee9d-71d2-4093-94fb-f1caa4ba6653 req-225ac73c-0a50-483f-ba90-48145a858863 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] No waiting events found dispatching network-vif-plugged-ccb930f6-9625-450b-b01b-c5009e610ae1 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:30:24 user nova-compute[72351]: WARNING nova.compute.manager [req-3a7cee9d-71d2-4093-94fb-f1caa4ba6653 req-225ac73c-0a50-483f-ba90-48145a858863 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received unexpected event network-vif-plugged-ccb930f6-9625-450b-b01b-c5009e610ae1 for instance with vm_state building and task_state spawning.
Aug 09 02:30:24 user nova-compute[72351]: DEBUG nova.compute.manager [req-3a7cee9d-71d2-4093-94fb-f1caa4ba6653 req-225ac73c-0a50-483f-ba90-48145a858863 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received event network-vif-plugged-d083d9f2-dfe4-4a73-a624-3336f2f02b0d {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:30:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-3a7cee9d-71d2-4093-94fb-f1caa4ba6653 req-225ac73c-0a50-483f-ba90-48145a858863 service nova] Acquiring lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:30:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-3a7cee9d-71d2-4093-94fb-f1caa4ba6653 req-225ac73c-0a50-483f-ba90-48145a858863 service nova] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:30:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-3a7cee9d-71d2-4093-94fb-f1caa4ba6653 req-225ac73c-0a50-483f-ba90-48145a858863 service nova] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:30:24 user nova-compute[72351]: DEBUG nova.compute.manager [req-3a7cee9d-71d2-4093-94fb-f1caa4ba6653 req-225ac73c-0a50-483f-ba90-48145a858863 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] No waiting events found dispatching network-vif-plugged-d083d9f2-dfe4-4a73-a624-3336f2f02b0d {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:30:24 user nova-compute[72351]: WARNING nova.compute.manager [req-3a7cee9d-71d2-4093-94fb-f1caa4ba6653 req-225ac73c-0a50-483f-ba90-48145a858863 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received unexpected event network-vif-plugged-d083d9f2-dfe4-4a73-a624-3336f2f02b0d for instance with vm_state building and task_state spawning.
Aug 09 02:30:24 user nova-compute[72351]: DEBUG nova.compute.manager [req-3a7cee9d-71d2-4093-94fb-f1caa4ba6653 req-225ac73c-0a50-483f-ba90-48145a858863 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received event network-vif-plugged-d083d9f2-dfe4-4a73-a624-3336f2f02b0d {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:30:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-3a7cee9d-71d2-4093-94fb-f1caa4ba6653 req-225ac73c-0a50-483f-ba90-48145a858863 service nova] Acquiring lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:30:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-3a7cee9d-71d2-4093-94fb-f1caa4ba6653 req-225ac73c-0a50-483f-ba90-48145a858863 service nova] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:30:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-3a7cee9d-71d2-4093-94fb-f1caa4ba6653 req-225ac73c-0a50-483f-ba90-48145a858863 service nova] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:30:24 user nova-compute[72351]: DEBUG nova.compute.manager [req-3a7cee9d-71d2-4093-94fb-f1caa4ba6653 req-225ac73c-0a50-483f-ba90-48145a858863 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] No waiting events found dispatching network-vif-plugged-d083d9f2-dfe4-4a73-a624-3336f2f02b0d {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:30:24 user nova-compute[72351]: WARNING nova.compute.manager [req-3a7cee9d-71d2-4093-94fb-f1caa4ba6653 req-225ac73c-0a50-483f-ba90-48145a858863 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received unexpected event network-vif-plugged-d083d9f2-dfe4-4a73-a624-3336f2f02b0d for instance with vm_state building and task_state spawning.
Aug 09 02:30:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:24 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-dab285c2-8b0d-46ad-8db3-440348d0fd27 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.451s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:30:24 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-dab285c2-8b0d-46ad-8db3-440348d0fd27 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] <== connect_volume: return (1452ms) {'type': 'block', 'scsi_wwn': '26534333464666430', 'path': '/dev/sde'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:30:24 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-dab285c2-8b0d-46ad-8db3-440348d0fd27 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26534333464666430', 'path': '/dev/sde'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 02:30:24 user nova-compute[72351]: DEBUG nova.objects.instance [None req-dab285c2-8b0d-46ad-8db3-440348d0fd27 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lazy-loading 'flavor' on Instance uuid 9df16e46-d097-4553-b44c-732ea24a4169 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:30:24 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-dab285c2-8b0d-46ad-8db3-440348d0fd27 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] attach device xml: <disk type="block" device="disk">
Aug 09 02:30:24 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:30:24 user nova-compute[72351]:   <source dev="/dev/sde"/>
Aug 09 02:30:24 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:30:24 user nova-compute[72351]:   <serial>3cb561e0-d189-4a63-abe1-3a68ffbe4616</serial>
Aug 09 02:30:24 user nova-compute[72351]: </disk>
Aug 09 02:30:24 user nova-compute[72351]:  {{(pid=72351) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Aug 09 02:30:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:25 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-dab285c2-8b0d-46ad-8db3-440348d0fd27 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:30:25 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-dab285c2-8b0d-46ad-8db3-440348d0fd27 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:30:25 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-dab285c2-8b0d-46ad-8db3-440348d0fd27 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] No VIF found with MAC fa:16:3e:69:6a:fa, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:30:25 user nova-compute[72351]: DEBUG nova.compute.manager [req-ef4c851b-0cf7-48bd-8dda-eebd9966113a req-3cc12f8e-8c5d-41ab-afbd-38f90a62c9b6 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received event network-vif-plugged-8b1245ae-85e9-49c3-9631-a288a172e4c1 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:30:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ef4c851b-0cf7-48bd-8dda-eebd9966113a req-3cc12f8e-8c5d-41ab-afbd-38f90a62c9b6 service nova] Acquiring lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:30:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ef4c851b-0cf7-48bd-8dda-eebd9966113a req-3cc12f8e-8c5d-41ab-afbd-38f90a62c9b6 service nova] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:30:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ef4c851b-0cf7-48bd-8dda-eebd9966113a req-3cc12f8e-8c5d-41ab-afbd-38f90a62c9b6 service nova] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:30:25 user nova-compute[72351]: DEBUG nova.compute.manager [req-ef4c851b-0cf7-48bd-8dda-eebd9966113a req-3cc12f8e-8c5d-41ab-afbd-38f90a62c9b6 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] No waiting events found dispatching network-vif-plugged-8b1245ae-85e9-49c3-9631-a288a172e4c1 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:30:25 user nova-compute[72351]: WARNING nova.compute.manager [req-ef4c851b-0cf7-48bd-8dda-eebd9966113a req-3cc12f8e-8c5d-41ab-afbd-38f90a62c9b6 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received unexpected event network-vif-plugged-8b1245ae-85e9-49c3-9631-a288a172e4c1 for instance with vm_state building and task_state spawning.
Aug 09 02:30:25 user nova-compute[72351]: DEBUG nova.compute.manager [req-ef4c851b-0cf7-48bd-8dda-eebd9966113a req-3cc12f8e-8c5d-41ab-afbd-38f90a62c9b6 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received event network-vif-plugged-a88b872b-9340-4597-8ab4-ea6173dc4e4c {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:30:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ef4c851b-0cf7-48bd-8dda-eebd9966113a req-3cc12f8e-8c5d-41ab-afbd-38f90a62c9b6 service nova] Acquiring lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:30:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ef4c851b-0cf7-48bd-8dda-eebd9966113a req-3cc12f8e-8c5d-41ab-afbd-38f90a62c9b6 service nova] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:30:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ef4c851b-0cf7-48bd-8dda-eebd9966113a req-3cc12f8e-8c5d-41ab-afbd-38f90a62c9b6 service nova] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:30:25 user nova-compute[72351]: DEBUG nova.compute.manager [req-ef4c851b-0cf7-48bd-8dda-eebd9966113a req-3cc12f8e-8c5d-41ab-afbd-38f90a62c9b6 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] No waiting events found dispatching network-vif-plugged-a88b872b-9340-4597-8ab4-ea6173dc4e4c {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:30:25 user nova-compute[72351]: WARNING nova.compute.manager [req-ef4c851b-0cf7-48bd-8dda-eebd9966113a req-3cc12f8e-8c5d-41ab-afbd-38f90a62c9b6 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received unexpected event network-vif-plugged-a88b872b-9340-4597-8ab4-ea6173dc4e4c for instance with vm_state building and task_state spawning.
Aug 09 02:30:25 user nova-compute[72351]: DEBUG nova.compute.manager [req-ef4c851b-0cf7-48bd-8dda-eebd9966113a req-3cc12f8e-8c5d-41ab-afbd-38f90a62c9b6 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received event network-vif-plugged-a88b872b-9340-4597-8ab4-ea6173dc4e4c {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:30:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ef4c851b-0cf7-48bd-8dda-eebd9966113a req-3cc12f8e-8c5d-41ab-afbd-38f90a62c9b6 service nova] Acquiring lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:30:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ef4c851b-0cf7-48bd-8dda-eebd9966113a req-3cc12f8e-8c5d-41ab-afbd-38f90a62c9b6 service nova] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:30:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ef4c851b-0cf7-48bd-8dda-eebd9966113a req-3cc12f8e-8c5d-41ab-afbd-38f90a62c9b6 service nova] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:30:25 user nova-compute[72351]: DEBUG nova.compute.manager [req-ef4c851b-0cf7-48bd-8dda-eebd9966113a req-3cc12f8e-8c5d-41ab-afbd-38f90a62c9b6 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] No waiting events found dispatching network-vif-plugged-a88b872b-9340-4597-8ab4-ea6173dc4e4c {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:30:25 user nova-compute[72351]: WARNING nova.compute.manager [req-ef4c851b-0cf7-48bd-8dda-eebd9966113a req-3cc12f8e-8c5d-41ab-afbd-38f90a62c9b6 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received unexpected event network-vif-plugged-a88b872b-9340-4597-8ab4-ea6173dc4e4c for instance with vm_state building and task_state spawning.
Aug 09 02:30:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-dab285c2-8b0d-46ad-8db3-440348d0fd27 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lock "9df16e46-d097-4553-b44c-732ea24a4169" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 11.622s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:30:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:26 user nova-compute[72351]: DEBUG nova.compute.manager [req-8bd6bb77-d96e-4975-9f18-e9b83aaf2a23 req-c16125b6-6cb4-4f0c-bbd9-78c88b57437c service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received event network-vif-plugged-48dfbf61-0142-4478-9ec1-585d5fbe11da {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:30:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-8bd6bb77-d96e-4975-9f18-e9b83aaf2a23 req-c16125b6-6cb4-4f0c-bbd9-78c88b57437c service nova] Acquiring lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:30:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-8bd6bb77-d96e-4975-9f18-e9b83aaf2a23 req-c16125b6-6cb4-4f0c-bbd9-78c88b57437c service nova] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:30:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-8bd6bb77-d96e-4975-9f18-e9b83aaf2a23 req-c16125b6-6cb4-4f0c-bbd9-78c88b57437c service nova] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:30:26 user nova-compute[72351]: DEBUG nova.compute.manager [req-8bd6bb77-d96e-4975-9f18-e9b83aaf2a23 req-c16125b6-6cb4-4f0c-bbd9-78c88b57437c service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] No waiting events found dispatching network-vif-plugged-48dfbf61-0142-4478-9ec1-585d5fbe11da {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:30:26 user nova-compute[72351]: WARNING nova.compute.manager [req-8bd6bb77-d96e-4975-9f18-e9b83aaf2a23 req-c16125b6-6cb4-4f0c-bbd9-78c88b57437c service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received unexpected event network-vif-plugged-48dfbf61-0142-4478-9ec1-585d5fbe11da for instance with vm_state building and task_state spawning.
Aug 09 02:30:26 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548226.4919496, 38a2b8e3-337d-416a-8a8c-da4921123db6 => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:30:26 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] VM Resumed (Lifecycle Event)
Aug 09 02:30:26 user nova-compute[72351]: DEBUG nova.compute.manager [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:30:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Guest created on hypervisor {{(pid=72351) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4411}}
Aug 09 02:30:26 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Instance spawned successfully.
Aug 09 02:30:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:894}}
Aug 09 02:30:26 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:30:26 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:30:26 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:30:26 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548226.4962955, 38a2b8e3-337d-416a-8a8c-da4921123db6 => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:30:26 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] VM Started (Lifecycle Event)
Aug 09 02:30:26 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:30:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Found default for hw_cdrom_bus of ide {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:30:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Found default for hw_disk_bus of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:30:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Found default for hw_input_bus of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:30:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Found default for hw_pointer_model of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:30:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Found default for hw_video_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:30:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Found default for hw_vif_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:30:26 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:30:26 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:30:26 user nova-compute[72351]: INFO nova.compute.manager [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Took 10.89 seconds to spawn the instance on the hypervisor.
Aug 09 02:30:26 user nova-compute[72351]: DEBUG nova.compute.manager [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:30:26 user nova-compute[72351]: INFO nova.compute.manager [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Took 38.49 seconds to build instance.
Aug 09 02:30:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-702d2ece-df27-4023-9f25-0ff4cba00e35 tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 38.579s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:30:28 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:28 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:30 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:33 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:35 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:36 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:38 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:38 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquiring lock "49a10075-0d3c-4cd6-bb43-10238cad3f37" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:30:38 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "49a10075-0d3c-4cd6-bb43-10238cad3f37" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:30:38 user nova-compute[72351]: DEBUG nova.compute.manager [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Starting instance... {{(pid=72351) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2406}}
Aug 09 02:30:38 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:30:38 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:30:38 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72351) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Aug 09 02:30:38 user nova-compute[72351]: INFO nova.compute.claims [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Claim successful on node user
Aug 09 02:30:39 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:30:39 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:30:39 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.754s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:30:39 user nova-compute[72351]: DEBUG nova.compute.manager [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Start building networks asynchronously for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2803}}
Aug 09 02:30:39 user nova-compute[72351]: DEBUG nova.compute.manager [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Allocating IP information in the background. {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1956}}
Aug 09 02:30:39 user nova-compute[72351]: DEBUG nova.network.neutron [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] allocate_for_instance() {{(pid=72351) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Aug 09 02:30:39 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Aug 09 02:30:39 user nova-compute[72351]: DEBUG nova.compute.manager [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Start building block device mappings for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2838}}
Aug 09 02:30:39 user nova-compute[72351]: DEBUG nova.policy [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '07d8d5a634274df19c2db769d2d27b29', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4c6bbfd8d904ba1ac69ba5da4564513', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72351) authorize /opt/stack/nova/nova/policy.py:203}}
Aug 09 02:30:39 user nova-compute[72351]: INFO nova.virt.block_device [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Booting with volume 4113a563-77a5-4b68-9198-9d0434737448 at /dev/vda
Aug 09 02:30:39 user nova-compute[72351]: DEBUG os_brick.utils [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:30:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.016s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:39 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[8abddceb-da50-4a74-8f93-255d2c812207]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:a52269602ff4\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:39 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[36f6be1d-72eb-48b7-9cbf-426b7d4a0688]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:39 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[4ce5d229-3bc6-44ed-93db-7cc91caac8ce]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:39 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[4ce71ab1-602a-4cd4-88c8-71afcde8a1fb]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Running cmd (subprocess): nvme version {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] 'nvme version' failed. Not Retrying. {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:30:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.nvmeof [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] nvme not present on system {{(pid=72351) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Aug 09 02:30:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:30:39 user nova-compute[72351]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Aug 09 02:30:39 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[11e90cfe-5607-4f1b-936c-e472b12ad3c8]: (4, '') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72351) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Aug 09 02:30:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Aug 09 02:30:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] LIGHTOS: no hostnqn found. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Aug 09 02:30:39 user nova-compute[72351]: DEBUG os_brick.utils [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] <== get_connector_properties: return (153ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:a52269602ff4', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:30:39 user nova-compute[72351]: DEBUG nova.virt.block_device [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Updating existing volume attachment record: 2644fb08-2d89-40eb-a122-337cd6a3cfb9 {{(pid=72351) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Aug 09 02:30:40 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:40 user nova-compute[72351]: DEBUG nova.network.neutron [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Successfully created port: 2dcfe7b8-34fe-4860-b820-f3de7c00df62 {{(pid=72351) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Aug 09 02:30:40 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:41 user nova-compute[72351]: DEBUG nova.network.neutron [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Successfully updated port: 2dcfe7b8-34fe-4860-b820-f3de7c00df62 {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:30:41 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquiring lock "refresh_cache-49a10075-0d3c-4cd6-bb43-10238cad3f37" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:30:41 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquired lock "refresh_cache-49a10075-0d3c-4cd6-bb43-10238cad3f37" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:30:41 user nova-compute[72351]: DEBUG nova.network.neutron [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:30:41 user nova-compute[72351]: DEBUG nova.compute.manager [req-14d8cece-7c8b-4479-99ba-6e6588ab2434 req-b6bf479e-b342-4a66-88c0-11524bb9dd20 service nova] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Received event network-changed-2dcfe7b8-34fe-4860-b820-f3de7c00df62 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:30:41 user nova-compute[72351]: DEBUG nova.compute.manager [req-14d8cece-7c8b-4479-99ba-6e6588ab2434 req-b6bf479e-b342-4a66-88c0-11524bb9dd20 service nova] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Refreshing instance network info cache due to event network-changed-2dcfe7b8-34fe-4860-b820-f3de7c00df62. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:30:41 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-14d8cece-7c8b-4479-99ba-6e6588ab2434 req-b6bf479e-b342-4a66-88c0-11524bb9dd20 service nova] Acquiring lock "refresh_cache-49a10075-0d3c-4cd6-bb43-10238cad3f37" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:30:41 user nova-compute[72351]: DEBUG nova.network.neutron [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:30:41 user nova-compute[72351]: DEBUG nova.network.neutron [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Updating instance_info_cache with network_info: [{"id": "2dcfe7b8-34fe-4860-b820-f3de7c00df62", "address": "fa:16:3e:01:0f:c9", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2dcfe7b8-34", "ovs_interfaceid": "2dcfe7b8-34fe-4860-b820-f3de7c00df62", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:30:41 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Releasing lock "refresh_cache-49a10075-0d3c-4cd6-bb43-10238cad3f37" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:30:41 user nova-compute[72351]: DEBUG nova.compute.manager [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Instance network_info: |[{"id": "2dcfe7b8-34fe-4860-b820-f3de7c00df62", "address": "fa:16:3e:01:0f:c9", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2dcfe7b8-34", "ovs_interfaceid": "2dcfe7b8-34fe-4860-b820-f3de7c00df62", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1971}}
Aug 09 02:30:41 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-14d8cece-7c8b-4479-99ba-6e6588ab2434 req-b6bf479e-b342-4a66-88c0-11524bb9dd20 service nova] Acquired lock "refresh_cache-49a10075-0d3c-4cd6-bb43-10238cad3f37" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:30:41 user nova-compute[72351]: DEBUG nova.network.neutron [req-14d8cece-7c8b-4479-99ba-6e6588ab2434 req-b6bf479e-b342-4a66-88c0-11524bb9dd20 service nova] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Refreshing network info cache for port 2dcfe7b8-34fe-4860-b820-f3de7c00df62 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:30:42 user nova-compute[72351]: DEBUG nova.compute.manager [None req-a729067e-a142-4972-b4f6-b7cae9769588 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:30:42 user nova-compute[72351]: INFO nova.compute.manager [None req-a729067e-a142-4972-b4f6-b7cae9769588 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: a03fbded-23ad-440b-97e7-bea07425e873] instance snapshotting
Aug 09 02:30:42 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-a729067e-a142-4972-b4f6-b7cae9769588 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Beginning live snapshot process
Aug 09 02:30:42 user nova-compute[72351]: DEBUG nova.network.neutron [req-14d8cece-7c8b-4479-99ba-6e6588ab2434 req-b6bf479e-b342-4a66-88c0-11524bb9dd20 service nova] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Updated VIF entry in instance network info cache for port 2dcfe7b8-34fe-4860-b820-f3de7c00df62. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:30:42 user nova-compute[72351]: DEBUG nova.network.neutron [req-14d8cece-7c8b-4479-99ba-6e6588ab2434 req-b6bf479e-b342-4a66-88c0-11524bb9dd20 service nova] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Updating instance_info_cache with network_info: [{"id": "2dcfe7b8-34fe-4860-b820-f3de7c00df62", "address": "fa:16:3e:01:0f:c9", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2dcfe7b8-34", "ovs_interfaceid": "2dcfe7b8-34fe-4860-b820-f3de7c00df62", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:30:42 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-14d8cece-7c8b-4479-99ba-6e6588ab2434 req-b6bf479e-b342-4a66-88c0-11524bb9dd20 service nova] Releasing lock "refresh_cache-49a10075-0d3c-4cd6-bb43-10238cad3f37" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:30:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-a729067e-a142-4972-b4f6-b7cae9769588 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a03fbded-23ad-440b-97e7-bea07425e873/disk --force-share --output=json -f qcow2 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-a729067e-a142-4972-b4f6-b7cae9769588 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a03fbded-23ad-440b-97e7-bea07425e873/disk --force-share --output=json -f qcow2" returned: 0 in 0.144s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-a729067e-a142-4972-b4f6-b7cae9769588 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a03fbded-23ad-440b-97e7-bea07425e873/disk --force-share --output=json -f qcow2 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-a729067e-a142-4972-b4f6-b7cae9769588 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a03fbded-23ad-440b-97e7-bea07425e873/disk --force-share --output=json -f qcow2" returned: 0 in 0.150s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-a729067e-a142-4972-b4f6-b7cae9769588 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-a729067e-a142-4972-b4f6-b7cae9769588 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.146s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-a729067e-a142-4972-b4f6-b7cae9769588 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/snapshots/tmpuvw3wcky/2d650c4618254e96ae17fd167a9f67b2.delta 1073741824 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-a729067e-a142-4972-b4f6-b7cae9769588 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/snapshots/tmpuvw3wcky/2d650c4618254e96ae17fd167a9f67b2.delta 1073741824" returned: 0 in 0.062s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:42 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-a729067e-a142-4972-b4f6-b7cae9769588 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Quiescing instance not available: QEMU guest agent is not enabled.
Aug 09 02:30:43 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:43 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-a729067e-a142-4972-b4f6-b7cae9769588 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] COPY block job progress, current cursor: 0 final cursor: 75563008 {{(pid=72351) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Aug 09 02:30:44 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-a729067e-a142-4972-b4f6-b7cae9769588 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] COPY block job progress, current cursor: 17104896 final cursor: 75563008 {{(pid=72351) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Aug 09 02:30:44 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-a729067e-a142-4972-b4f6-b7cae9769588 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] COPY block job progress, current cursor: 75563008 final cursor: 75563008 {{(pid=72351) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Aug 09 02:30:44 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-a729067e-a142-4972-b4f6-b7cae9769588 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Skipping quiescing instance: QEMU guest agent is not enabled.
Aug 09 02:30:45 user nova-compute[72351]: DEBUG nova.privsep.utils [None req-a729067e-a142-4972-b4f6-b7cae9769588 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=72351) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Aug 09 02:30:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-a729067e-a142-4972-b4f6-b7cae9769588 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Running cmd (subprocess): qemu-img convert -t none -O qcow2 -f qcow2 /opt/stack/data/nova/instances/snapshots/tmpuvw3wcky/2d650c4618254e96ae17fd167a9f67b2.delta /opt/stack/data/nova/instances/snapshots/tmpuvw3wcky/2d650c4618254e96ae17fd167a9f67b2 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:45 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:45 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-a729067e-a142-4972-b4f6-b7cae9769588 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] CMD "qemu-img convert -t none -O qcow2 -f qcow2 /opt/stack/data/nova/instances/snapshots/tmpuvw3wcky/2d650c4618254e96ae17fd167a9f67b2.delta /opt/stack/data/nova/instances/snapshots/tmpuvw3wcky/2d650c4618254e96ae17fd167a9f67b2" returned: 0 in 0.773s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:45 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-a729067e-a142-4972-b4f6-b7cae9769588 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Snapshot extracted, beginning image upload
Aug 09 02:30:48 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:48 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-a729067e-a142-4972-b4f6-b7cae9769588 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Snapshot image upload complete
Aug 09 02:30:48 user nova-compute[72351]: INFO nova.compute.manager [None req-a729067e-a142-4972-b4f6-b7cae9769588 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Took 6.73 seconds to snapshot the instance on the hypervisor.
Aug 09 02:30:48 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:50 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:51 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Acquiring lock "f37b7fb7-1b94-421b-be79-7066010efabf" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:30:51 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lock "f37b7fb7-1b94-421b-be79-7066010efabf" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:30:51 user nova-compute[72351]: DEBUG nova.compute.manager [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Starting instance... {{(pid=72351) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2406}}
Aug 09 02:30:51 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:30:51 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:30:51 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72351) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Aug 09 02:30:51 user nova-compute[72351]: INFO nova.compute.claims [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Claim successful on node user
Aug 09 02:30:51 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:30:51 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:30:51 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.650s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:30:51 user nova-compute[72351]: DEBUG nova.compute.manager [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Start building networks asynchronously for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2803}}
Aug 09 02:30:51 user nova-compute[72351]: DEBUG nova.compute.manager [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Allocating IP information in the background. {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1956}}
Aug 09 02:30:51 user nova-compute[72351]: DEBUG nova.network.neutron [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] allocate_for_instance() {{(pid=72351) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Aug 09 02:30:51 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Aug 09 02:30:52 user nova-compute[72351]: DEBUG nova.compute.manager [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Start building block device mappings for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2838}}
Aug 09 02:30:52 user nova-compute[72351]: DEBUG nova.policy [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'a103ebb9b63f4a259163ec25a33a0dcc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc12f9037cd54bb6a2fdff3071132dd4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72351) authorize /opt/stack/nova/nova/policy.py:203}}
Aug 09 02:30:52 user nova-compute[72351]: DEBUG nova.compute.manager [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Start spawning the instance on the hypervisor. {{(pid=72351) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2612}}
Aug 09 02:30:52 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Creating instance directory {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4717}}
Aug 09 02:30:52 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Creating image(s)
Aug 09 02:30:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Acquiring lock "/opt/stack/data/nova/instances/f37b7fb7-1b94-421b-be79-7066010efabf/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:30:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lock "/opt/stack/data/nova/instances/f37b7fb7-1b94-421b-be79-7066010efabf/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:30:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lock "/opt/stack/data/nova/instances/f37b7fb7-1b94-421b-be79-7066010efabf/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:30:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Acquiring lock "d13d16d0ab57e27681822ca11b323960a692a169" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:30:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lock "d13d16d0ab57e27681822ca11b323960a692a169" acquired by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:30:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/d13d16d0ab57e27681822ca11b323960a692a169.part --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:53 user nova-compute[72351]: DEBUG nova.network.neutron [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Successfully created port: 1f2da7a0-a0f0-4715-8c5d-1df2a85fd295 {{(pid=72351) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Aug 09 02:30:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/d13d16d0ab57e27681822ca11b323960a692a169.part --force-share --output=json" returned: 0 in 0.168s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:53 user nova-compute[72351]: DEBUG nova.virt.images [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] abaa1849-3122-41bc-9c40-d1d80b6911e7 was qcow2, converting to raw {{(pid=72351) fetch_to_raw /opt/stack/nova/nova/virt/images.py:165}}
Aug 09 02:30:53 user nova-compute[72351]: DEBUG nova.privsep.utils [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=72351) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Aug 09 02:30:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Running cmd (subprocess): qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/d13d16d0ab57e27681822ca11b323960a692a169.part /opt/stack/data/nova/instances/_base/d13d16d0ab57e27681822ca11b323960a692a169.converted {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] CMD "qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/d13d16d0ab57e27681822ca11b323960a692a169.part /opt/stack/data/nova/instances/_base/d13d16d0ab57e27681822ca11b323960a692a169.converted" returned: 0 in 0.617s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/d13d16d0ab57e27681822ca11b323960a692a169.converted --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/d13d16d0ab57e27681822ca11b323960a692a169.converted --force-share --output=json" returned: 0 in 0.164s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lock "d13d16d0ab57e27681822ca11b323960a692a169" "released" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: held 2.366s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:30:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/d13d16d0ab57e27681822ca11b323960a692a169 --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:54 user nova-compute[72351]: DEBUG nova.network.neutron [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Successfully updated port: 1f2da7a0-a0f0-4715-8c5d-1df2a85fd295 {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:30:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Acquiring lock "refresh_cache-f37b7fb7-1b94-421b-be79-7066010efabf" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:30:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Acquired lock "refresh_cache-f37b7fb7-1b94-421b-be79-7066010efabf" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:30:54 user nova-compute[72351]: DEBUG nova.network.neutron [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:30:54 user nova-compute[72351]: DEBUG nova.compute.manager [req-0b4f0ede-a5f4-4842-a2de-14e3f6fb4e43 req-6e559756-b9d2-41f5-87e7-736c49ffd220 service nova] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Received event network-changed-1f2da7a0-a0f0-4715-8c5d-1df2a85fd295 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:30:54 user nova-compute[72351]: DEBUG nova.compute.manager [req-0b4f0ede-a5f4-4842-a2de-14e3f6fb4e43 req-6e559756-b9d2-41f5-87e7-736c49ffd220 service nova] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Refreshing instance network info cache due to event network-changed-1f2da7a0-a0f0-4715-8c5d-1df2a85fd295. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:30:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-0b4f0ede-a5f4-4842-a2de-14e3f6fb4e43 req-6e559756-b9d2-41f5-87e7-736c49ffd220 service nova] Acquiring lock "refresh_cache-f37b7fb7-1b94-421b-be79-7066010efabf" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:30:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/d13d16d0ab57e27681822ca11b323960a692a169 --force-share --output=json" returned: 0 in 0.162s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Acquiring lock "d13d16d0ab57e27681822ca11b323960a692a169" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:30:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lock "d13d16d0ab57e27681822ca11b323960a692a169" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:30:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/d13d16d0ab57e27681822ca11b323960a692a169 --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:54 user nova-compute[72351]: DEBUG nova.network.neutron [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:30:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/d13d16d0ab57e27681822ca11b323960a692a169 --force-share --output=json" returned: 0 in 0.223s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/d13d16d0ab57e27681822ca11b323960a692a169,backing_fmt=raw /opt/stack/data/nova/instances/f37b7fb7-1b94-421b-be79-7066010efabf/disk 1073741824 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/d13d16d0ab57e27681822ca11b323960a692a169,backing_fmt=raw /opt/stack/data/nova/instances/f37b7fb7-1b94-421b-be79-7066010efabf/disk 1073741824" returned: 0 in 0.056s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lock "d13d16d0ab57e27681822ca11b323960a692a169" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.283s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:30:54 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/d13d16d0ab57e27681822ca11b323960a692a169 --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/d13d16d0ab57e27681822ca11b323960a692a169 --force-share --output=json" returned: 0 in 0.145s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:55 user nova-compute[72351]: DEBUG nova.objects.instance [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lazy-loading 'migration_context' on Instance uuid f37b7fb7-1b94-421b-be79-7066010efabf {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:30:55 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Created local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4851}}
Aug 09 02:30:55 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Ensure instance console log exists: /opt/stack/data/nova/instances/f37b7fb7-1b94-421b-be79-7066010efabf/console.log {{(pid=72351) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4603}}
Aug 09 02:30:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:30:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:30:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:30:55 user nova-compute[72351]: DEBUG nova.network.neutron [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Updating instance_info_cache with network_info: [{"id": "1f2da7a0-a0f0-4715-8c5d-1df2a85fd295", "address": "fa:16:3e:0a:ba:2e", "network": {"id": "49bfc70e-93c7-4607-9507-4711bda1a686", "bridge": "br-int", "label": "tempest-TestStampPattern-1656229455-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fc12f9037cd54bb6a2fdff3071132dd4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1f2da7a0-a0", "ovs_interfaceid": "1f2da7a0-a0f0-4715-8c5d-1df2a85fd295", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:30:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Releasing lock "refresh_cache-f37b7fb7-1b94-421b-be79-7066010efabf" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:30:55 user nova-compute[72351]: DEBUG nova.compute.manager [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Instance network_info: |[{"id": "1f2da7a0-a0f0-4715-8c5d-1df2a85fd295", "address": "fa:16:3e:0a:ba:2e", "network": {"id": "49bfc70e-93c7-4607-9507-4711bda1a686", "bridge": "br-int", "label": "tempest-TestStampPattern-1656229455-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fc12f9037cd54bb6a2fdff3071132dd4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1f2da7a0-a0", "ovs_interfaceid": "1f2da7a0-a0f0-4715-8c5d-1df2a85fd295", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1971}}
Aug 09 02:30:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-0b4f0ede-a5f4-4842-a2de-14e3f6fb4e43 req-6e559756-b9d2-41f5-87e7-736c49ffd220 service nova] Acquired lock "refresh_cache-f37b7fb7-1b94-421b-be79-7066010efabf" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:30:55 user nova-compute[72351]: DEBUG nova.network.neutron [req-0b4f0ede-a5f4-4842-a2de-14e3f6fb4e43 req-6e559756-b9d2-41f5-87e7-736c49ffd220 service nova] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Refreshing network info cache for port 1f2da7a0-a0f0-4715-8c5d-1df2a85fd295 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:30:55 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Start _get_guest_xml network_info=[{"id": "1f2da7a0-a0f0-4715-8c5d-1df2a85fd295", "address": "fa:16:3e:0a:ba:2e", "network": {"id": "49bfc70e-93c7-4607-9507-4711bda1a686", "bridge": "br-int", "label": "tempest-TestStampPattern-1656229455-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fc12f9037cd54bb6a2fdff3071132dd4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1f2da7a0-a0", "ovs_interfaceid": "1f2da7a0-a0f0-4715-8c5d-1df2a85fd295", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='14313fa6f6cbb0e0adbd8b6aca47ca66',container_format='bare',created_at=2023-08-09T02:30:42Z,direct_url=<?>,disk_format='qcow2',id=abaa1849-3122-41bc-9c40-d1d80b6911e7,min_disk=1,min_ram=0,name='tempest-TestStampPatternsnapshot-1265561708',owner='fc12f9037cd54bb6a2fdff3071132dd4',properties=ImageMetaProps,protected=<?>,size=52363264,status='active',tags=<?>,updated_at=2023-08-09T02:30:49Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'size': 0, 'device_type': 'disk', 'encryption_secret_uuid': None, 'guest_format': None, 'encryption_options': None, 'disk_bus': 'virtio', 'encrypted': False, 'encryption_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'image_id': 'abaa1849-3122-41bc-9c40-d1d80b6911e7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:30:55 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:30:55 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:30:55 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:30:55 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:30:55 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:30:55 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:30:55 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:30:55 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='14313fa6f6cbb0e0adbd8b6aca47ca66',container_format='bare',created_at=2023-08-09T02:30:42Z,direct_url=<?>,disk_format='qcow2',id=abaa1849-3122-41bc-9c40-d1d80b6911e7,min_disk=1,min_ram=0,name='tempest-TestStampPatternsnapshot-1265561708',owner='fc12f9037cd54bb6a2fdff3071132dd4',properties=ImageMetaProps,protected=<?>,size=52363264,status='active',tags=<?>,updated_at=2023-08-09T02:30:49Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:30:55 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:30:55 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:30:55 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:30:55 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:30:55 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:30:55 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:30:55 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:30:55 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:30:55 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:30:55 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:30:55 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:30:50Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestStampPattern-server-1757563702',display_name='tempest-TestStampPattern-server-1757563702',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-teststamppattern-server-1757563702',id=36,image_ref='abaa1849-3122-41bc-9c40-d1d80b6911e7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBPzyrlB54Gi/8YMv0Cfs/esdmr1xNOdiloZGKR/XYFcg8Nos/w3Zm8ZOseVT8bKKFqeq3DP2KQSbuWg07AAvODzdTY0X+5ZdNLlP+cvLLuP4ef0UvshCJkUtWTANxgqmUw==',key_name='tempest-TestStampPattern-2061411239',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='fc12f9037cd54bb6a2fdff3071132dd4',ramdisk_id='',reservation_id='r-qrv9pcf2',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_boot_roles='reader,member',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_image_location='snapshot',image_image_state='available',image_image_type='snapshot',image_instance_uuid='a03fbded-23ad-440b-97e7-bea07425e873',image_min_disk='1',image_min_ram='0',image_owner_id='fc12f9037cd54bb6a2fdff3071132dd4',image_owner_project_name='tempest-TestStampPattern-1244733964',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',image_owner_user_name='tempest-TestStampPattern-1244733964-project-member',image_user_id='a103ebb9b63f4a259163ec25a33a0dcc',network_allocated='True',owner_project_name='tempest-TestStampPattern-1244733964',owner_user_name='tempest-TestStampPattern-1244733964-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:30:52Z,user_data=None,user_id='a103ebb9b63f4a259163ec25a33a0dcc',uuid=f37b7fb7-1b94-421b-be79-7066010efabf,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "1f2da7a0-a0f0-4715-8c5d-1df2a85fd295", "address": "fa:16:3e:0a:ba:2e", "network": {"id": "49bfc70e-93c7-4607-9507-4711bda1a686", "bridge": "br-int", "label": "tempest-TestStampPattern-1656229455-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fc12f9037cd54bb6a2fdff3071132dd4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1f2da7a0-a0", "ovs_interfaceid": "1f2da7a0-a0f0-4715-8c5d-1df2a85fd295", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:30:55 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Converting VIF {"id": "1f2da7a0-a0f0-4715-8c5d-1df2a85fd295", "address": "fa:16:3e:0a:ba:2e", "network": {"id": "49bfc70e-93c7-4607-9507-4711bda1a686", "bridge": "br-int", "label": "tempest-TestStampPattern-1656229455-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fc12f9037cd54bb6a2fdff3071132dd4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1f2da7a0-a0", "ovs_interfaceid": "1f2da7a0-a0f0-4715-8c5d-1df2a85fd295", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:30:55 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:0a:ba:2e,bridge_name='br-int',has_traffic_filtering=True,id=1f2da7a0-a0f0-4715-8c5d-1df2a85fd295,network=Network(49bfc70e-93c7-4607-9507-4711bda1a686),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1f2da7a0-a0') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:30:55 user nova-compute[72351]: DEBUG nova.objects.instance [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lazy-loading 'pci_devices' on Instance uuid f37b7fb7-1b94-421b-be79-7066010efabf {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:30:55 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:30:55 user nova-compute[72351]:   <uuid>f37b7fb7-1b94-421b-be79-7066010efabf</uuid>
Aug 09 02:30:55 user nova-compute[72351]:   <name>instance-00000024</name>
Aug 09 02:30:55 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:30:55 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:30:55 user nova-compute[72351]:   <metadata>
Aug 09 02:30:55 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:30:55 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:30:55 user nova-compute[72351]:       <nova:name>tempest-TestStampPattern-server-1757563702</nova:name>
Aug 09 02:30:55 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:30:55</nova:creationTime>
Aug 09 02:30:55 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:30:55 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:30:55 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:30:55 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:30:55 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:30:55 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:30:55 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:30:55 user nova-compute[72351]:       <nova:owner>
Aug 09 02:30:55 user nova-compute[72351]:         <nova:user uuid="a103ebb9b63f4a259163ec25a33a0dcc">tempest-TestStampPattern-1244733964-project-member</nova:user>
Aug 09 02:30:55 user nova-compute[72351]:         <nova:project uuid="fc12f9037cd54bb6a2fdff3071132dd4">tempest-TestStampPattern-1244733964</nova:project>
Aug 09 02:30:55 user nova-compute[72351]:       </nova:owner>
Aug 09 02:30:55 user nova-compute[72351]:       <nova:root type="image" uuid="abaa1849-3122-41bc-9c40-d1d80b6911e7"/>
Aug 09 02:30:55 user nova-compute[72351]:       <nova:ports>
Aug 09 02:30:55 user nova-compute[72351]:         <nova:port uuid="1f2da7a0-a0f0-4715-8c5d-1df2a85fd295">
Aug 09 02:30:55 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.5" ipVersion="4"/>
Aug 09 02:30:55 user nova-compute[72351]:         </nova:port>
Aug 09 02:30:55 user nova-compute[72351]:       </nova:ports>
Aug 09 02:30:55 user nova-compute[72351]:     </nova:instance>
Aug 09 02:30:55 user nova-compute[72351]:   </metadata>
Aug 09 02:30:55 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:30:55 user nova-compute[72351]:     <system>
Aug 09 02:30:55 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:30:55 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:30:55 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:30:55 user nova-compute[72351]:       <entry name="serial">f37b7fb7-1b94-421b-be79-7066010efabf</entry>
Aug 09 02:30:55 user nova-compute[72351]:       <entry name="uuid">f37b7fb7-1b94-421b-be79-7066010efabf</entry>
Aug 09 02:30:55 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:30:55 user nova-compute[72351]:     </system>
Aug 09 02:30:55 user nova-compute[72351]:   </sysinfo>
Aug 09 02:30:55 user nova-compute[72351]:   <os>
Aug 09 02:30:55 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:30:55 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:30:55 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:30:55 user nova-compute[72351]:   </os>
Aug 09 02:30:55 user nova-compute[72351]:   <features>
Aug 09 02:30:55 user nova-compute[72351]:     <acpi/>
Aug 09 02:30:55 user nova-compute[72351]:     <apic/>
Aug 09 02:30:55 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:30:55 user nova-compute[72351]:   </features>
Aug 09 02:30:55 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:30:55 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:30:55 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:30:55 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:30:55 user nova-compute[72351]:   </clock>
Aug 09 02:30:55 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:30:55 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:30:55 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:30:55 user nova-compute[72351]:   </cpu>
Aug 09 02:30:55 user nova-compute[72351]:   <devices>
Aug 09 02:30:55 user nova-compute[72351]:     <disk type="file" device="disk">
Aug 09 02:30:55 user nova-compute[72351]:       <driver name="qemu" type="qcow2" cache="none"/>
Aug 09 02:30:55 user nova-compute[72351]:       <source file="/opt/stack/data/nova/instances/f37b7fb7-1b94-421b-be79-7066010efabf/disk"/>
Aug 09 02:30:55 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:30:55 user nova-compute[72351]:     </disk>
Aug 09 02:30:55 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:30:55 user nova-compute[72351]:       <mac address="fa:16:3e:0a:ba:2e"/>
Aug 09 02:30:55 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:30:55 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:30:55 user nova-compute[72351]:       <target dev="tap1f2da7a0-a0"/>
Aug 09 02:30:55 user nova-compute[72351]:     </interface>
Aug 09 02:30:55 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:30:55 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/f37b7fb7-1b94-421b-be79-7066010efabf/console.log" append="off"/>
Aug 09 02:30:55 user nova-compute[72351]:     </serial>
Aug 09 02:30:55 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:30:55 user nova-compute[72351]:     <video>
Aug 09 02:30:55 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:30:55 user nova-compute[72351]:     </video>
Aug 09 02:30:55 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:30:55 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:30:55 user nova-compute[72351]:     </rng>
Aug 09 02:30:55 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:30:55 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:30:55 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:30:55 user nova-compute[72351]:     </memballoon>
Aug 09 02:30:55 user nova-compute[72351]:   </devices>
Aug 09 02:30:55 user nova-compute[72351]: </domain>
Aug 09 02:30:55 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:30:55 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:30:50Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestStampPattern-server-1757563702',display_name='tempest-TestStampPattern-server-1757563702',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-teststamppattern-server-1757563702',id=36,image_ref='abaa1849-3122-41bc-9c40-d1d80b6911e7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBPzyrlB54Gi/8YMv0Cfs/esdmr1xNOdiloZGKR/XYFcg8Nos/w3Zm8ZOseVT8bKKFqeq3DP2KQSbuWg07AAvODzdTY0X+5ZdNLlP+cvLLuP4ef0UvshCJkUtWTANxgqmUw==',key_name='tempest-TestStampPattern-2061411239',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='fc12f9037cd54bb6a2fdff3071132dd4',ramdisk_id='',reservation_id='r-qrv9pcf2',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_boot_roles='reader,member',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_image_location='snapshot',image_image_state='available',image_image_type='snapshot',image_instance_uuid='a03fbded-23ad-440b-97e7-bea07425e873',image_min_disk='1',image_min_ram='0',image_owner_id='fc12f9037cd54bb6a2fdff3071132dd4',image_owner_project_name='tempest-TestStampPattern-1244733964',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',image_owner_user_name='tempest-TestStampPattern-1244733964-project-member',image_user_id='a103ebb9b63f4a259163ec25a33a0dcc',network_allocated='True',owner_project_name='tempest-TestStampPattern-1244733964',owner_user_name='tempest-TestStampPattern-1244733964-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:30:52Z,user_data=None,user_id='a103ebb9b63f4a259163ec25a33a0dcc',uuid=f37b7fb7-1b94-421b-be79-7066010efabf,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "1f2da7a0-a0f0-4715-8c5d-1df2a85fd295", "address": "fa:16:3e:0a:ba:2e", "network": {"id": "49bfc70e-93c7-4607-9507-4711bda1a686", "bridge": "br-int", "label": "tempest-TestStampPattern-1656229455-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fc12f9037cd54bb6a2fdff3071132dd4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1f2da7a0-a0", "ovs_interfaceid": "1f2da7a0-a0f0-4715-8c5d-1df2a85fd295", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:30:55 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Converting VIF {"id": "1f2da7a0-a0f0-4715-8c5d-1df2a85fd295", "address": "fa:16:3e:0a:ba:2e", "network": {"id": "49bfc70e-93c7-4607-9507-4711bda1a686", "bridge": "br-int", "label": "tempest-TestStampPattern-1656229455-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fc12f9037cd54bb6a2fdff3071132dd4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1f2da7a0-a0", "ovs_interfaceid": "1f2da7a0-a0f0-4715-8c5d-1df2a85fd295", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:30:55 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:0a:ba:2e,bridge_name='br-int',has_traffic_filtering=True,id=1f2da7a0-a0f0-4715-8c5d-1df2a85fd295,network=Network(49bfc70e-93c7-4607-9507-4711bda1a686),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1f2da7a0-a0') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:30:55 user nova-compute[72351]: DEBUG os_vif [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:0a:ba:2e,bridge_name='br-int',has_traffic_filtering=True,id=1f2da7a0-a0f0-4715-8c5d-1df2a85fd295,network=Network(49bfc70e-93c7-4607-9507-4711bda1a686),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1f2da7a0-a0') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:30:55 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:55 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:30:55 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:30:55 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:55 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:55 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap1f2da7a0-a0, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:30:55 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap1f2da7a0-a0, col_values=(('external_ids', {'iface-id': '1f2da7a0-a0f0-4715-8c5d-1df2a85fd295', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:0a:ba:2e', 'vm-uuid': 'f37b7fb7-1b94-421b-be79-7066010efabf'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:30:55 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:55 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:30:55 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:55 user nova-compute[72351]: INFO os_vif [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:0a:ba:2e,bridge_name='br-int',has_traffic_filtering=True,id=1f2da7a0-a0f0-4715-8c5d-1df2a85fd295,network=Network(49bfc70e-93c7-4607-9507-4711bda1a686),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1f2da7a0-a0')
Aug 09 02:30:55 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:30:55 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] No VIF found with MAC fa:16:3e:0a:ba:2e, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:30:55 user nova-compute[72351]: DEBUG nova.network.neutron [req-0b4f0ede-a5f4-4842-a2de-14e3f6fb4e43 req-6e559756-b9d2-41f5-87e7-736c49ffd220 service nova] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Updated VIF entry in instance network info cache for port 1f2da7a0-a0f0-4715-8c5d-1df2a85fd295. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:30:55 user nova-compute[72351]: DEBUG nova.network.neutron [req-0b4f0ede-a5f4-4842-a2de-14e3f6fb4e43 req-6e559756-b9d2-41f5-87e7-736c49ffd220 service nova] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Updating instance_info_cache with network_info: [{"id": "1f2da7a0-a0f0-4715-8c5d-1df2a85fd295", "address": "fa:16:3e:0a:ba:2e", "network": {"id": "49bfc70e-93c7-4607-9507-4711bda1a686", "bridge": "br-int", "label": "tempest-TestStampPattern-1656229455-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fc12f9037cd54bb6a2fdff3071132dd4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1f2da7a0-a0", "ovs_interfaceid": "1f2da7a0-a0f0-4715-8c5d-1df2a85fd295", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:30:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-0b4f0ede-a5f4-4842-a2de-14e3f6fb4e43 req-6e559756-b9d2-41f5-87e7-736c49ffd220 service nova] Releasing lock "refresh_cache-f37b7fb7-1b94-421b-be79-7066010efabf" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:30:56 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:56 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:56 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:56 user nova-compute[72351]: DEBUG nova.compute.manager [req-c5e7812c-0a83-4da0-b987-c9413f38d508 req-0d579d60-1221-4891-81df-d340fcb30f80 service nova] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Received event network-vif-plugged-1f2da7a0-a0f0-4715-8c5d-1df2a85fd295 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:30:56 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c5e7812c-0a83-4da0-b987-c9413f38d508 req-0d579d60-1221-4891-81df-d340fcb30f80 service nova] Acquiring lock "f37b7fb7-1b94-421b-be79-7066010efabf-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:30:56 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c5e7812c-0a83-4da0-b987-c9413f38d508 req-0d579d60-1221-4891-81df-d340fcb30f80 service nova] Lock "f37b7fb7-1b94-421b-be79-7066010efabf-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:30:56 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c5e7812c-0a83-4da0-b987-c9413f38d508 req-0d579d60-1221-4891-81df-d340fcb30f80 service nova] Lock "f37b7fb7-1b94-421b-be79-7066010efabf-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:30:56 user nova-compute[72351]: DEBUG nova.compute.manager [req-c5e7812c-0a83-4da0-b987-c9413f38d508 req-0d579d60-1221-4891-81df-d340fcb30f80 service nova] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] No waiting events found dispatching network-vif-plugged-1f2da7a0-a0f0-4715-8c5d-1df2a85fd295 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:30:56 user nova-compute[72351]: WARNING nova.compute.manager [req-c5e7812c-0a83-4da0-b987-c9413f38d508 req-0d579d60-1221-4891-81df-d340fcb30f80 service nova] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Received unexpected event network-vif-plugged-1f2da7a0-a0f0-4715-8c5d-1df2a85fd295 for instance with vm_state building and task_state spawning.
Aug 09 02:30:56 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:56 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:58 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:30:58 user nova-compute[72351]: DEBUG nova.compute.manager [req-ae6c210f-5b51-44b8-82a2-7b8f770c2694 req-54a604d6-63ef-4ed3-822d-5a2a7560eac7 service nova] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Received event network-vif-plugged-1f2da7a0-a0f0-4715-8c5d-1df2a85fd295 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:30:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ae6c210f-5b51-44b8-82a2-7b8f770c2694 req-54a604d6-63ef-4ed3-822d-5a2a7560eac7 service nova] Acquiring lock "f37b7fb7-1b94-421b-be79-7066010efabf-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:30:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ae6c210f-5b51-44b8-82a2-7b8f770c2694 req-54a604d6-63ef-4ed3-822d-5a2a7560eac7 service nova] Lock "f37b7fb7-1b94-421b-be79-7066010efabf-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:30:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ae6c210f-5b51-44b8-82a2-7b8f770c2694 req-54a604d6-63ef-4ed3-822d-5a2a7560eac7 service nova] Lock "f37b7fb7-1b94-421b-be79-7066010efabf-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:30:58 user nova-compute[72351]: DEBUG nova.compute.manager [req-ae6c210f-5b51-44b8-82a2-7b8f770c2694 req-54a604d6-63ef-4ed3-822d-5a2a7560eac7 service nova] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] No waiting events found dispatching network-vif-plugged-1f2da7a0-a0f0-4715-8c5d-1df2a85fd295 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:30:58 user nova-compute[72351]: WARNING nova.compute.manager [req-ae6c210f-5b51-44b8-82a2-7b8f770c2694 req-54a604d6-63ef-4ed3-822d-5a2a7560eac7 service nova] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Received unexpected event network-vif-plugged-1f2da7a0-a0f0-4715-8c5d-1df2a85fd295 for instance with vm_state building and task_state spawning.
Aug 09 02:30:58 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548258.9853528, f37b7fb7-1b94-421b-be79-7066010efabf => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:30:58 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] VM Resumed (Lifecycle Event)
Aug 09 02:30:58 user nova-compute[72351]: DEBUG nova.compute.manager [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:30:58 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Guest created on hypervisor {{(pid=72351) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4411}}
Aug 09 02:30:58 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Instance spawned successfully.
Aug 09 02:30:58 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Attempting to register defaults for the following image properties: ['hw_input_bus', 'hw_pointer_model'] {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:894}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Found default for hw_input_bus of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Found default for hw_pointer_model of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:30:59 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:30:59 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548258.9871032, f37b7fb7-1b94-421b-be79-7066010efabf => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:30:59 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] VM Started (Lifecycle Event)
Aug 09 02:30:59 user nova-compute[72351]: DEBUG nova.compute.manager [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Start spawning the instance on the hypervisor. {{(pid=72351) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2612}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Creating instance directory {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4717}}
Aug 09 02:30:59 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Creating image(s)
Aug 09 02:30:59 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Did not create local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4853}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Ensure instance console log exists: /opt/stack/data/nova/instances/49a10075-0d3c-4cd6-bb43-10238cad3f37/console.log {{(pid=72351) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4603}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Start _get_guest_xml network_info=[{"id": "2dcfe7b8-34fe-4860-b820-f3de7c00df62", "address": "fa:16:3e:01:0f:c9", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2dcfe7b8-34", "ovs_interfaceid": "2dcfe7b8-34fe-4860-b820-f3de7c00df62", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'device_type': 'disk', 'guest_format': None, 'disk_bus': 'virtio', 'mount_device': '/dev/vda', 'boot_index': 0, 'attachment_id': '2644fb08-2d89-40eb-a122-337cd6a3cfb9', 'delete_on_termination': False, 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqns': ['iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'CdzJhfwW', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '49a10075-0d3c-4cd6-bb43-10238cad3f37', 'attached_at': '', 'detached_at': '', 'volume_id': '4113a563-77a5-4b68-9198-9d0434737448', 'serial': '4113a563-77a5-4b68-9198-9d0434737448'}, 'volume_type': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:30:59 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:30:59 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:30:59 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:30:59 user nova-compute[72351]: INFO nova.compute.manager [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Took 6.94 seconds to spawn the instance on the hypervisor.
Aug 09 02:30:59 user nova-compute[72351]: DEBUG nova.compute.manager [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'CdzJhfwW', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:30:59 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:30:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:30:59 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:30:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:30:59 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 02:30:59 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:30:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 -p 172.16.0.220:3260" returned: 21 in 0.024s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[89af120c-280c-482d-ab1f-fa0695ba8ea3]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Aug 09 02:30:59 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 -p 172.16.0.220:3260 --interface default --op new {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.034s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[3df39abb-9d3a-43e1-9222-7b09c6951d18]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448] added\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448] added
Aug 09 02:30:59 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:59 user nova-compute[72351]: INFO nova.compute.manager [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Took 8.05 seconds to build instance.
Aug 09 02:30:59 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-789c10ea-a939-4b53-b1a1-f7b05f842ac3 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lock "f37b7fb7-1b94-421b-be79-7066010efabf" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 8.158s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.036s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[9af5253e-344d-410c-8ad6-604034160dcb]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[55a2cc0f-a023-4084-8d03-1998debd38f2]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v CdzJhfwW {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v CdzJhfwW" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[29496cb9-3cbe-4c0f-ad62-902a2763ea78]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'CdzJhfwW'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[33e31588-7067-4377-a8fc-9f361dab47e7]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.029s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[54fc84e0-0987-4824-811b-fb5e2853e19a]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e (non-flash)\ntcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70 (non-flash)\ntcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3cb561e0-d189-4a63-abe1-3a68ffbe4616 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:30:59 user nova-compute[72351]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 (non-flash)
Aug 09 02:30:59 user nova-compute[72351]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e (non-flash)
Aug 09 02:30:59 user nova-compute[72351]: tcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70 (non-flash)
Aug 09 02:30:59 user nova-compute[72351]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3cb561e0-d189-4a63-abe1-3a68ffbe4616 (non-flash)
Aug 09 02:30:59 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:30:59 user nova-compute[72351]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 (non-flash)
Aug 09 02:30:59 user nova-compute[72351]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e (non-flash)
Aug 09 02:30:59 user nova-compute[72351]: tcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70 (non-flash)
Aug 09 02:30:59 user nova-compute[72351]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3cb561e0-d189-4a63-abe1-3a68ffbe4616 (non-flash)
Aug 09 02:30:59 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 -p 172.16.0.220:3260 --login {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 -p 172.16.0.220:3260 --login" returned: 0 in 0.088s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[2b48d1a0-9c15-4e61-8aed-6f9611d58c28]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448, portal: 172.16.0.220,3260]
Aug 09 02:30:59 user nova-compute[72351]: Login to [iface: default, target: iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448, portal: 172.16.0.220,3260] successful.
Aug 09 02:30:59 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.025s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[9f2a7adc-921b-4c78-85f3-c6bc48fb2b13]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.028s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[b17cd0d6-d429-4642-84c5-4a0438451ca0]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e (non-flash)\ntcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70 (non-flash)\ntcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3cb561e0-d189-4a63-abe1-3a68ffbe4616 (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:30:59 user nova-compute[72351]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 (non-flash)
Aug 09 02:30:59 user nova-compute[72351]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e (non-flash)
Aug 09 02:30:59 user nova-compute[72351]: tcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70 (non-flash)
Aug 09 02:30:59 user nova-compute[72351]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3cb561e0-d189-4a63-abe1-3a68ffbe4616 (non-flash)
Aug 09 02:30:59 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:30:59 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:30:59 user nova-compute[72351]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 (non-flash)
Aug 09 02:30:59 user nova-compute[72351]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e (non-flash)
Aug 09 02:30:59 user nova-compute[72351]: tcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70 (non-flash)
Aug 09 02:30:59 user nova-compute[72351]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3cb561e0-d189-4a63-abe1-3a68ffbe4616 (non-flash)
Aug 09 02:30:59 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:30:59 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.374s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] HCTL ('37', '-', '-', 0) found on session 37 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Scanning host 37 c: -, t: -, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host37/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host37/scan" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a71649cb-b5e1-49e7-a486-cee069e4d4c1]: (4, ('- - 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Searching for a device in session 37 and hctl ['37', '*', '*', 0] yield: sdf {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:30:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Connected to sdf using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'CdzJhfwW', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 02:31:00 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:00 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.411s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:31:00 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] <== connect_volume: return (1413ms) {'type': 'block', 'scsi_wwn': '23931306338316130', 'path': '/dev/sdf'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:31:00 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23931306338316130', 'path': '/dev/sdf'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 02:31:00 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:30:37Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-server-1123873481',display_name='tempest-TestVolumeBootPattern-server-1123873481',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testvolumebootpattern-server-1123873481',id=35,image_ref='',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBM+1oHdu/+hmVTCW73FChAt+AvG2bL++YbEuGYMvrUaUYAyZQrt9Vwv7j34dhFmoVuCI0Rk18KgJ1X07X8pPcTOnOW0XFALfY2BbRHpJcmwARDVbYh8uPoNVxqSRoIR2gA==',key_name='tempest-TestVolumeBootPattern-805107831',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='a4c6bbfd8d904ba1ac69ba5da4564513',ramdisk_id='',reservation_id='r-tfy77r2o',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestVolumeBootPattern-350743460',owner_user_name='tempest-TestVolumeBootPattern-350743460-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:30:40Z,user_data=None,user_id='07d8d5a634274df19c2db769d2d27b29',uuid=49a10075-0d3c-4cd6-bb43-10238cad3f37,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "2dcfe7b8-34fe-4860-b820-f3de7c00df62", "address": "fa:16:3e:01:0f:c9", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2dcfe7b8-34", "ovs_interfaceid": "2dcfe7b8-34fe-4860-b820-f3de7c00df62", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:31:00 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Converting VIF {"id": "2dcfe7b8-34fe-4860-b820-f3de7c00df62", "address": "fa:16:3e:01:0f:c9", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2dcfe7b8-34", "ovs_interfaceid": "2dcfe7b8-34fe-4860-b820-f3de7c00df62", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:31:00 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:01:0f:c9,bridge_name='br-int',has_traffic_filtering=True,id=2dcfe7b8-34fe-4860-b820-f3de7c00df62,network=Network(5ef428cf-e408-490c-8f60-70934f760f4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2dcfe7b8-34') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:31:00 user nova-compute[72351]: DEBUG nova.objects.instance [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lazy-loading 'pci_devices' on Instance uuid 49a10075-0d3c-4cd6-bb43-10238cad3f37 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:31:00 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:31:00 user nova-compute[72351]:   <uuid>49a10075-0d3c-4cd6-bb43-10238cad3f37</uuid>
Aug 09 02:31:00 user nova-compute[72351]:   <name>instance-00000023</name>
Aug 09 02:31:00 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:31:00 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:31:00 user nova-compute[72351]:   <metadata>
Aug 09 02:31:00 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:31:00 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:31:00 user nova-compute[72351]:       <nova:name>tempest-TestVolumeBootPattern-server-1123873481</nova:name>
Aug 09 02:31:00 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:30:59</nova:creationTime>
Aug 09 02:31:00 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:31:00 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:31:00 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:31:00 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:31:00 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:31:00 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:31:00 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:31:00 user nova-compute[72351]:       <nova:owner>
Aug 09 02:31:00 user nova-compute[72351]:         <nova:user uuid="07d8d5a634274df19c2db769d2d27b29">tempest-TestVolumeBootPattern-350743460-project-member</nova:user>
Aug 09 02:31:00 user nova-compute[72351]:         <nova:project uuid="a4c6bbfd8d904ba1ac69ba5da4564513">tempest-TestVolumeBootPattern-350743460</nova:project>
Aug 09 02:31:00 user nova-compute[72351]:       </nova:owner>
Aug 09 02:31:00 user nova-compute[72351]:       <nova:ports>
Aug 09 02:31:00 user nova-compute[72351]:         <nova:port uuid="2dcfe7b8-34fe-4860-b820-f3de7c00df62">
Aug 09 02:31:00 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.10" ipVersion="4"/>
Aug 09 02:31:00 user nova-compute[72351]:         </nova:port>
Aug 09 02:31:00 user nova-compute[72351]:       </nova:ports>
Aug 09 02:31:00 user nova-compute[72351]:     </nova:instance>
Aug 09 02:31:00 user nova-compute[72351]:   </metadata>
Aug 09 02:31:00 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:31:00 user nova-compute[72351]:     <system>
Aug 09 02:31:00 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:31:00 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:31:00 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:31:00 user nova-compute[72351]:       <entry name="serial">49a10075-0d3c-4cd6-bb43-10238cad3f37</entry>
Aug 09 02:31:00 user nova-compute[72351]:       <entry name="uuid">49a10075-0d3c-4cd6-bb43-10238cad3f37</entry>
Aug 09 02:31:00 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:31:00 user nova-compute[72351]:     </system>
Aug 09 02:31:00 user nova-compute[72351]:   </sysinfo>
Aug 09 02:31:00 user nova-compute[72351]:   <os>
Aug 09 02:31:00 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:31:00 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:31:00 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:31:00 user nova-compute[72351]:   </os>
Aug 09 02:31:00 user nova-compute[72351]:   <features>
Aug 09 02:31:00 user nova-compute[72351]:     <acpi/>
Aug 09 02:31:00 user nova-compute[72351]:     <apic/>
Aug 09 02:31:00 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:31:00 user nova-compute[72351]:   </features>
Aug 09 02:31:00 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:31:00 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:31:00 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:31:00 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:31:00 user nova-compute[72351]:   </clock>
Aug 09 02:31:00 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:31:00 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:31:00 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:31:00 user nova-compute[72351]:   </cpu>
Aug 09 02:31:00 user nova-compute[72351]:   <devices>
Aug 09 02:31:00 user nova-compute[72351]:     <disk type="block" device="disk">
Aug 09 02:31:00 user nova-compute[72351]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:31:00 user nova-compute[72351]:       <source dev="/dev/sdf"/>
Aug 09 02:31:00 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:31:00 user nova-compute[72351]:       <serial>4113a563-77a5-4b68-9198-9d0434737448</serial>
Aug 09 02:31:00 user nova-compute[72351]:     </disk>
Aug 09 02:31:00 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:31:00 user nova-compute[72351]:       <mac address="fa:16:3e:01:0f:c9"/>
Aug 09 02:31:00 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:31:00 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:31:00 user nova-compute[72351]:       <target dev="tap2dcfe7b8-34"/>
Aug 09 02:31:00 user nova-compute[72351]:     </interface>
Aug 09 02:31:00 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:31:00 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/49a10075-0d3c-4cd6-bb43-10238cad3f37/console.log" append="off"/>
Aug 09 02:31:00 user nova-compute[72351]:     </serial>
Aug 09 02:31:00 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:31:00 user nova-compute[72351]:     <video>
Aug 09 02:31:00 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:31:00 user nova-compute[72351]:     </video>
Aug 09 02:31:00 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:31:00 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:31:00 user nova-compute[72351]:     </rng>
Aug 09 02:31:00 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:31:00 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:31:00 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:31:00 user nova-compute[72351]:     </memballoon>
Aug 09 02:31:00 user nova-compute[72351]:   </devices>
Aug 09 02:31:00 user nova-compute[72351]: </domain>
Aug 09 02:31:00 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:31:00 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:30:37Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-server-1123873481',display_name='tempest-TestVolumeBootPattern-server-1123873481',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testvolumebootpattern-server-1123873481',id=35,image_ref='',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBM+1oHdu/+hmVTCW73FChAt+AvG2bL++YbEuGYMvrUaUYAyZQrt9Vwv7j34dhFmoVuCI0Rk18KgJ1X07X8pPcTOnOW0XFALfY2BbRHpJcmwARDVbYh8uPoNVxqSRoIR2gA==',key_name='tempest-TestVolumeBootPattern-805107831',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='a4c6bbfd8d904ba1ac69ba5da4564513',ramdisk_id='',reservation_id='r-tfy77r2o',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestVolumeBootPattern-350743460',owner_user_name='tempest-TestVolumeBootPattern-350743460-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:30:40Z,user_data=None,user_id='07d8d5a634274df19c2db769d2d27b29',uuid=49a10075-0d3c-4cd6-bb43-10238cad3f37,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "2dcfe7b8-34fe-4860-b820-f3de7c00df62", "address": "fa:16:3e:01:0f:c9", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2dcfe7b8-34", "ovs_interfaceid": "2dcfe7b8-34fe-4860-b820-f3de7c00df62", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:31:00 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Converting VIF {"id": "2dcfe7b8-34fe-4860-b820-f3de7c00df62", "address": "fa:16:3e:01:0f:c9", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2dcfe7b8-34", "ovs_interfaceid": "2dcfe7b8-34fe-4860-b820-f3de7c00df62", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:31:00 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:01:0f:c9,bridge_name='br-int',has_traffic_filtering=True,id=2dcfe7b8-34fe-4860-b820-f3de7c00df62,network=Network(5ef428cf-e408-490c-8f60-70934f760f4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2dcfe7b8-34') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:31:00 user nova-compute[72351]: DEBUG os_vif [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:01:0f:c9,bridge_name='br-int',has_traffic_filtering=True,id=2dcfe7b8-34fe-4860-b820-f3de7c00df62,network=Network(5ef428cf-e408-490c-8f60-70934f760f4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2dcfe7b8-34') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:31:00 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:00 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:31:00 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:31:00 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:00 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap2dcfe7b8-34, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:31:00 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap2dcfe7b8-34, col_values=(('external_ids', {'iface-id': '2dcfe7b8-34fe-4860-b820-f3de7c00df62', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:01:0f:c9', 'vm-uuid': '49a10075-0d3c-4cd6-bb43-10238cad3f37'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:31:00 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:00 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:31:00 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:00 user nova-compute[72351]: INFO os_vif [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:01:0f:c9,bridge_name='br-int',has_traffic_filtering=True,id=2dcfe7b8-34fe-4860-b820-f3de7c00df62,network=Network(5ef428cf-e408-490c-8f60-70934f760f4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2dcfe7b8-34')
Aug 09 02:31:00 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:31:00 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] No VIF found with MAC fa:16:3e:01:0f:c9, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:31:00 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:31:02 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:02 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:02 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:02 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:02 user nova-compute[72351]: DEBUG nova.compute.manager [req-f1b6211e-23a5-415c-9b8a-45f5bc8c90e3 req-e9dc9a56-5f57-4063-9836-5153af9c9aee service nova] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Received event network-vif-plugged-2dcfe7b8-34fe-4860-b820-f3de7c00df62 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:31:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-f1b6211e-23a5-415c-9b8a-45f5bc8c90e3 req-e9dc9a56-5f57-4063-9836-5153af9c9aee service nova] Acquiring lock "49a10075-0d3c-4cd6-bb43-10238cad3f37-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:31:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-f1b6211e-23a5-415c-9b8a-45f5bc8c90e3 req-e9dc9a56-5f57-4063-9836-5153af9c9aee service nova] Lock "49a10075-0d3c-4cd6-bb43-10238cad3f37-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:31:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-f1b6211e-23a5-415c-9b8a-45f5bc8c90e3 req-e9dc9a56-5f57-4063-9836-5153af9c9aee service nova] Lock "49a10075-0d3c-4cd6-bb43-10238cad3f37-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:31:02 user nova-compute[72351]: DEBUG nova.compute.manager [req-f1b6211e-23a5-415c-9b8a-45f5bc8c90e3 req-e9dc9a56-5f57-4063-9836-5153af9c9aee service nova] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] No waiting events found dispatching network-vif-plugged-2dcfe7b8-34fe-4860-b820-f3de7c00df62 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:31:02 user nova-compute[72351]: WARNING nova.compute.manager [req-f1b6211e-23a5-415c-9b8a-45f5bc8c90e3 req-e9dc9a56-5f57-4063-9836-5153af9c9aee service nova] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Received unexpected event network-vif-plugged-2dcfe7b8-34fe-4860-b820-f3de7c00df62 for instance with vm_state building and task_state spawning.
Aug 09 02:31:02 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:31:02 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72351) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10461}}
Aug 09 02:31:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:03 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:31:03 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:31:03 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:31:03 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:31:03 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Auditing locally available compute resources for user (node: user) {{(pid=72351) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Aug 09 02:31:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-3da51dcd-c16a-4e72-9a2c-302f242116ee tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Acquiring lock "9df16e46-d097-4553-b44c-732ea24a4169" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:31:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-3da51dcd-c16a-4e72-9a2c-302f242116ee tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lock "9df16e46-d097-4553-b44c-732ea24a4169" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:31:04 user nova-compute[72351]: INFO nova.compute.manager [None req-3da51dcd-c16a-4e72-9a2c-302f242116ee tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Detaching volume 3cb561e0-d189-4a63-abe1-3a68ffbe4616
Aug 09 02:31:04 user nova-compute[72351]: INFO nova.virt.block_device [None req-3da51dcd-c16a-4e72-9a2c-302f242116ee tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Attempting to driver detach volume 3cb561e0-d189-4a63-abe1-3a68ffbe4616 from mountpoint /dev/vdb
Aug 09 02:31:04 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-3da51dcd-c16a-4e72-9a2c-302f242116ee tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Attempting to detach device vdb from instance 9df16e46-d097-4553-b44c-732ea24a4169 from the persistent domain config. {{(pid=72351) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2477}}
Aug 09 02:31:04 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-3da51dcd-c16a-4e72-9a2c-302f242116ee tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 02:31:04 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:31:04 user nova-compute[72351]:   <source dev="/dev/sde"/>
Aug 09 02:31:04 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:31:04 user nova-compute[72351]:   <serial>3cb561e0-d189-4a63-abe1-3a68ffbe4616</serial>
Aug 09 02:31:04 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Aug 09 02:31:04 user nova-compute[72351]: </disk>
Aug 09 02:31:04 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 02:31:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-dfb7407b-1a34-4ea2-bb71-c1827c95d882 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Acquiring lock "9df16e46-d097-4553-b44c-732ea24a4169" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:31:04 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-3da51dcd-c16a-4e72-9a2c-302f242116ee tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Successfully detached device vdb from instance 9df16e46-d097-4553-b44c-732ea24a4169 from the persistent domain config.
Aug 09 02:31:04 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-3da51dcd-c16a-4e72-9a2c-302f242116ee tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 9df16e46-d097-4553-b44c-732ea24a4169 from the live domain config. {{(pid=72351) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2513}}
Aug 09 02:31:04 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-3da51dcd-c16a-4e72-9a2c-302f242116ee tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 02:31:04 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:31:04 user nova-compute[72351]:   <source dev="/dev/sde"/>
Aug 09 02:31:04 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:31:04 user nova-compute[72351]:   <serial>3cb561e0-d189-4a63-abe1-3a68ffbe4616</serial>
Aug 09 02:31:04 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Aug 09 02:31:04 user nova-compute[72351]: </disk>
Aug 09 02:31:04 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 02:31:04 user nova-compute[72351]: DEBUG nova.compute.manager [req-8d950c1b-51a0-4fa3-b0b9-ebdb580b66f0 req-dd440d57-e1ee-441a-a11f-b60651a6c0dd service nova] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Received event network-vif-plugged-2dcfe7b8-34fe-4860-b820-f3de7c00df62 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:31:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-8d950c1b-51a0-4fa3-b0b9-ebdb580b66f0 req-dd440d57-e1ee-441a-a11f-b60651a6c0dd service nova] Acquiring lock "49a10075-0d3c-4cd6-bb43-10238cad3f37-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:31:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-8d950c1b-51a0-4fa3-b0b9-ebdb580b66f0 req-dd440d57-e1ee-441a-a11f-b60651a6c0dd service nova] Lock "49a10075-0d3c-4cd6-bb43-10238cad3f37-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:31:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-8d950c1b-51a0-4fa3-b0b9-ebdb580b66f0 req-dd440d57-e1ee-441a-a11f-b60651a6c0dd service nova] Lock "49a10075-0d3c-4cd6-bb43-10238cad3f37-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:31:04 user nova-compute[72351]: DEBUG nova.compute.manager [req-8d950c1b-51a0-4fa3-b0b9-ebdb580b66f0 req-dd440d57-e1ee-441a-a11f-b60651a6c0dd service nova] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] No waiting events found dispatching network-vif-plugged-2dcfe7b8-34fe-4860-b820-f3de7c00df62 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:31:04 user nova-compute[72351]: WARNING nova.compute.manager [req-8d950c1b-51a0-4fa3-b0b9-ebdb580b66f0 req-dd440d57-e1ee-441a-a11f-b60651a6c0dd service nova] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Received unexpected event network-vif-plugged-2dcfe7b8-34fe-4860-b820-f3de7c00df62 for instance with vm_state building and task_state spawning.
Aug 09 02:31:04 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548264.9083576, 49a10075-0d3c-4cd6-bb43-10238cad3f37 => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:31:04 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] VM Resumed (Lifecycle Event)
Aug 09 02:31:04 user nova-compute[72351]: DEBUG nova.compute.manager [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:31:04 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Guest created on hypervisor {{(pid=72351) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4411}}
Aug 09 02:31:04 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Instance spawned successfully.
Aug 09 02:31:04 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:894}}
Aug 09 02:31:04 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:31:04 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:31:04 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Found default for hw_cdrom_bus of ide {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:31:04 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Found default for hw_disk_bus of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:31:04 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Found default for hw_input_bus of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:31:04 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Found default for hw_pointer_model of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:31:04 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Found default for hw_video_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:31:04 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Found default for hw_vif_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:31:04 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:31:04 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548264.9105873, 49a10075-0d3c-4cd6-bb43-10238cad3f37 => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:31:04 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] VM Started (Lifecycle Event)
Aug 09 02:31:05 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:31:05 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:31:05 user nova-compute[72351]: INFO nova.compute.manager [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Took 6.02 seconds to spawn the instance on the hypervisor.
Aug 09 02:31:05 user nova-compute[72351]: DEBUG nova.compute.manager [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:31:05 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:31:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-3da51dcd-c16a-4e72-9a2c-302f242116ee tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 9df16e46-d097-4553-b44c-732ea24a4169 {{(pid=72351) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2589}}
Aug 09 02:31:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Received event <DeviceRemovedEvent: 1691548265.1134915, 9df16e46-d097-4553-b44c-732ea24a4169 => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=72351) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2360}}
Aug 09 02:31:05 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-3da51dcd-c16a-4e72-9a2c-302f242116ee tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Successfully detached device vdb from instance 9df16e46-d097-4553-b44c-732ea24a4169 from the live domain config.
Aug 09 02:31:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-3da51dcd-c16a-4e72-9a2c-302f242116ee tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 02:31:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-3da51dcd-c16a-4e72-9a2c-302f242116ee tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:3cb561e0-d189-4a63-abe1-3a68ffbe4616'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'GKjCnTvV', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sde'}, None), 'kwargs': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:31:05 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-3da51dcd-c16a-4e72-9a2c-302f242116ee tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:31:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-3da51dcd-c16a-4e72-9a2c-302f242116ee tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:31:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-3da51dcd-c16a-4e72-9a2c-302f242116ee tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:31:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-3da51dcd-c16a-4e72-9a2c-302f242116ee tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:3cb561e0-d189-4a63-abe1-3a68ffbe4616', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 02:31:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.035s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:05 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[42409811-4f14-46d8-9ead-b1a5629ce752]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:3cb561e0-d189-4a63-abe1-3a68ffbe4616\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.049s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:05 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[8ef76e7f-27b1-4dbb-a08c-f002f399b035]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e (non-flash)\ntcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70 (non-flash)\ntcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3cb561e0-d189-4a63-abe1-3a68ffbe4616 (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-3da51dcd-c16a-4e72-9a2c-302f242116ee tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:31:05 user nova-compute[72351]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 (non-flash)
Aug 09 02:31:05 user nova-compute[72351]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e (non-flash)
Aug 09 02:31:05 user nova-compute[72351]: tcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70 (non-flash)
Aug 09 02:31:05 user nova-compute[72351]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3cb561e0-d189-4a63-abe1-3a68ffbe4616 (non-flash)
Aug 09 02:31:05 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:31:05 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:31:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-3da51dcd-c16a-4e72-9a2c-302f242116ee tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:31:05 user nova-compute[72351]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 (non-flash)
Aug 09 02:31:05 user nova-compute[72351]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e (non-flash)
Aug 09 02:31:05 user nova-compute[72351]: tcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70 (non-flash)
Aug 09 02:31:05 user nova-compute[72351]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3cb561e0-d189-4a63-abe1-3a68ffbe4616 (non-flash)
Aug 09 02:31:05 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:31:05 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:31:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-3da51dcd-c16a-4e72-9a2c-302f242116ee tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fcfbc63c5e0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:3cb561e0-d189-4a63-abe1-3a68ffbe4616'): ({'sde'}, set())}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 02:31:05 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-3da51dcd-c16a-4e72-9a2c-302f242116ee tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Removing single pathed devices sde {{(pid=72351) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Aug 09 02:31:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:05 user nova-compute[72351]: INFO nova.compute.manager [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Took 26.74 seconds to build instance.
Aug 09 02:31:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.026s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:05 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[36adb895-63a7-4dc3-9c19-fc5d519e169d]: (4, ('path checker states:\nup                  6\n\npaths: 0\nbusy: False\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sde {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a5ec17e8-e884-40b6-ba29-0ff10d333ade tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "49a10075-0d3c-4cd6-bb43-10238cad3f37" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 26.896s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:31:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sde" returned: 0 in 0.032s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:05 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[26e3f6b1-e74c-46e2-9917-3a7243debc3c]: (4, ('ok\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:05 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-3da51dcd-c16a-4e72-9a2c-302f242116ee tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Flushing IO for device /dev/sde {{(pid=72351) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Aug 09 02:31:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sde {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sde" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:05 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[edcefc12-4fba-49bc-9280-7384af66cb07]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:05 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-3da51dcd-c16a-4e72-9a2c-302f242116ee tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Remove SCSI device /dev/sde with /sys/block/sde/device/delete {{(pid=72351) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Aug 09 02:31:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sde/device/delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sde/device/delete" returned: 0 in 0.059s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:05 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[0df9de29-7767-4cac-a24e-63a54e22bd04]: (4, ('1', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:05 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-3da51dcd-c16a-4e72-9a2c-302f242116ee tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Checking to see if SCSI volumes sde have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Aug 09 02:31:05 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-3da51dcd-c16a-4e72-9a2c-302f242116ee tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] SCSI volumes sde have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Aug 09 02:31:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-3da51dcd-c16a-4e72-9a2c-302f242116ee tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:3cb561e0-d189-4a63-abe1-3a68ffbe4616')] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 02:31:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3cb561e0-d189-4a63-abe1-3a68ffbe4616 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3cb561e0-d189-4a63-abe1-3a68ffbe4616 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.025s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:05 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[7f5173a6-23b9-471a-9eb0-128ae72f6058]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-3da51dcd-c16a-4e72-9a2c-302f242116ee tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:31:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3cb561e0-d189-4a63-abe1-3a68ffbe4616 -p 172.16.0.220:3260 --logout {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json" returned: 0 in 0.176s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3cb561e0-d189-4a63-abe1-3a68ffbe4616 -p 172.16.0.220:3260 --logout" returned: 0 in 0.078s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:05 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[1efddbdf-c49c-46ec-ab13-6559e7c1159e]: (4, ('Logging out of session [sid: 36, target: iqn.2010-10.org.openstack:3cb561e0-d189-4a63-abe1-3a68ffbe4616, portal: 172.16.0.220,3260]\nLogout of [sid: 36, target: iqn.2010-10.org.openstack:3cb561e0-d189-4a63-abe1-3a68ffbe4616, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-3da51dcd-c16a-4e72-9a2c-302f242116ee tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 36, target: iqn.2010-10.org.openstack:3cb561e0-d189-4a63-abe1-3a68ffbe4616, portal: 172.16.0.220,3260]
Aug 09 02:31:05 user nova-compute[72351]: Logout of [sid: 36, target: iqn.2010-10.org.openstack:3cb561e0-d189-4a63-abe1-3a68ffbe4616, portal: 172.16.0.220,3260] successful.
Aug 09 02:31:05 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:31:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3cb561e0-d189-4a63-abe1-3a68ffbe4616 -p 172.16.0.220:3260 --op delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3cb561e0-d189-4a63-abe1-3a68ffbe4616 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.037s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:05 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[b50a51fd-78f1-4c1e-aa2b-3177c1154c14]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-3da51dcd-c16a-4e72-9a2c-302f242116ee tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:31:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-3da51dcd-c16a-4e72-9a2c-302f242116ee tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.484s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:31:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-3da51dcd-c16a-4e72-9a2c-302f242116ee tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] <== disconnect_volume: return (486ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:31:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-3da51dcd-c16a-4e72-9a2c-302f242116ee tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 02:31:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json" returned: 0 in 0.163s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdg (vda) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:31:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/47afce95-e5f7-4703-a99f-7ef8b28c4bd2/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/47afce95-e5f7-4703-a99f-7ef8b28c4bd2/disk --force-share --output=json" returned: 0 in 0.195s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/47afce95-e5f7-4703-a99f-7ef8b28c4bd2/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/47afce95-e5f7-4703-a99f-7ef8b28c4bd2/disk --force-share --output=json" returned: 0 in 0.196s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.rescue --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.rescue --force-share --output=json" returned: 0 in 0.170s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.rescue --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.rescue --force-share --output=json" returned: 0 in 0.186s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json" returned: 0 in 0.178s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json" returned: 0 in 0.184s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json" returned: 0 in 0.161s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json" returned: 0 in 0.157s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:07 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdf (vda) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:31:07 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:31:07 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdc (vdb) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:31:07 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdd (vdc) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:31:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:07 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json" returned: 0 in 0.184s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json" returned: 0 in 0.159s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7b6f5d60-5f23-4ef1-b972-9ba7086f56cb/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7b6f5d60-5f23-4ef1-b972-9ba7086f56cb/disk --force-share --output=json" returned: 0 in 0.191s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7b6f5d60-5f23-4ef1-b972-9ba7086f56cb/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7b6f5d60-5f23-4ef1-b972-9ba7086f56cb/disk --force-share --output=json" returned: 0 in 0.203s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a03fbded-23ad-440b-97e7-bea07425e873/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a03fbded-23ad-440b-97e7-bea07425e873/disk --force-share --output=json" returned: 0 in 0.192s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a03fbded-23ad-440b-97e7-bea07425e873/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a03fbded-23ad-440b-97e7-bea07425e873/disk --force-share --output=json" returned: 0 in 0.179s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f37b7fb7-1b94-421b-be79-7066010efabf/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Acquiring lock "8e970bdd-54a1-4058-b09d-784cb6b5b1a4" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:31:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Lock "8e970bdd-54a1-4058-b09d-784cb6b5b1a4" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:31:08 user nova-compute[72351]: DEBUG nova.compute.manager [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Starting instance... {{(pid=72351) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2406}}
Aug 09 02:31:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f37b7fb7-1b94-421b-be79-7066010efabf/disk --force-share --output=json" returned: 0 in 0.182s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f37b7fb7-1b94-421b-be79-7066010efabf/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:31:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.003s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:31:08 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72351) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Aug 09 02:31:08 user nova-compute[72351]: INFO nova.compute.claims [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Claim successful on node user
Aug 09 02:31:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f37b7fb7-1b94-421b-be79-7066010efabf/disk --force-share --output=json" returned: 0 in 0.180s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9df16e46-d097-4553-b44c-732ea24a4169/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:09 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9df16e46-d097-4553-b44c-732ea24a4169/disk --force-share --output=json" returned: 0 in 0.167s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:09 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9df16e46-d097-4553-b44c-732ea24a4169/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:09 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9df16e46-d097-4553-b44c-732ea24a4169/disk --force-share --output=json" returned: 0 in 0.178s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:09 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:31:09 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:31:09 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Hypervisor/Node resource view: name=user free_ram=7117MB free_disk=25.133861541748047GB free_vcpus=0 pci_devices=[{"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72351) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Aug 09 02:31:09 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:31:10 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:31:10 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:31:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 1.400s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:31:10 user nova-compute[72351]: DEBUG nova.compute.manager [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Start building networks asynchronously for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2803}}
Aug 09 02:31:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.307s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:31:10 user nova-compute[72351]: DEBUG nova.compute.manager [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Allocating IP information in the background. {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1956}}
Aug 09 02:31:10 user nova-compute[72351]: DEBUG nova.network.neutron [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] allocate_for_instance() {{(pid=72351) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Aug 09 02:31:10 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Aug 09 02:31:10 user nova-compute[72351]: DEBUG nova.compute.manager [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Start building block device mappings for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2838}}
Aug 09 02:31:10 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:31:10 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance eb86cb49-2018-42c2-bd97-c662b829775d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:31:10 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:31:10 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance a7f3d076-864b-4b68-9207-214ce2c21444 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:31:10 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 47afce95-e5f7-4703-a99f-7ef8b28c4bd2 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:31:10 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 957c1510-1320-4492-9052-a58ff540319d actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:31:10 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance a03fbded-23ad-440b-97e7-bea07425e873 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:31:10 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 9df16e46-d097-4553-b44c-732ea24a4169 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:31:10 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 38a2b8e3-337d-416a-8a8c-da4921123db6 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:31:10 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:31:10 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 49a10075-0d3c-4cd6-bb43-10238cad3f37 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:31:10 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance f37b7fb7-1b94-421b-be79-7066010efabf actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:31:10 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 8e970bdd-54a1-4058-b09d-784cb6b5b1a4 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:31:10 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Total usable vcpus: 12, total allocated vcpus: 13 {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Aug 09 02:31:10 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Final resource view: name=user phys_ram=16011MB used_ram=7168MB phys_disk=40GB used_disk=9GB total_vcpus=12 used_vcpus=13 pci_stats=[] {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Aug 09 02:31:10 user nova-compute[72351]: INFO nova.virt.block_device [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Booting with volume bc173cc3-1a94-467d-8d0b-d0f50936cee3 at /dev/vda
Aug 09 02:31:10 user nova-compute[72351]: DEBUG os_brick.utils [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:31:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:10 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[063a6e2d-0437-4e98-abce-6d6f0e293828]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:a52269602ff4\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:10 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[c8d4741a-7e06-4233-a3e1-3440e44651d0]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:10 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:10 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[8dae4cc4-3c3d-4c29-8671-1a36a0e314d6]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:10 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[08d3311b-cd67-4f46-b0bf-54621b070343]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Running cmd (subprocess): nvme version {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] 'nvme version' failed. Not Retrying. {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:31:10 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.nvmeof [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] nvme not present on system {{(pid=72351) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Aug 09 02:31:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:31:10 user nova-compute[72351]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Aug 09 02:31:10 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[7efec8d0-a173-4bef-85bb-85279deb8ad0]: (4, '') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:10 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72351) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Aug 09 02:31:10 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Aug 09 02:31:10 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] LIGHTOS: no hostnqn found. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Aug 09 02:31:10 user nova-compute[72351]: DEBUG os_brick.utils [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] <== get_connector_properties: return (151ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:a52269602ff4', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:31:10 user nova-compute[72351]: DEBUG nova.virt.block_device [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Updating existing volume attachment record: 74c12cc6-0e3f-4efa-91a0-029acb5b9d5a {{(pid=72351) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Aug 09 02:31:10 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:31:10 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:31:10 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Compute_service record updated for user:user {{(pid=72351) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Aug 09 02:31:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.738s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:31:11 user nova-compute[72351]: DEBUG nova.policy [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '0aa403f4ead84aecb78a538c2ced64e9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4d0cc057c220422a9541bcb9fa7f598b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72351) authorize /opt/stack/nova/nova/policy.py:203}}
Aug 09 02:31:11 user nova-compute[72351]: DEBUG nova.objects.instance [None req-3da51dcd-c16a-4e72-9a2c-302f242116ee tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lazy-loading 'flavor' on Instance uuid 9df16e46-d097-4553-b44c-732ea24a4169 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:31:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-3da51dcd-c16a-4e72-9a2c-302f242116ee tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lock "9df16e46-d097-4553-b44c-732ea24a4169" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 7.307s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:31:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-dfb7407b-1a34-4ea2-bb71-c1827c95d882 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lock "9df16e46-d097-4553-b44c-732ea24a4169" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 7.108s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:31:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-dfb7407b-1a34-4ea2-bb71-c1827c95d882 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Acquiring lock "9df16e46-d097-4553-b44c-732ea24a4169-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:31:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-dfb7407b-1a34-4ea2-bb71-c1827c95d882 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lock "9df16e46-d097-4553-b44c-732ea24a4169-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:31:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-dfb7407b-1a34-4ea2-bb71-c1827c95d882 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lock "9df16e46-d097-4553-b44c-732ea24a4169-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:31:11 user nova-compute[72351]: INFO nova.compute.manager [None req-dfb7407b-1a34-4ea2-bb71-c1827c95d882 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Terminating instance
Aug 09 02:31:11 user nova-compute[72351]: DEBUG nova.compute.manager [None req-dfb7407b-1a34-4ea2-bb71-c1827c95d882 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Start destroying the instance on the hypervisor. {{(pid=72351) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3124}}
Aug 09 02:31:11 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:11 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:11 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:11 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:31:11 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:31:11 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:31:11 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Starting heal instance info cache {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9842}}
Aug 09 02:31:11 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:11 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "refresh_cache-eb86cb49-2018-42c2-bd97-c662b829775d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:31:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquired lock "refresh_cache-eb86cb49-2018-42c2-bd97-c662b829775d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:31:12 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Forcefully refreshing network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1996}}
Aug 09 02:31:12 user nova-compute[72351]: DEBUG nova.compute.manager [req-a823a9b9-ad4b-4c6b-898d-53f7c5251ca4 req-5810ff36-1f98-4488-8f99-35de8e4ead74 service nova] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Received event network-vif-unplugged-e8c25a1c-a605-4d7a-bb03-395b3d844571 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:31:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a823a9b9-ad4b-4c6b-898d-53f7c5251ca4 req-5810ff36-1f98-4488-8f99-35de8e4ead74 service nova] Acquiring lock "9df16e46-d097-4553-b44c-732ea24a4169-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:31:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a823a9b9-ad4b-4c6b-898d-53f7c5251ca4 req-5810ff36-1f98-4488-8f99-35de8e4ead74 service nova] Lock "9df16e46-d097-4553-b44c-732ea24a4169-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:31:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a823a9b9-ad4b-4c6b-898d-53f7c5251ca4 req-5810ff36-1f98-4488-8f99-35de8e4ead74 service nova] Lock "9df16e46-d097-4553-b44c-732ea24a4169-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:31:12 user nova-compute[72351]: DEBUG nova.compute.manager [req-a823a9b9-ad4b-4c6b-898d-53f7c5251ca4 req-5810ff36-1f98-4488-8f99-35de8e4ead74 service nova] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] No waiting events found dispatching network-vif-unplugged-e8c25a1c-a605-4d7a-bb03-395b3d844571 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:31:12 user nova-compute[72351]: DEBUG nova.compute.manager [req-a823a9b9-ad4b-4c6b-898d-53f7c5251ca4 req-5810ff36-1f98-4488-8f99-35de8e4ead74 service nova] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Received event network-vif-unplugged-e8c25a1c-a605-4d7a-bb03-395b3d844571 for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:31:12 user nova-compute[72351]: DEBUG nova.network.neutron [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Successfully created port: a83afb1f-511f-406c-b925-eb70b32bed90 {{(pid=72351) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Aug 09 02:31:12 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Instance destroyed successfully.
Aug 09 02:31:12 user nova-compute[72351]: DEBUG nova.objects.instance [None req-dfb7407b-1a34-4ea2-bb71-c1827c95d882 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lazy-loading 'resources' on Instance uuid 9df16e46-d097-4553-b44c-732ea24a4169 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:31:12 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-dfb7407b-1a34-4ea2-bb71-c1827c95d882 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:28:20Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-VolumesSnapshotTestJSON-instance-331359218',display_name='tempest-VolumesSnapshotTestJSON-instance-331359218',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-volumessnapshottestjson-instance-331359218',id=32,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBK3B9vHv/qSnUSCwj6Rdzu2263gZuLWjCHjmlx8FcOAqP+5T4EY3PruhW7fbCfv9oefdX6GMInUABslwsRJw+NQCIBUj+l8LlSPF2Ts4uLlBIlXGEb16HrWbz9sTVz9uHQ==',key_name='tempest-keypair-1656343598',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:28:29Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='1c8ff52319444079b754f79843c67b2b',ramdisk_id='',reservation_id='r-w9rkeroj',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-VolumesSnapshotTestJSON-343261326',owner_user_name='tempest-VolumesSnapshotTestJSON-343261326-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:28:29Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='97267b9f897e4034b71627a698210071',uuid=9df16e46-d097-4553-b44c-732ea24a4169,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "e8c25a1c-a605-4d7a-bb03-395b3d844571", "address": "fa:16:3e:69:6a:fa", "network": {"id": "96018a29-3b2e-44de-b2c4-67abc92e3a30", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-924549970-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.167", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "1c8ff52319444079b754f79843c67b2b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape8c25a1c-a6", "ovs_interfaceid": "e8c25a1c-a605-4d7a-bb03-395b3d844571", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:31:12 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-dfb7407b-1a34-4ea2-bb71-c1827c95d882 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Converting VIF {"id": "e8c25a1c-a605-4d7a-bb03-395b3d844571", "address": "fa:16:3e:69:6a:fa", "network": {"id": "96018a29-3b2e-44de-b2c4-67abc92e3a30", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-924549970-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.167", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "1c8ff52319444079b754f79843c67b2b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape8c25a1c-a6", "ovs_interfaceid": "e8c25a1c-a605-4d7a-bb03-395b3d844571", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:31:12 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-dfb7407b-1a34-4ea2-bb71-c1827c95d882 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:69:6a:fa,bridge_name='br-int',has_traffic_filtering=True,id=e8c25a1c-a605-4d7a-bb03-395b3d844571,network=Network(96018a29-3b2e-44de-b2c4-67abc92e3a30),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape8c25a1c-a6') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:31:12 user nova-compute[72351]: DEBUG os_vif [None req-dfb7407b-1a34-4ea2-bb71-c1827c95d882 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:69:6a:fa,bridge_name='br-int',has_traffic_filtering=True,id=e8c25a1c-a605-4d7a-bb03-395b3d844571,network=Network(96018a29-3b2e-44de-b2c4-67abc92e3a30),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape8c25a1c-a6') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:31:12 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:12 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tape8c25a1c-a6, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:31:12 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:12 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:31:12 user nova-compute[72351]: INFO os_vif [None req-dfb7407b-1a34-4ea2-bb71-c1827c95d882 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:69:6a:fa,bridge_name='br-int',has_traffic_filtering=True,id=e8c25a1c-a605-4d7a-bb03-395b3d844571,network=Network(96018a29-3b2e-44de-b2c4-67abc92e3a30),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape8c25a1c-a6')
Aug 09 02:31:12 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-dfb7407b-1a34-4ea2-bb71-c1827c95d882 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 02:31:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-dfb7407b-1a34-4ea2-bb71-c1827c95d882 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:3cb561e0-d189-4a63-abe1-3a68ffbe4616'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'GKjCnTvV', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sde'}, None), 'kwargs': True}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:31:12 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-dfb7407b-1a34-4ea2-bb71-c1827c95d882 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:31:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-dfb7407b-1a34-4ea2-bb71-c1827c95d882 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:31:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-dfb7407b-1a34-4ea2-bb71-c1827c95d882 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:31:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-dfb7407b-1a34-4ea2-bb71-c1827c95d882 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:3cb561e0-d189-4a63-abe1-3a68ffbe4616', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 02:31:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.033s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:12 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ab00e21a-5867-49df-a1f4-90b09bbe59bd]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.029s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:12 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[26421117-54cd-4c5e-ad46-9c6ed2e44526]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e (non-flash)\ntcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70 (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-dfb7407b-1a34-4ea2-bb71-c1827c95d882 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:31:12 user nova-compute[72351]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 (non-flash)
Aug 09 02:31:12 user nova-compute[72351]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e (non-flash)
Aug 09 02:31:12 user nova-compute[72351]: tcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70 (non-flash)
Aug 09 02:31:12 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:31:12 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:31:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-dfb7407b-1a34-4ea2-bb71-c1827c95d882 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:31:12 user nova-compute[72351]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 (non-flash)
Aug 09 02:31:12 user nova-compute[72351]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e (non-flash)
Aug 09 02:31:12 user nova-compute[72351]: tcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70 (non-flash)
Aug 09 02:31:12 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:31:12 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:31:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-dfb7407b-1a34-4ea2-bb71-c1827c95d882 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fcfdc18b250>, {}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 02:31:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-dfb7407b-1a34-4ea2-bb71-c1827c95d882 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Disconnecting from: [] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 02:31:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-dfb7407b-1a34-4ea2-bb71-c1827c95d882 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.080s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:31:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-dfb7407b-1a34-4ea2-bb71-c1827c95d882 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] <== disconnect_volume: return (83ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:31:12 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-dfb7407b-1a34-4ea2-bb71-c1827c95d882 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 02:31:12 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-dfb7407b-1a34-4ea2-bb71-c1827c95d882 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Deleting instance files /opt/stack/data/nova/instances/9df16e46-d097-4553-b44c-732ea24a4169_del
Aug 09 02:31:12 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-dfb7407b-1a34-4ea2-bb71-c1827c95d882 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Deletion of /opt/stack/data/nova/instances/9df16e46-d097-4553-b44c-732ea24a4169_del complete
Aug 09 02:31:12 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Updating instance_info_cache with network_info: [{"id": "5f97db70-9e5b-4d42-b765-b832fb522f41", "address": "fa:16:3e:dc:da:c7", "network": {"id": "49e8751c-e119-430f-9110-3dc6069b1589", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-899275135-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3aea1e66805146f7a8cd1d97e64a24af", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5f97db70-9e", "ovs_interfaceid": "5f97db70-9e5b-4d42-b765-b832fb522f41", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:31:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Releasing lock "refresh_cache-eb86cb49-2018-42c2-bd97-c662b829775d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:31:12 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Updated the network info_cache for instance {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9913}}
Aug 09 02:31:12 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:31:12 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:31:12 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:31:12 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:31:12 user nova-compute[72351]: INFO nova.compute.manager [None req-dfb7407b-1a34-4ea2-bb71-c1827c95d882 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Took 1.25 seconds to destroy the instance on the hypervisor.
Aug 09 02:31:12 user nova-compute[72351]: DEBUG oslo.service.loopingcall [None req-dfb7407b-1a34-4ea2-bb71-c1827c95d882 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72351) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Aug 09 02:31:12 user nova-compute[72351]: DEBUG nova.compute.manager [-] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Deallocating network for instance {{(pid=72351) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2263}}
Aug 09 02:31:12 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] deallocate_for_instance() {{(pid=72351) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1795}}
Aug 09 02:31:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:14 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:31:14 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Took 1.12 seconds to deallocate network for instance.
Aug 09 02:31:14 user nova-compute[72351]: DEBUG nova.compute.manager [req-0924341d-1b11-4429-afdc-d7e323179ed2 req-090fc504-5f15-4164-90f6-0dd3fa348dca service nova] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Received event network-vif-deleted-e8c25a1c-a605-4d7a-bb03-395b3d844571 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:31:14 user nova-compute[72351]: INFO nova.compute.manager [req-0924341d-1b11-4429-afdc-d7e323179ed2 req-090fc504-5f15-4164-90f6-0dd3fa348dca service nova] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Neutron deleted interface e8c25a1c-a605-4d7a-bb03-395b3d844571; detaching it from the instance and deleting it from the info cache
Aug 09 02:31:14 user nova-compute[72351]: DEBUG nova.network.neutron [req-0924341d-1b11-4429-afdc-d7e323179ed2 req-090fc504-5f15-4164-90f6-0dd3fa348dca service nova] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:31:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Acquiring lock "80e87454-90b9-4168-bf00-b7d93b245526" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:31:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "80e87454-90b9-4168-bf00-b7d93b245526" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:31:14 user nova-compute[72351]: WARNING nova.volume.cinder [None req-dfb7407b-1a34-4ea2-bb71-c1827c95d882 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Attachment f35a4e78-cb7a-4727-9763-80a042d9174d does not exist. Ignoring.: cinderclient.exceptions.NotFound: Volume attachment could not be found with filter: attachment_id = f35a4e78-cb7a-4727-9763-80a042d9174d. (HTTP 404) (Request-ID: req-cb496297-c20c-49a5-90ea-20352776ca87)
Aug 09 02:31:14 user nova-compute[72351]: INFO nova.compute.manager [None req-dfb7407b-1a34-4ea2-bb71-c1827c95d882 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Took 0.05 seconds to detach 1 volumes for instance.
Aug 09 02:31:14 user nova-compute[72351]: DEBUG nova.compute.manager [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Starting instance... {{(pid=72351) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2406}}
Aug 09 02:31:14 user nova-compute[72351]: DEBUG nova.compute.manager [req-0924341d-1b11-4429-afdc-d7e323179ed2 req-090fc504-5f15-4164-90f6-0dd3fa348dca service nova] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Detach interface failed, port_id=e8c25a1c-a605-4d7a-bb03-395b3d844571, reason: Instance 9df16e46-d097-4553-b44c-732ea24a4169 could not be found. {{(pid=72351) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10866}}
Aug 09 02:31:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-dfb7407b-1a34-4ea2-bb71-c1827c95d882 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:31:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-dfb7407b-1a34-4ea2-bb71-c1827c95d882 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:31:14 user nova-compute[72351]: DEBUG nova.compute.manager [req-46044446-3e69-44fd-bff0-334d3297d5f5 req-a1ca7097-c8ee-4c0e-a614-7bd56a44d8a2 service nova] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Received event network-vif-plugged-e8c25a1c-a605-4d7a-bb03-395b3d844571 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:31:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-46044446-3e69-44fd-bff0-334d3297d5f5 req-a1ca7097-c8ee-4c0e-a614-7bd56a44d8a2 service nova] Acquiring lock "9df16e46-d097-4553-b44c-732ea24a4169-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:31:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-46044446-3e69-44fd-bff0-334d3297d5f5 req-a1ca7097-c8ee-4c0e-a614-7bd56a44d8a2 service nova] Lock "9df16e46-d097-4553-b44c-732ea24a4169-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.004s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:31:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-46044446-3e69-44fd-bff0-334d3297d5f5 req-a1ca7097-c8ee-4c0e-a614-7bd56a44d8a2 service nova] Lock "9df16e46-d097-4553-b44c-732ea24a4169-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:31:14 user nova-compute[72351]: DEBUG nova.compute.manager [req-46044446-3e69-44fd-bff0-334d3297d5f5 req-a1ca7097-c8ee-4c0e-a614-7bd56a44d8a2 service nova] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] No waiting events found dispatching network-vif-plugged-e8c25a1c-a605-4d7a-bb03-395b3d844571 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:31:14 user nova-compute[72351]: WARNING nova.compute.manager [req-46044446-3e69-44fd-bff0-334d3297d5f5 req-a1ca7097-c8ee-4c0e-a614-7bd56a44d8a2 service nova] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Received unexpected event network-vif-plugged-e8c25a1c-a605-4d7a-bb03-395b3d844571 for instance with vm_state active and task_state deleting.
Aug 09 02:31:14 user nova-compute[72351]: DEBUG nova.network.neutron [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Successfully updated port: a83afb1f-511f-406c-b925-eb70b32bed90 {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:31:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Acquiring lock "refresh_cache-8e970bdd-54a1-4058-b09d-784cb6b5b1a4" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:31:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Acquired lock "refresh_cache-8e970bdd-54a1-4058-b09d-784cb6b5b1a4" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:31:14 user nova-compute[72351]: DEBUG nova.network.neutron [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:31:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:31:14 user nova-compute[72351]: DEBUG nova.compute.manager [req-fa581cbf-45f3-4c75-8955-2411fcf89adc req-84a6b459-48a1-4796-b7f6-249de29b973a service nova] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Received event network-changed-a83afb1f-511f-406c-b925-eb70b32bed90 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:31:14 user nova-compute[72351]: DEBUG nova.compute.manager [req-fa581cbf-45f3-4c75-8955-2411fcf89adc req-84a6b459-48a1-4796-b7f6-249de29b973a service nova] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Refreshing instance network info cache due to event network-changed-a83afb1f-511f-406c-b925-eb70b32bed90. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:31:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-fa581cbf-45f3-4c75-8955-2411fcf89adc req-84a6b459-48a1-4796-b7f6-249de29b973a service nova] Acquiring lock "refresh_cache-8e970bdd-54a1-4058-b09d-784cb6b5b1a4" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:31:14 user nova-compute[72351]: DEBUG nova.network.neutron [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:31:14 user nova-compute[72351]: DEBUG nova.network.neutron [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Updating instance_info_cache with network_info: [{"id": "a83afb1f-511f-406c-b925-eb70b32bed90", "address": "fa:16:3e:16:83:e9", "network": {"id": "0b3a34b6-6d32-4414-9135-22bd2a1dae20", "bridge": "br-int", "label": "tempest-TestShelveInstance-2024538322-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4d0cc057c220422a9541bcb9fa7f598b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa83afb1f-51", "ovs_interfaceid": "a83afb1f-511f-406c-b925-eb70b32bed90", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:31:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Releasing lock "refresh_cache-8e970bdd-54a1-4058-b09d-784cb6b5b1a4" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:31:14 user nova-compute[72351]: DEBUG nova.compute.manager [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Instance network_info: |[{"id": "a83afb1f-511f-406c-b925-eb70b32bed90", "address": "fa:16:3e:16:83:e9", "network": {"id": "0b3a34b6-6d32-4414-9135-22bd2a1dae20", "bridge": "br-int", "label": "tempest-TestShelveInstance-2024538322-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4d0cc057c220422a9541bcb9fa7f598b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa83afb1f-51", "ovs_interfaceid": "a83afb1f-511f-406c-b925-eb70b32bed90", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1971}}
Aug 09 02:31:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-fa581cbf-45f3-4c75-8955-2411fcf89adc req-84a6b459-48a1-4796-b7f6-249de29b973a service nova] Acquired lock "refresh_cache-8e970bdd-54a1-4058-b09d-784cb6b5b1a4" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:31:14 user nova-compute[72351]: DEBUG nova.network.neutron [req-fa581cbf-45f3-4c75-8955-2411fcf89adc req-84a6b459-48a1-4796-b7f6-249de29b973a service nova] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Refreshing network info cache for port a83afb1f-511f-406c-b925-eb70b32bed90 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:31:14 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-dfb7407b-1a34-4ea2-bb71-c1827c95d882 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:31:14 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-dfb7407b-1a34-4ea2-bb71-c1827c95d882 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:31:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-dfb7407b-1a34-4ea2-bb71-c1827c95d882 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.529s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:31:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.464s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:31:14 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72351) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Aug 09 02:31:14 user nova-compute[72351]: INFO nova.compute.claims [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Claim successful on node user
Aug 09 02:31:14 user nova-compute[72351]: INFO nova.scheduler.client.report [None req-dfb7407b-1a34-4ea2-bb71-c1827c95d882 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Deleted allocations for instance 9df16e46-d097-4553-b44c-732ea24a4169
Aug 09 02:31:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-dfb7407b-1a34-4ea2-bb71-c1827c95d882 tempest-VolumesSnapshotTestJSON-343261326 tempest-VolumesSnapshotTestJSON-343261326-project-member] Lock "9df16e46-d097-4553-b44c-732ea24a4169" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 3.405s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:31:15 user nova-compute[72351]: DEBUG nova.network.neutron [req-fa581cbf-45f3-4c75-8955-2411fcf89adc req-84a6b459-48a1-4796-b7f6-249de29b973a service nova] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Updated VIF entry in instance network info cache for port a83afb1f-511f-406c-b925-eb70b32bed90. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:31:15 user nova-compute[72351]: DEBUG nova.network.neutron [req-fa581cbf-45f3-4c75-8955-2411fcf89adc req-84a6b459-48a1-4796-b7f6-249de29b973a service nova] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Updating instance_info_cache with network_info: [{"id": "a83afb1f-511f-406c-b925-eb70b32bed90", "address": "fa:16:3e:16:83:e9", "network": {"id": "0b3a34b6-6d32-4414-9135-22bd2a1dae20", "bridge": "br-int", "label": "tempest-TestShelveInstance-2024538322-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4d0cc057c220422a9541bcb9fa7f598b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa83afb1f-51", "ovs_interfaceid": "a83afb1f-511f-406c-b925-eb70b32bed90", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:31:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-fa581cbf-45f3-4c75-8955-2411fcf89adc req-84a6b459-48a1-4796-b7f6-249de29b973a service nova] Releasing lock "refresh_cache-8e970bdd-54a1-4058-b09d-784cb6b5b1a4" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:31:15 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:31:15 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:31:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.768s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:31:15 user nova-compute[72351]: DEBUG nova.compute.manager [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Start building networks asynchronously for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2803}}
Aug 09 02:31:15 user nova-compute[72351]: DEBUG nova.compute.manager [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Allocating IP information in the background. {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1956}}
Aug 09 02:31:15 user nova-compute[72351]: DEBUG nova.network.neutron [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] allocate_for_instance() {{(pid=72351) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Aug 09 02:31:15 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Aug 09 02:31:15 user nova-compute[72351]: DEBUG nova.compute.manager [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Start building block device mappings for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2838}}
Aug 09 02:31:15 user nova-compute[72351]: INFO nova.virt.block_device [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Booting with volume 7e32fe78-f2c4-409c-813a-36924e611238 at /dev/vda
Aug 09 02:31:15 user nova-compute[72351]: DEBUG nova.policy [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '5dca34fdadbb4386a1ff1cb255265d47', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '69d584c230b94ff8aae722eb2f364c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72351) authorize /opt/stack/nova/nova/policy.py:203}}
Aug 09 02:31:15 user nova-compute[72351]: DEBUG os_brick.utils [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:31:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:15 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[9ff49ba4-acf5-4e83-bdf3-f228dc214ce7]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:a52269602ff4\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:15 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[f0478ce0-ad13-4939-84b7-23c92911e724]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:15 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[06e95029-8b35-4192-bf26-9630d1193ee6]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:15 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[40f4975a-e5cc-4909-9542-2ba0c794ac5c]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Running cmd (subprocess): nvme version {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] 'nvme version' failed. Not Retrying. {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:31:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.nvmeof [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] nvme not present on system {{(pid=72351) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Aug 09 02:31:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:31:16 user nova-compute[72351]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Aug 09 02:31:16 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[3fba86d7-77d1-47b0-8b50-93aa42de8ab3]: (4, '') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:16 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72351) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Aug 09 02:31:16 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Aug 09 02:31:16 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] LIGHTOS: no hostnqn found. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Aug 09 02:31:16 user nova-compute[72351]: DEBUG os_brick.utils [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] <== get_connector_properties: return (139ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:a52269602ff4', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:31:16 user nova-compute[72351]: DEBUG nova.virt.block_device [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Updating existing volume attachment record: 481ac50c-3df0-468a-80c8-d0e993871c77 {{(pid=72351) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Aug 09 02:31:16 user nova-compute[72351]: DEBUG nova.network.neutron [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Successfully created port: fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4 {{(pid=72351) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Aug 09 02:31:17 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:17 user nova-compute[72351]: DEBUG nova.network.neutron [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Successfully updated port: fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4 {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:31:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Acquiring lock "refresh_cache-80e87454-90b9-4168-bf00-b7d93b245526" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:31:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Acquired lock "refresh_cache-80e87454-90b9-4168-bf00-b7d93b245526" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:31:17 user nova-compute[72351]: DEBUG nova.network.neutron [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:31:17 user nova-compute[72351]: DEBUG nova.compute.manager [req-ef166962-6b97-4054-ab6a-30c0d051f1e6 req-105aedd7-88d0-47d1-8c6e-d0faf7870e0c service nova] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Received event network-changed-fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:31:17 user nova-compute[72351]: DEBUG nova.compute.manager [req-ef166962-6b97-4054-ab6a-30c0d051f1e6 req-105aedd7-88d0-47d1-8c6e-d0faf7870e0c service nova] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Refreshing instance network info cache due to event network-changed-fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:31:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ef166962-6b97-4054-ab6a-30c0d051f1e6 req-105aedd7-88d0-47d1-8c6e-d0faf7870e0c service nova] Acquiring lock "refresh_cache-80e87454-90b9-4168-bf00-b7d93b245526" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:31:18 user nova-compute[72351]: DEBUG nova.network.neutron [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:31:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:18 user nova-compute[72351]: DEBUG nova.network.neutron [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Updating instance_info_cache with network_info: [{"id": "fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4", "address": "fa:16:3e:7a:77:69", "network": {"id": "b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-210689817-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "69d584c230b94ff8aae722eb2f364c5c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfcb0ceaf-ff", "ovs_interfaceid": "fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:31:18 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Releasing lock "refresh_cache-80e87454-90b9-4168-bf00-b7d93b245526" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:31:18 user nova-compute[72351]: DEBUG nova.compute.manager [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Instance network_info: |[{"id": "fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4", "address": "fa:16:3e:7a:77:69", "network": {"id": "b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-210689817-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "69d584c230b94ff8aae722eb2f364c5c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfcb0ceaf-ff", "ovs_interfaceid": "fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1971}}
Aug 09 02:31:18 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ef166962-6b97-4054-ab6a-30c0d051f1e6 req-105aedd7-88d0-47d1-8c6e-d0faf7870e0c service nova] Acquired lock "refresh_cache-80e87454-90b9-4168-bf00-b7d93b245526" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:31:18 user nova-compute[72351]: DEBUG nova.network.neutron [req-ef166962-6b97-4054-ab6a-30c0d051f1e6 req-105aedd7-88d0-47d1-8c6e-d0faf7870e0c service nova] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Refreshing network info cache for port fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:31:19 user nova-compute[72351]: DEBUG nova.network.neutron [req-ef166962-6b97-4054-ab6a-30c0d051f1e6 req-105aedd7-88d0-47d1-8c6e-d0faf7870e0c service nova] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Updated VIF entry in instance network info cache for port fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:31:19 user nova-compute[72351]: DEBUG nova.network.neutron [req-ef166962-6b97-4054-ab6a-30c0d051f1e6 req-105aedd7-88d0-47d1-8c6e-d0faf7870e0c service nova] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Updating instance_info_cache with network_info: [{"id": "fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4", "address": "fa:16:3e:7a:77:69", "network": {"id": "b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-210689817-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "69d584c230b94ff8aae722eb2f364c5c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfcb0ceaf-ff", "ovs_interfaceid": "fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:31:19 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ef166962-6b97-4054-ab6a-30c0d051f1e6 req-105aedd7-88d0-47d1-8c6e-d0faf7870e0c service nova] Releasing lock "refresh_cache-80e87454-90b9-4168-bf00-b7d93b245526" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:31:20 user nova-compute[72351]: DEBUG nova.compute.manager [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Start spawning the instance on the hypervisor. {{(pid=72351) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2612}}
Aug 09 02:31:20 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Creating instance directory {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4717}}
Aug 09 02:31:20 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Creating image(s)
Aug 09 02:31:20 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Did not create local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4853}}
Aug 09 02:31:20 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Ensure instance console log exists: /opt/stack/data/nova/instances/8e970bdd-54a1-4058-b09d-784cb6b5b1a4/console.log {{(pid=72351) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4603}}
Aug 09 02:31:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:31:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:31:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:31:20 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Start _get_guest_xml network_info=[{"id": "a83afb1f-511f-406c-b925-eb70b32bed90", "address": "fa:16:3e:16:83:e9", "network": {"id": "0b3a34b6-6d32-4414-9135-22bd2a1dae20", "bridge": "br-int", "label": "tempest-TestShelveInstance-2024538322-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4d0cc057c220422a9541bcb9fa7f598b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa83afb1f-51", "ovs_interfaceid": "a83afb1f-511f-406c-b925-eb70b32bed90", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'device_type': 'disk', 'guest_format': None, 'disk_bus': 'virtio', 'mount_device': '/dev/vda', 'boot_index': 0, 'attachment_id': '74c12cc6-0e3f-4efa-91a0-029acb5b9d5a', 'delete_on_termination': True, 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqns': ['iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'EalQALYR', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '8e970bdd-54a1-4058-b09d-784cb6b5b1a4', 'attached_at': '', 'detached_at': '', 'volume_id': 'bc173cc3-1a94-467d-8d0b-d0f50936cee3', 'serial': 'bc173cc3-1a94-467d-8d0b-d0f50936cee3'}, 'volume_type': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:31:20 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:31:20 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:31:20 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:31:20 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:31:20 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:31:20 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:31:20 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:31:20 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:31:20 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:31:20 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:31:20 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:31:20 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:31:20 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:31:20 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:31:20 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:31:20 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:31:20 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:31:20 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:31:20 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 02:31:20 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'EalQALYR', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:31:20 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:31:20 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:31:20 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:31:20 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:31:20 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:31:20 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:31:20 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 02:31:20 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:20 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 -p 172.16.0.220:3260" returned: 21 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:20 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[287dc70f-12fc-46eb-8661-8a30185c9af6]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:20 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Aug 09 02:31:20 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:31:20 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 -p 172.16.0.220:3260 --interface default --op new {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:21 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[484d58b4-a533-4fc1-a83c-9692f3df1d6a]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3] added\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:21 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3] added
Aug 09 02:31:21 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:31:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:21 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[750561f9-7091-4cf1-8822-e0bcc1c8071d]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:21 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:31:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:21 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[5036b194-772b-4866-9eff-14be6604fbaa]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:21 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:31:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v EalQALYR {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v EalQALYR" returned: 0 in 0.030s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:21 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[fc0647eb-ad4e-4fea-8b40-834ba3ba6fef]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:21 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'EalQALYR'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:31:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.026s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:21 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[b67816e8-6140-4d5b-b0a3-20fd634a7938]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:21 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:31:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.028s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:21 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[88cdfda6-9ffc-400c-84c4-df63df5bea3e]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e (non-flash)\ntcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70 (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:21 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:31:21 user nova-compute[72351]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 (non-flash)
Aug 09 02:31:21 user nova-compute[72351]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e (non-flash)
Aug 09 02:31:21 user nova-compute[72351]: tcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70 (non-flash)
Aug 09 02:31:21 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:31:21 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:31:21 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:31:21 user nova-compute[72351]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 (non-flash)
Aug 09 02:31:21 user nova-compute[72351]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e (non-flash)
Aug 09 02:31:21 user nova-compute[72351]: tcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70 (non-flash)
Aug 09 02:31:21 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:31:21 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:31:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 -p 172.16.0.220:3260 --login {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 -p 172.16.0.220:3260 --login" returned: 0 in 0.082s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:21 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[b8bbc257-4880-4efb-8287-5509874fd3fd]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:21 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3, portal: 172.16.0.220,3260]
Aug 09 02:31:21 user nova-compute[72351]: Login to [iface: default, target: iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3, portal: 172.16.0.220,3260] successful.
Aug 09 02:31:21 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:31:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:21 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[b49e7b44-3670-467a-bc52-7de4b2e150fe]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:21 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:31:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.027s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:21 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[0839341e-de5b-4837-953c-b2605b07046c]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e (non-flash)\ntcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70 (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)\ntcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:21 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:31:21 user nova-compute[72351]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 (non-flash)
Aug 09 02:31:21 user nova-compute[72351]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e (non-flash)
Aug 09 02:31:21 user nova-compute[72351]: tcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70 (non-flash)
Aug 09 02:31:21 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:31:21 user nova-compute[72351]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)
Aug 09 02:31:21 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:31:21 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:31:21 user nova-compute[72351]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 (non-flash)
Aug 09 02:31:21 user nova-compute[72351]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e (non-flash)
Aug 09 02:31:21 user nova-compute[72351]: tcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70 (non-flash)
Aug 09 02:31:21 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:31:21 user nova-compute[72351]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)
Aug 09 02:31:21 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:31:21 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.337s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:31:21 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 02:31:21 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] HCTL ('36', '-', '-', 0) found on session 38 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 02:31:21 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Scanning host 36 c: -, t: -, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 02:31:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host36/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host36/scan" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:21 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ec7d7c89-41da-4bb0-9ebb-72c608a238cd]: (4, ('- - 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:21 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Searching for a device in session 38 and hctl ['36', '*', '*', 0] yield: sde {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:31:21 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Connected to sde using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'EalQALYR', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 02:31:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.378s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:31:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] <== connect_volume: return (1380ms) {'type': 'block', 'scsi_wwn': '26661326636383833', 'path': '/dev/sde'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:31:22 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26661326636383833', 'path': '/dev/sde'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 02:31:22 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:31:07Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestShelveInstance-server-805905709',display_name='tempest-TestShelveInstance-server-805905709',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testshelveinstance-server-805905709',id=37,image_ref='',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBD9usWQkTfQS1QEHlx5U391/Ea1m4d+DjLQmZxWBZf5X7UiiUjcuxsL0cAFVTRKCtfIt43KygEG5qAmqMe/mW/kt3gzCiL7+L73rWvmAvMWlhU7Lvh6YyPMPT8wlQXad2w==',key_name='tempest-TestShelveInstance-826835257',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='4d0cc057c220422a9541bcb9fa7f598b',ramdisk_id='',reservation_id='r-ndibkwoj',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestShelveInstance-1998695689',owner_user_name='tempest-TestShelveInstance-1998695689-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:31:10Z,user_data=None,user_id='0aa403f4ead84aecb78a538c2ced64e9',uuid=8e970bdd-54a1-4058-b09d-784cb6b5b1a4,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "a83afb1f-511f-406c-b925-eb70b32bed90", "address": "fa:16:3e:16:83:e9", "network": {"id": "0b3a34b6-6d32-4414-9135-22bd2a1dae20", "bridge": "br-int", "label": "tempest-TestShelveInstance-2024538322-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4d0cc057c220422a9541bcb9fa7f598b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa83afb1f-51", "ovs_interfaceid": "a83afb1f-511f-406c-b925-eb70b32bed90", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:31:22 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Converting VIF {"id": "a83afb1f-511f-406c-b925-eb70b32bed90", "address": "fa:16:3e:16:83:e9", "network": {"id": "0b3a34b6-6d32-4414-9135-22bd2a1dae20", "bridge": "br-int", "label": "tempest-TestShelveInstance-2024538322-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4d0cc057c220422a9541bcb9fa7f598b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa83afb1f-51", "ovs_interfaceid": "a83afb1f-511f-406c-b925-eb70b32bed90", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:31:22 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:16:83:e9,bridge_name='br-int',has_traffic_filtering=True,id=a83afb1f-511f-406c-b925-eb70b32bed90,network=Network(0b3a34b6-6d32-4414-9135-22bd2a1dae20),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa83afb1f-51') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:31:22 user nova-compute[72351]: DEBUG nova.objects.instance [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Lazy-loading 'pci_devices' on Instance uuid 8e970bdd-54a1-4058-b09d-784cb6b5b1a4 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:31:22 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:31:22 user nova-compute[72351]:   <uuid>8e970bdd-54a1-4058-b09d-784cb6b5b1a4</uuid>
Aug 09 02:31:22 user nova-compute[72351]:   <name>instance-00000025</name>
Aug 09 02:31:22 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:31:22 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:31:22 user nova-compute[72351]:   <metadata>
Aug 09 02:31:22 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:31:22 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:31:22 user nova-compute[72351]:       <nova:name>tempest-TestShelveInstance-server-805905709</nova:name>
Aug 09 02:31:22 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:31:20</nova:creationTime>
Aug 09 02:31:22 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:31:22 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:31:22 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:31:22 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:31:22 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:31:22 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:31:22 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:31:22 user nova-compute[72351]:       <nova:owner>
Aug 09 02:31:22 user nova-compute[72351]:         <nova:user uuid="0aa403f4ead84aecb78a538c2ced64e9">tempest-TestShelveInstance-1998695689-project-member</nova:user>
Aug 09 02:31:22 user nova-compute[72351]:         <nova:project uuid="4d0cc057c220422a9541bcb9fa7f598b">tempest-TestShelveInstance-1998695689</nova:project>
Aug 09 02:31:22 user nova-compute[72351]:       </nova:owner>
Aug 09 02:31:22 user nova-compute[72351]:       <nova:ports>
Aug 09 02:31:22 user nova-compute[72351]:         <nova:port uuid="a83afb1f-511f-406c-b925-eb70b32bed90">
Aug 09 02:31:22 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.6" ipVersion="4"/>
Aug 09 02:31:22 user nova-compute[72351]:         </nova:port>
Aug 09 02:31:22 user nova-compute[72351]:       </nova:ports>
Aug 09 02:31:22 user nova-compute[72351]:     </nova:instance>
Aug 09 02:31:22 user nova-compute[72351]:   </metadata>
Aug 09 02:31:22 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:31:22 user nova-compute[72351]:     <system>
Aug 09 02:31:22 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:31:22 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:31:22 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:31:22 user nova-compute[72351]:       <entry name="serial">8e970bdd-54a1-4058-b09d-784cb6b5b1a4</entry>
Aug 09 02:31:22 user nova-compute[72351]:       <entry name="uuid">8e970bdd-54a1-4058-b09d-784cb6b5b1a4</entry>
Aug 09 02:31:22 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:31:22 user nova-compute[72351]:     </system>
Aug 09 02:31:22 user nova-compute[72351]:   </sysinfo>
Aug 09 02:31:22 user nova-compute[72351]:   <os>
Aug 09 02:31:22 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:31:22 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:31:22 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:31:22 user nova-compute[72351]:   </os>
Aug 09 02:31:22 user nova-compute[72351]:   <features>
Aug 09 02:31:22 user nova-compute[72351]:     <acpi/>
Aug 09 02:31:22 user nova-compute[72351]:     <apic/>
Aug 09 02:31:22 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:31:22 user nova-compute[72351]:   </features>
Aug 09 02:31:22 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:31:22 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:31:22 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:31:22 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:31:22 user nova-compute[72351]:   </clock>
Aug 09 02:31:22 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:31:22 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:31:22 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:31:22 user nova-compute[72351]:   </cpu>
Aug 09 02:31:22 user nova-compute[72351]:   <devices>
Aug 09 02:31:22 user nova-compute[72351]:     <disk type="block" device="disk">
Aug 09 02:31:22 user nova-compute[72351]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:31:22 user nova-compute[72351]:       <source dev="/dev/sde"/>
Aug 09 02:31:22 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:31:22 user nova-compute[72351]:       <serial>bc173cc3-1a94-467d-8d0b-d0f50936cee3</serial>
Aug 09 02:31:22 user nova-compute[72351]:     </disk>
Aug 09 02:31:22 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:31:22 user nova-compute[72351]:       <mac address="fa:16:3e:16:83:e9"/>
Aug 09 02:31:22 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:31:22 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:31:22 user nova-compute[72351]:       <target dev="tapa83afb1f-51"/>
Aug 09 02:31:22 user nova-compute[72351]:     </interface>
Aug 09 02:31:22 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:31:22 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/8e970bdd-54a1-4058-b09d-784cb6b5b1a4/console.log" append="off"/>
Aug 09 02:31:22 user nova-compute[72351]:     </serial>
Aug 09 02:31:22 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:31:22 user nova-compute[72351]:     <video>
Aug 09 02:31:22 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:31:22 user nova-compute[72351]:     </video>
Aug 09 02:31:22 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:31:22 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:31:22 user nova-compute[72351]:     </rng>
Aug 09 02:31:22 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:31:22 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:31:22 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:31:22 user nova-compute[72351]:     </memballoon>
Aug 09 02:31:22 user nova-compute[72351]:   </devices>
Aug 09 02:31:22 user nova-compute[72351]: </domain>
Aug 09 02:31:22 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:31:22 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:31:07Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestShelveInstance-server-805905709',display_name='tempest-TestShelveInstance-server-805905709',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testshelveinstance-server-805905709',id=37,image_ref='',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBD9usWQkTfQS1QEHlx5U391/Ea1m4d+DjLQmZxWBZf5X7UiiUjcuxsL0cAFVTRKCtfIt43KygEG5qAmqMe/mW/kt3gzCiL7+L73rWvmAvMWlhU7Lvh6YyPMPT8wlQXad2w==',key_name='tempest-TestShelveInstance-826835257',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='4d0cc057c220422a9541bcb9fa7f598b',ramdisk_id='',reservation_id='r-ndibkwoj',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestShelveInstance-1998695689',owner_user_name='tempest-TestShelveInstance-1998695689-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:31:10Z,user_data=None,user_id='0aa403f4ead84aecb78a538c2ced64e9',uuid=8e970bdd-54a1-4058-b09d-784cb6b5b1a4,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "a83afb1f-511f-406c-b925-eb70b32bed90", "address": "fa:16:3e:16:83:e9", "network": {"id": "0b3a34b6-6d32-4414-9135-22bd2a1dae20", "bridge": "br-int", "label": "tempest-TestShelveInstance-2024538322-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4d0cc057c220422a9541bcb9fa7f598b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa83afb1f-51", "ovs_interfaceid": "a83afb1f-511f-406c-b925-eb70b32bed90", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:31:22 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Converting VIF {"id": "a83afb1f-511f-406c-b925-eb70b32bed90", "address": "fa:16:3e:16:83:e9", "network": {"id": "0b3a34b6-6d32-4414-9135-22bd2a1dae20", "bridge": "br-int", "label": "tempest-TestShelveInstance-2024538322-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4d0cc057c220422a9541bcb9fa7f598b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa83afb1f-51", "ovs_interfaceid": "a83afb1f-511f-406c-b925-eb70b32bed90", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:31:22 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:16:83:e9,bridge_name='br-int',has_traffic_filtering=True,id=a83afb1f-511f-406c-b925-eb70b32bed90,network=Network(0b3a34b6-6d32-4414-9135-22bd2a1dae20),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa83afb1f-51') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:31:22 user nova-compute[72351]: DEBUG os_vif [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:16:83:e9,bridge_name='br-int',has_traffic_filtering=True,id=a83afb1f-511f-406c-b925-eb70b32bed90,network=Network(0b3a34b6-6d32-4414-9135-22bd2a1dae20),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa83afb1f-51') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:31:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:31:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:31:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapa83afb1f-51, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:31:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapa83afb1f-51, col_values=(('external_ids', {'iface-id': 'a83afb1f-511f-406c-b925-eb70b32bed90', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:16:83:e9', 'vm-uuid': '8e970bdd-54a1-4058-b09d-784cb6b5b1a4'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:31:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:31:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:22 user nova-compute[72351]: INFO os_vif [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:16:83:e9,bridge_name='br-int',has_traffic_filtering=True,id=a83afb1f-511f-406c-b925-eb70b32bed90,network=Network(0b3a34b6-6d32-4414-9135-22bd2a1dae20),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa83afb1f-51')
Aug 09 02:31:22 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:31:22 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] No VIF found with MAC fa:16:3e:16:83:e9, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:31:23 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:23 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:23 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:23 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:24 user nova-compute[72351]: DEBUG nova.compute.manager [req-567415bb-5b93-4e69-a880-a365beab48d8 req-3b06203f-c8a9-43ce-91c4-f1f69b094466 service nova] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Received event network-vif-plugged-a83afb1f-511f-406c-b925-eb70b32bed90 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:31:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-567415bb-5b93-4e69-a880-a365beab48d8 req-3b06203f-c8a9-43ce-91c4-f1f69b094466 service nova] Acquiring lock "8e970bdd-54a1-4058-b09d-784cb6b5b1a4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:31:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-567415bb-5b93-4e69-a880-a365beab48d8 req-3b06203f-c8a9-43ce-91c4-f1f69b094466 service nova] Lock "8e970bdd-54a1-4058-b09d-784cb6b5b1a4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:31:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-567415bb-5b93-4e69-a880-a365beab48d8 req-3b06203f-c8a9-43ce-91c4-f1f69b094466 service nova] Lock "8e970bdd-54a1-4058-b09d-784cb6b5b1a4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:31:24 user nova-compute[72351]: DEBUG nova.compute.manager [req-567415bb-5b93-4e69-a880-a365beab48d8 req-3b06203f-c8a9-43ce-91c4-f1f69b094466 service nova] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] No waiting events found dispatching network-vif-plugged-a83afb1f-511f-406c-b925-eb70b32bed90 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:31:24 user nova-compute[72351]: WARNING nova.compute.manager [req-567415bb-5b93-4e69-a880-a365beab48d8 req-3b06203f-c8a9-43ce-91c4-f1f69b094466 service nova] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Received unexpected event network-vif-plugged-a83afb1f-511f-406c-b925-eb70b32bed90 for instance with vm_state building and task_state spawning.
Aug 09 02:31:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:26 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548286.0649261, 8e970bdd-54a1-4058-b09d-784cb6b5b1a4 => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:31:26 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] VM Resumed (Lifecycle Event)
Aug 09 02:31:26 user nova-compute[72351]: DEBUG nova.compute.manager [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:31:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Guest created on hypervisor {{(pid=72351) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4411}}
Aug 09 02:31:26 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Instance spawned successfully.
Aug 09 02:31:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:894}}
Aug 09 02:31:26 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:31:26 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:31:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Found default for hw_cdrom_bus of ide {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:31:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Found default for hw_disk_bus of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:31:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Found default for hw_input_bus of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:31:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Found default for hw_pointer_model of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:31:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Found default for hw_video_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:31:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Found default for hw_vif_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:31:26 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:31:26 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548286.066217, 8e970bdd-54a1-4058-b09d-784cb6b5b1a4 => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:31:26 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] VM Started (Lifecycle Event)
Aug 09 02:31:26 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:31:26 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:31:26 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:31:26 user nova-compute[72351]: INFO nova.compute.manager [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Took 5.27 seconds to spawn the instance on the hypervisor.
Aug 09 02:31:26 user nova-compute[72351]: DEBUG nova.compute.manager [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:31:26 user nova-compute[72351]: INFO nova.compute.manager [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Took 17.49 seconds to build instance.
Aug 09 02:31:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-49638eec-d952-4b6d-95d0-bf1cca872918 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Lock "8e970bdd-54a1-4058-b09d-784cb6b5b1a4" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 17.653s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:31:26 user nova-compute[72351]: DEBUG nova.compute.manager [req-1f70a869-3c1a-4ab4-9606-cf548ff32823 req-0e919a5c-f3cf-4f44-8376-63ee77b4e0a7 service nova] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Received event network-vif-plugged-a83afb1f-511f-406c-b925-eb70b32bed90 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:31:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-1f70a869-3c1a-4ab4-9606-cf548ff32823 req-0e919a5c-f3cf-4f44-8376-63ee77b4e0a7 service nova] Acquiring lock "8e970bdd-54a1-4058-b09d-784cb6b5b1a4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:31:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-1f70a869-3c1a-4ab4-9606-cf548ff32823 req-0e919a5c-f3cf-4f44-8376-63ee77b4e0a7 service nova] Lock "8e970bdd-54a1-4058-b09d-784cb6b5b1a4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:31:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-1f70a869-3c1a-4ab4-9606-cf548ff32823 req-0e919a5c-f3cf-4f44-8376-63ee77b4e0a7 service nova] Lock "8e970bdd-54a1-4058-b09d-784cb6b5b1a4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:31:26 user nova-compute[72351]: DEBUG nova.compute.manager [req-1f70a869-3c1a-4ab4-9606-cf548ff32823 req-0e919a5c-f3cf-4f44-8376-63ee77b4e0a7 service nova] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] No waiting events found dispatching network-vif-plugged-a83afb1f-511f-406c-b925-eb70b32bed90 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:31:26 user nova-compute[72351]: WARNING nova.compute.manager [req-1f70a869-3c1a-4ab4-9606-cf548ff32823 req-0e919a5c-f3cf-4f44-8376-63ee77b4e0a7 service nova] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Received unexpected event network-vif-plugged-a83afb1f-511f-406c-b925-eb70b32bed90 for instance with vm_state active and task_state None.
Aug 09 02:31:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:27 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691548272.7012722, 9df16e46-d097-4553-b44c-732ea24a4169 => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:31:27 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] VM Stopped (Lifecycle Event)
Aug 09 02:31:27 user nova-compute[72351]: DEBUG nova.compute.manager [None req-ee1eded9-5721-4622-8230-1e234c283bb8 None None] [instance: 9df16e46-d097-4553-b44c-732ea24a4169] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:31:28 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:30 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:32 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:33 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:33 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:33 user nova-compute[72351]: DEBUG nova.compute.manager [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Start spawning the instance on the hypervisor. {{(pid=72351) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2612}}
Aug 09 02:31:33 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Creating instance directory {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4717}}
Aug 09 02:31:33 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Creating image(s)
Aug 09 02:31:33 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Did not create local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4853}}
Aug 09 02:31:33 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Ensure instance console log exists: /opt/stack/data/nova/instances/80e87454-90b9-4168-bf00-b7d93b245526/console.log {{(pid=72351) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4603}}
Aug 09 02:31:33 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:31:33 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:31:33 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:31:33 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Start _get_guest_xml network_info=[{"id": "fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4", "address": "fa:16:3e:7a:77:69", "network": {"id": "b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-210689817-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "69d584c230b94ff8aae722eb2f364c5c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfcb0ceaf-ff", "ovs_interfaceid": "fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'device_type': 'disk', 'guest_format': None, 'disk_bus': 'virtio', 'mount_device': '/dev/vda', 'boot_index': 0, 'attachment_id': '481ac50c-3df0-468a-80c8-d0e993871c77', 'delete_on_termination': True, 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqns': ['iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'oxcrdOpE', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '80e87454-90b9-4168-bf00-b7d93b245526', 'attached_at': '', 'detached_at': '', 'volume_id': '7e32fe78-f2c4-409c-813a-36924e611238', 'serial': '7e32fe78-f2c4-409c-813a-36924e611238'}, 'volume_type': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:31:33 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:31:33 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:31:33 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:31:33 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:31:33 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:31:33 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:31:33 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:31:33 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:31:33 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:31:33 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:31:33 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:31:33 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:31:33 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:31:33 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:31:33 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:31:33 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:31:33 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:31:33 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:31:33 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 02:31:33 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'oxcrdOpE', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:31:33 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:31:33 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:31:33 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:31:33 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:31:33 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:31:33 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:31:33 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 02:31:33 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:33 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 -p 172.16.0.220:3260" returned: 21 in 0.025s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:33 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[089e7d7c-aa7a-464e-9824-984203ef72ab]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:33 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Aug 09 02:31:33 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:31:33 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 -p 172.16.0.220:3260 --interface default --op new {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:33 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.033s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:33 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[7e0ee4f7-74fd-4fbe-a1d9-8da1fa215a4e]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238] added\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:33 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238] added
Aug 09 02:31:33 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:31:33 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:33 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.032s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:33 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e590b246-d237-44c8-bf74-5acc01636ab6]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:33 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:31:33 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:34 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.037s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:34 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[10606d75-e528-4a5e-90fa-0e6c9e904865]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:34 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:31:34 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v oxcrdOpE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:34 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v oxcrdOpE" returned: 0 in 0.024s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:34 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[51401ec3-0727-429f-9fc6-7e6789976e3a]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:34 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'oxcrdOpE'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:31:34 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:34 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.026s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:34 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[dde2acc8-1b14-4767-b385-055c0e021162]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:34 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:31:34 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:34 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.040s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:34 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[cad1504c-8e54-4eb5-9a97-c367c43ef37f]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e (non-flash)\ntcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70 (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)\ntcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:34 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:31:34 user nova-compute[72351]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 (non-flash)
Aug 09 02:31:34 user nova-compute[72351]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e (non-flash)
Aug 09 02:31:34 user nova-compute[72351]: tcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70 (non-flash)
Aug 09 02:31:34 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:31:34 user nova-compute[72351]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)
Aug 09 02:31:34 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:31:34 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:31:34 user nova-compute[72351]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 (non-flash)
Aug 09 02:31:34 user nova-compute[72351]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e (non-flash)
Aug 09 02:31:34 user nova-compute[72351]: tcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70 (non-flash)
Aug 09 02:31:34 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:31:34 user nova-compute[72351]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)
Aug 09 02:31:34 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:31:34 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 -p 172.16.0.220:3260 --login {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:34 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 -p 172.16.0.220:3260 --login" returned: 0 in 0.083s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:34 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[8f9279d2-f850-4291-ae38-23573a371df7]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:34 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238, portal: 172.16.0.220,3260]
Aug 09 02:31:34 user nova-compute[72351]: Login to [iface: default, target: iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238, portal: 172.16.0.220,3260] successful.
Aug 09 02:31:34 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:31:34 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:34 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:34 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[0f361406-3f18-4bef-b19c-97fcbb687324]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:34 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:31:34 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:34 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.029s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:34 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[da462e2d-9cd1-439c-b23e-91354ec94120]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e (non-flash)\ntcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70 (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)\ntcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)\ntcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:34 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:31:34 user nova-compute[72351]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 (non-flash)
Aug 09 02:31:34 user nova-compute[72351]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e (non-flash)
Aug 09 02:31:34 user nova-compute[72351]: tcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70 (non-flash)
Aug 09 02:31:34 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:31:34 user nova-compute[72351]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)
Aug 09 02:31:34 user nova-compute[72351]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:31:34 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:31:34 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:31:34 user nova-compute[72351]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 (non-flash)
Aug 09 02:31:34 user nova-compute[72351]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e (non-flash)
Aug 09 02:31:34 user nova-compute[72351]: tcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70 (non-flash)
Aug 09 02:31:34 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:31:34 user nova-compute[72351]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)
Aug 09 02:31:34 user nova-compute[72351]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:31:34 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:31:34 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.409s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:31:34 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 02:31:34 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] HCTL ('39', '-', '-', 0) found on session 39 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 02:31:34 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Scanning host 39 c: -, t: -, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 02:31:34 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host39/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:34 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host39/scan" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:34 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[66ae46b5-b42f-4971-8b1a-eb2aa2143d41]: (4, ('- - 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:34 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Searching for a device in session 39 and hctl ['39', '*', '*', 0] yield: sdh {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:31:34 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Connected to sdh using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'oxcrdOpE', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 02:31:35 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.454s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:31:35 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] <== connect_volume: return (1456ms) {'type': 'block', 'scsi_wwn': '23832643035363835', 'path': '/dev/sdh'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:31:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23832643035363835', 'path': '/dev/sdh'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 02:31:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:31:13Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerActionsTestOtherA-server-2126139398',display_name='tempest-ServerActionsTestOtherA-server-2126139398',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serveractionstestothera-server-2126139398',id=38,image_ref='',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBB4HNXglactRKXsG6uozX3XR7A39DBllYpgpCz+5+1RhfLOeXdmVkNfDkIG7vQrgoVNKWK/MwqtjPDLEVURcnWQfOuv4ySYdfEPj4hs6FpDqfJbAa5qX0FAbZQuEg69FVQ==',key_name='tempest-keypair-976839412',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='69d584c230b94ff8aae722eb2f364c5c',ramdisk_id='',reservation_id='r-97d5c06t',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerActionsTestOtherA-1303549305',owner_user_name='tempest-ServerActionsTestOtherA-1303549305-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:31:16Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='5dca34fdadbb4386a1ff1cb255265d47',uuid=80e87454-90b9-4168-bf00-b7d93b245526,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4", "address": "fa:16:3e:7a:77:69", "network": {"id": "b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-210689817-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "69d584c230b94ff8aae722eb2f364c5c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfcb0ceaf-ff", "ovs_interfaceid": "fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:31:35 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Converting VIF {"id": "fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4", "address": "fa:16:3e:7a:77:69", "network": {"id": "b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-210689817-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "69d584c230b94ff8aae722eb2f364c5c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfcb0ceaf-ff", "ovs_interfaceid": "fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:31:35 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:7a:77:69,bridge_name='br-int',has_traffic_filtering=True,id=fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4,network=Network(b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfcb0ceaf-ff') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:31:35 user nova-compute[72351]: DEBUG nova.objects.instance [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lazy-loading 'pci_devices' on Instance uuid 80e87454-90b9-4168-bf00-b7d93b245526 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:31:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:31:35 user nova-compute[72351]:   <uuid>80e87454-90b9-4168-bf00-b7d93b245526</uuid>
Aug 09 02:31:35 user nova-compute[72351]:   <name>instance-00000026</name>
Aug 09 02:31:35 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:31:35 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:31:35 user nova-compute[72351]:   <metadata>
Aug 09 02:31:35 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:31:35 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:31:35 user nova-compute[72351]:       <nova:name>tempest-ServerActionsTestOtherA-server-2126139398</nova:name>
Aug 09 02:31:35 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:31:33</nova:creationTime>
Aug 09 02:31:35 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:31:35 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:31:35 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:31:35 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:31:35 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:31:35 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:31:35 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:31:35 user nova-compute[72351]:       <nova:owner>
Aug 09 02:31:35 user nova-compute[72351]:         <nova:user uuid="5dca34fdadbb4386a1ff1cb255265d47">tempest-ServerActionsTestOtherA-1303549305-project-member</nova:user>
Aug 09 02:31:35 user nova-compute[72351]:         <nova:project uuid="69d584c230b94ff8aae722eb2f364c5c">tempest-ServerActionsTestOtherA-1303549305</nova:project>
Aug 09 02:31:35 user nova-compute[72351]:       </nova:owner>
Aug 09 02:31:35 user nova-compute[72351]:       <nova:ports>
Aug 09 02:31:35 user nova-compute[72351]:         <nova:port uuid="fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4">
Aug 09 02:31:35 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.9" ipVersion="4"/>
Aug 09 02:31:35 user nova-compute[72351]:         </nova:port>
Aug 09 02:31:35 user nova-compute[72351]:       </nova:ports>
Aug 09 02:31:35 user nova-compute[72351]:     </nova:instance>
Aug 09 02:31:35 user nova-compute[72351]:   </metadata>
Aug 09 02:31:35 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:31:35 user nova-compute[72351]:     <system>
Aug 09 02:31:35 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:31:35 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:31:35 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:31:35 user nova-compute[72351]:       <entry name="serial">80e87454-90b9-4168-bf00-b7d93b245526</entry>
Aug 09 02:31:35 user nova-compute[72351]:       <entry name="uuid">80e87454-90b9-4168-bf00-b7d93b245526</entry>
Aug 09 02:31:35 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:31:35 user nova-compute[72351]:     </system>
Aug 09 02:31:35 user nova-compute[72351]:   </sysinfo>
Aug 09 02:31:35 user nova-compute[72351]:   <os>
Aug 09 02:31:35 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:31:35 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:31:35 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:31:35 user nova-compute[72351]:   </os>
Aug 09 02:31:35 user nova-compute[72351]:   <features>
Aug 09 02:31:35 user nova-compute[72351]:     <acpi/>
Aug 09 02:31:35 user nova-compute[72351]:     <apic/>
Aug 09 02:31:35 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:31:35 user nova-compute[72351]:   </features>
Aug 09 02:31:35 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:31:35 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:31:35 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:31:35 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:31:35 user nova-compute[72351]:   </clock>
Aug 09 02:31:35 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:31:35 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:31:35 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:31:35 user nova-compute[72351]:   </cpu>
Aug 09 02:31:35 user nova-compute[72351]:   <devices>
Aug 09 02:31:35 user nova-compute[72351]:     <disk type="block" device="disk">
Aug 09 02:31:35 user nova-compute[72351]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:31:35 user nova-compute[72351]:       <source dev="/dev/sdh"/>
Aug 09 02:31:35 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:31:35 user nova-compute[72351]:       <serial>7e32fe78-f2c4-409c-813a-36924e611238</serial>
Aug 09 02:31:35 user nova-compute[72351]:     </disk>
Aug 09 02:31:35 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:31:35 user nova-compute[72351]:       <mac address="fa:16:3e:7a:77:69"/>
Aug 09 02:31:35 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:31:35 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:31:35 user nova-compute[72351]:       <target dev="tapfcb0ceaf-ff"/>
Aug 09 02:31:35 user nova-compute[72351]:     </interface>
Aug 09 02:31:35 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:31:35 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/80e87454-90b9-4168-bf00-b7d93b245526/console.log" append="off"/>
Aug 09 02:31:35 user nova-compute[72351]:     </serial>
Aug 09 02:31:35 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:31:35 user nova-compute[72351]:     <video>
Aug 09 02:31:35 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:31:35 user nova-compute[72351]:     </video>
Aug 09 02:31:35 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:31:35 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:31:35 user nova-compute[72351]:     </rng>
Aug 09 02:31:35 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:31:35 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:31:35 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:31:35 user nova-compute[72351]:     </memballoon>
Aug 09 02:31:35 user nova-compute[72351]:   </devices>
Aug 09 02:31:35 user nova-compute[72351]: </domain>
Aug 09 02:31:35 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:31:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:31:13Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerActionsTestOtherA-server-2126139398',display_name='tempest-ServerActionsTestOtherA-server-2126139398',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serveractionstestothera-server-2126139398',id=38,image_ref='',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBB4HNXglactRKXsG6uozX3XR7A39DBllYpgpCz+5+1RhfLOeXdmVkNfDkIG7vQrgoVNKWK/MwqtjPDLEVURcnWQfOuv4ySYdfEPj4hs6FpDqfJbAa5qX0FAbZQuEg69FVQ==',key_name='tempest-keypair-976839412',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='69d584c230b94ff8aae722eb2f364c5c',ramdisk_id='',reservation_id='r-97d5c06t',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerActionsTestOtherA-1303549305',owner_user_name='tempest-ServerActionsTestOtherA-1303549305-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:31:16Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='5dca34fdadbb4386a1ff1cb255265d47',uuid=80e87454-90b9-4168-bf00-b7d93b245526,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4", "address": "fa:16:3e:7a:77:69", "network": {"id": "b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-210689817-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "69d584c230b94ff8aae722eb2f364c5c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfcb0ceaf-ff", "ovs_interfaceid": "fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:31:35 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Converting VIF {"id": "fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4", "address": "fa:16:3e:7a:77:69", "network": {"id": "b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-210689817-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "69d584c230b94ff8aae722eb2f364c5c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfcb0ceaf-ff", "ovs_interfaceid": "fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:31:35 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:7a:77:69,bridge_name='br-int',has_traffic_filtering=True,id=fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4,network=Network(b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfcb0ceaf-ff') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:31:35 user nova-compute[72351]: DEBUG os_vif [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:7a:77:69,bridge_name='br-int',has_traffic_filtering=True,id=fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4,network=Network(b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfcb0ceaf-ff') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:31:35 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:35 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:31:35 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:31:35 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:35 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapfcb0ceaf-ff, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:31:35 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapfcb0ceaf-ff, col_values=(('external_ids', {'iface-id': 'fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:7a:77:69', 'vm-uuid': '80e87454-90b9-4168-bf00-b7d93b245526'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:31:35 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:35 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:31:35 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:35 user nova-compute[72351]: INFO os_vif [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:7a:77:69,bridge_name='br-int',has_traffic_filtering=True,id=fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4,network=Network(b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfcb0ceaf-ff')
Aug 09 02:31:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:31:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] No VIF found with MAC fa:16:3e:7a:77:69, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:31:36 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:36 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:36 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:37 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:37 user nova-compute[72351]: DEBUG nova.compute.manager [req-1ce90a69-9827-4752-8a9b-6924d333a17a req-5818f2b6-0f6d-4a99-9287-dd102db662e0 service nova] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Received event network-vif-plugged-fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:31:37 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-1ce90a69-9827-4752-8a9b-6924d333a17a req-5818f2b6-0f6d-4a99-9287-dd102db662e0 service nova] Acquiring lock "80e87454-90b9-4168-bf00-b7d93b245526-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:31:37 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-1ce90a69-9827-4752-8a9b-6924d333a17a req-5818f2b6-0f6d-4a99-9287-dd102db662e0 service nova] Lock "80e87454-90b9-4168-bf00-b7d93b245526-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:31:37 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-1ce90a69-9827-4752-8a9b-6924d333a17a req-5818f2b6-0f6d-4a99-9287-dd102db662e0 service nova] Lock "80e87454-90b9-4168-bf00-b7d93b245526-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:31:37 user nova-compute[72351]: DEBUG nova.compute.manager [req-1ce90a69-9827-4752-8a9b-6924d333a17a req-5818f2b6-0f6d-4a99-9287-dd102db662e0 service nova] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] No waiting events found dispatching network-vif-plugged-fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:31:37 user nova-compute[72351]: WARNING nova.compute.manager [req-1ce90a69-9827-4752-8a9b-6924d333a17a req-5818f2b6-0f6d-4a99-9287-dd102db662e0 service nova] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Received unexpected event network-vif-plugged-fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4 for instance with vm_state building and task_state spawning.
Aug 09 02:31:38 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:39 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Acquiring lock "7bef74f4-3bb3-4ad0-bff7-34156be31249" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:31:39 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "7bef74f4-3bb3-4ad0-bff7-34156be31249" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:31:39 user nova-compute[72351]: DEBUG nova.compute.manager [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Starting instance... {{(pid=72351) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2406}}
Aug 09 02:31:39 user nova-compute[72351]: DEBUG nova.compute.manager [req-daad9099-aaa5-48d4-932e-6c4e3346857e req-b027b383-bf20-4ab7-be8b-4166c73dd220 service nova] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Received event network-vif-plugged-fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:31:39 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-daad9099-aaa5-48d4-932e-6c4e3346857e req-b027b383-bf20-4ab7-be8b-4166c73dd220 service nova] Acquiring lock "80e87454-90b9-4168-bf00-b7d93b245526-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:31:39 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-daad9099-aaa5-48d4-932e-6c4e3346857e req-b027b383-bf20-4ab7-be8b-4166c73dd220 service nova] Lock "80e87454-90b9-4168-bf00-b7d93b245526-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:31:39 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-daad9099-aaa5-48d4-932e-6c4e3346857e req-b027b383-bf20-4ab7-be8b-4166c73dd220 service nova] Lock "80e87454-90b9-4168-bf00-b7d93b245526-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:31:39 user nova-compute[72351]: DEBUG nova.compute.manager [req-daad9099-aaa5-48d4-932e-6c4e3346857e req-b027b383-bf20-4ab7-be8b-4166c73dd220 service nova] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] No waiting events found dispatching network-vif-plugged-fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:31:39 user nova-compute[72351]: WARNING nova.compute.manager [req-daad9099-aaa5-48d4-932e-6c4e3346857e req-b027b383-bf20-4ab7-be8b-4166c73dd220 service nova] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Received unexpected event network-vif-plugged-fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4 for instance with vm_state building and task_state spawning.
Aug 09 02:31:39 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:31:39 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:31:39 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72351) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Aug 09 02:31:39 user nova-compute[72351]: INFO nova.compute.claims [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Claim successful on node user
Aug 09 02:31:39 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548299.273246, 80e87454-90b9-4168-bf00-b7d93b245526 => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:31:39 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] VM Resumed (Lifecycle Event)
Aug 09 02:31:39 user nova-compute[72351]: DEBUG nova.compute.manager [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:31:39 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Guest created on hypervisor {{(pid=72351) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4411}}
Aug 09 02:31:39 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:31:39 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Instance spawned successfully.
Aug 09 02:31:39 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:894}}
Aug 09 02:31:39 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:31:39 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:31:39 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548299.2744026, 80e87454-90b9-4168-bf00-b7d93b245526 => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:31:39 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] VM Started (Lifecycle Event)
Aug 09 02:31:39 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Found default for hw_cdrom_bus of ide {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:31:39 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Found default for hw_disk_bus of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:31:39 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Found default for hw_input_bus of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:31:39 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Found default for hw_pointer_model of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:31:39 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Found default for hw_video_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:31:39 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Found default for hw_vif_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:31:39 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:31:39 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:31:39 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:31:39 user nova-compute[72351]: INFO nova.compute.manager [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Took 5.59 seconds to spawn the instance on the hypervisor.
Aug 09 02:31:39 user nova-compute[72351]: DEBUG nova.compute.manager [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:31:39 user nova-compute[72351]: INFO nova.compute.manager [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Took 25.31 seconds to build instance.
Aug 09 02:31:39 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b6adb164-20a7-4d8b-9cb0-a12bd18a925e tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "80e87454-90b9-4168-bf00-b7d93b245526" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 25.508s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:31:39 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:31:39 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:31:39 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.678s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:31:39 user nova-compute[72351]: DEBUG nova.compute.manager [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Start building networks asynchronously for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2803}}
Aug 09 02:31:39 user nova-compute[72351]: DEBUG nova.compute.manager [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Allocating IP information in the background. {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1956}}
Aug 09 02:31:39 user nova-compute[72351]: DEBUG nova.network.neutron [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] allocate_for_instance() {{(pid=72351) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Aug 09 02:31:39 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Aug 09 02:31:39 user nova-compute[72351]: DEBUG nova.compute.manager [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Start building block device mappings for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2838}}
Aug 09 02:31:40 user nova-compute[72351]: DEBUG nova.policy [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '1d460843653049dda604faa051159936', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0dd686dc07ca42f39aa898351ca281c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72351) authorize /opt/stack/nova/nova/policy.py:203}}
Aug 09 02:31:40 user nova-compute[72351]: DEBUG nova.compute.manager [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Start spawning the instance on the hypervisor. {{(pid=72351) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2612}}
Aug 09 02:31:40 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Creating instance directory {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4717}}
Aug 09 02:31:40 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Creating image(s)
Aug 09 02:31:40 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Acquiring lock "/opt/stack/data/nova/instances/7bef74f4-3bb3-4ad0-bff7-34156be31249/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:31:40 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "/opt/stack/data/nova/instances/7bef74f4-3bb3-4ad0-bff7-34156be31249/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:31:40 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "/opt/stack/data/nova/instances/7bef74f4-3bb3-4ad0-bff7-34156be31249/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.006s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:31:40 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:40 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.170s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:40 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Acquiring lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:31:40 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:31:40 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:40 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:40 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.163s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:40 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/7bef74f4-3bb3-4ad0-bff7-34156be31249/disk 1073741824 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:40 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/7bef74f4-3bb3-4ad0-bff7-34156be31249/disk 1073741824" returned: 0 in 0.051s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:40 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.221s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:31:40 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:40 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.179s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:40 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Checking if we can resize image /opt/stack/data/nova/instances/7bef74f4-3bb3-4ad0-bff7-34156be31249/disk. size=1073741824 {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Aug 09 02:31:40 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7bef74f4-3bb3-4ad0-bff7-34156be31249/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:40 user nova-compute[72351]: DEBUG nova.network.neutron [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Successfully created port: d8727f3f-d838-442c-a9a8-278f38aa87d4 {{(pid=72351) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Aug 09 02:31:40 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7bef74f4-3bb3-4ad0-bff7-34156be31249/disk --force-share --output=json" returned: 0 in 0.182s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:40 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Cannot resize image /opt/stack/data/nova/instances/7bef74f4-3bb3-4ad0-bff7-34156be31249/disk to a smaller size. {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Aug 09 02:31:40 user nova-compute[72351]: DEBUG nova.objects.instance [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lazy-loading 'migration_context' on Instance uuid 7bef74f4-3bb3-4ad0-bff7-34156be31249 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:31:40 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Created local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4851}}
Aug 09 02:31:40 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Ensure instance console log exists: /opt/stack/data/nova/instances/7bef74f4-3bb3-4ad0-bff7-34156be31249/console.log {{(pid=72351) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4603}}
Aug 09 02:31:40 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:31:40 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:31:40 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:31:41 user nova-compute[72351]: DEBUG nova.compute.manager [req-6aefcc45-ab3e-4d07-83df-bc16f2f8467f req-dc077078-8c94-4fa5-9ea3-bf8f680c1e86 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received event network-changed-0ebd83d0-5519-4945-8637-90c2ce024ac1 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:31:41 user nova-compute[72351]: DEBUG nova.compute.manager [req-6aefcc45-ab3e-4d07-83df-bc16f2f8467f req-dc077078-8c94-4fa5-9ea3-bf8f680c1e86 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Refreshing instance network info cache due to event network-changed-0ebd83d0-5519-4945-8637-90c2ce024ac1. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:31:41 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-6aefcc45-ab3e-4d07-83df-bc16f2f8467f req-dc077078-8c94-4fa5-9ea3-bf8f680c1e86 service nova] Acquiring lock "refresh_cache-38a2b8e3-337d-416a-8a8c-da4921123db6" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:31:41 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-6aefcc45-ab3e-4d07-83df-bc16f2f8467f req-dc077078-8c94-4fa5-9ea3-bf8f680c1e86 service nova] Acquired lock "refresh_cache-38a2b8e3-337d-416a-8a8c-da4921123db6" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:31:41 user nova-compute[72351]: DEBUG nova.network.neutron [req-6aefcc45-ab3e-4d07-83df-bc16f2f8467f req-dc077078-8c94-4fa5-9ea3-bf8f680c1e86 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Refreshing network info cache for port 0ebd83d0-5519-4945-8637-90c2ce024ac1 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:31:41 user nova-compute[72351]: DEBUG nova.network.neutron [req-6aefcc45-ab3e-4d07-83df-bc16f2f8467f req-dc077078-8c94-4fa5-9ea3-bf8f680c1e86 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Updated VIF entry in instance network info cache for port 0ebd83d0-5519-4945-8637-90c2ce024ac1. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:31:41 user nova-compute[72351]: DEBUG nova.network.neutron [req-6aefcc45-ab3e-4d07-83df-bc16f2f8467f req-dc077078-8c94-4fa5-9ea3-bf8f680c1e86 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Updating instance_info_cache with network_info: [{"id": "0ebd83d0-5519-4945-8637-90c2ce024ac1", "address": "fa:16:3e:81:a2:62", "network": {"id": "c45d6719-95f9-4856-9b34-9d89c22ff899", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-927662772-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.149", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0ebd83d0-55", "ovs_interfaceid": "0ebd83d0-5519-4945-8637-90c2ce024ac1", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "8b1245ae-85e9-49c3-9631-a288a172e4c1", "address": "fa:16:3e:a8:7b:e4", "network": {"id": "4617a3d0-dcb5-42a4-85d3-358087aeac69", "bridge": "br-int", "label": "tempest-device-tagging-net1-1844430962", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.34", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8b1245ae-85", "ovs_interfaceid": "8b1245ae-85e9-49c3-9631-a288a172e4c1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "1be27a59-07a4-4b6a-b25a-ce6d7efac4a3", "address": "fa:16:3e:38:62:75", "network": {"id": "4617a3d0-dcb5-42a4-85d3-358087aeac69", "bridge": "br-int", "label": "tempest-device-tagging-net1-1844430962", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.226", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1be27a59-07", "ovs_interfaceid": "1be27a59-07a4-4b6a-b25a-ce6d7efac4a3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "ccb930f6-9625-450b-b01b-c5009e610ae1", "address": "fa:16:3e:a9:0c:63", "network": {"id": "4617a3d0-dcb5-42a4-85d3-358087aeac69", "bridge": "br-int", "label": "tempest-device-tagging-net1-1844430962", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.237", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapccb930f6-96", "ovs_interfaceid": "ccb930f6-9625-450b-b01b-c5009e610ae1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "d083d9f2-dfe4-4a73-a624-3336f2f02b0d", "address": "fa:16:3e:a0:b1:db", "network": {"id": "4617a3d0-dcb5-42a4-85d3-358087aeac69", "bridge": "br-int", "label": "tempest-device-tagging-net1-1844430962", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.185", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd083d9f2-df", "ovs_interfaceid": "d083d9f2-dfe4-4a73-a624-3336f2f02b0d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "a88b872b-9340-4597-8ab4-ea6173dc4e4c", "address": "fa:16:3e:cf:3d:24", "network": {"id": "2984e5a6-dc0d-40ee-8064-f78a7686663e", "bridge": "br-int", "label": "tempest-device-tagging-net2-1630262622", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa88b872b-93", "ovs_interfaceid": "a88b872b-9340-4597-8ab4-ea6173dc4e4c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "48dfbf61-0142-4478-9ec1-585d5fbe11da", "address": "fa:16:3e:5b:d8:08", "network": {"id": "2984e5a6-dc0d-40ee-8064-f78a7686663e", "bridge": "br-int", "label": "tempest-device-tagging-net2-1630262622", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap48dfbf61-01", "ovs_interfaceid": "48dfbf61-0142-4478-9ec1-585d5fbe11da", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:31:41 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-6aefcc45-ab3e-4d07-83df-bc16f2f8467f req-dc077078-8c94-4fa5-9ea3-bf8f680c1e86 service nova] Releasing lock "refresh_cache-38a2b8e3-337d-416a-8a8c-da4921123db6" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:31:41 user nova-compute[72351]: DEBUG nova.network.neutron [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Successfully updated port: d8727f3f-d838-442c-a9a8-278f38aa87d4 {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:31:41 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Acquiring lock "refresh_cache-7bef74f4-3bb3-4ad0-bff7-34156be31249" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:31:41 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Acquired lock "refresh_cache-7bef74f4-3bb3-4ad0-bff7-34156be31249" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:31:41 user nova-compute[72351]: DEBUG nova.network.neutron [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:31:41 user nova-compute[72351]: DEBUG nova.compute.manager [req-91f1989e-94b6-4779-9ae9-ed13fcf1e7ec req-4893c95b-d735-4f81-a89a-b78ed4d8b50e service nova] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Received event network-changed-d8727f3f-d838-442c-a9a8-278f38aa87d4 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:31:41 user nova-compute[72351]: DEBUG nova.compute.manager [req-91f1989e-94b6-4779-9ae9-ed13fcf1e7ec req-4893c95b-d735-4f81-a89a-b78ed4d8b50e service nova] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Refreshing instance network info cache due to event network-changed-d8727f3f-d838-442c-a9a8-278f38aa87d4. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:31:41 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-91f1989e-94b6-4779-9ae9-ed13fcf1e7ec req-4893c95b-d735-4f81-a89a-b78ed4d8b50e service nova] Acquiring lock "refresh_cache-7bef74f4-3bb3-4ad0-bff7-34156be31249" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:31:42 user nova-compute[72351]: DEBUG nova.network.neutron [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:31:42 user nova-compute[72351]: DEBUG nova.network.neutron [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Updating instance_info_cache with network_info: [{"id": "d8727f3f-d838-442c-a9a8-278f38aa87d4", "address": "fa:16:3e:b4:8f:2e", "network": {"id": "626455de-9781-4ee8-b0be-c51ac124b189", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-1560904912-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0dd686dc07ca42f39aa898351ca281c2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd8727f3f-d8", "ovs_interfaceid": "d8727f3f-d838-442c-a9a8-278f38aa87d4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:31:42 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Releasing lock "refresh_cache-7bef74f4-3bb3-4ad0-bff7-34156be31249" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:31:42 user nova-compute[72351]: DEBUG nova.compute.manager [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Instance network_info: |[{"id": "d8727f3f-d838-442c-a9a8-278f38aa87d4", "address": "fa:16:3e:b4:8f:2e", "network": {"id": "626455de-9781-4ee8-b0be-c51ac124b189", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-1560904912-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0dd686dc07ca42f39aa898351ca281c2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd8727f3f-d8", "ovs_interfaceid": "d8727f3f-d838-442c-a9a8-278f38aa87d4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1971}}
Aug 09 02:31:42 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-91f1989e-94b6-4779-9ae9-ed13fcf1e7ec req-4893c95b-d735-4f81-a89a-b78ed4d8b50e service nova] Acquired lock "refresh_cache-7bef74f4-3bb3-4ad0-bff7-34156be31249" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:31:42 user nova-compute[72351]: DEBUG nova.network.neutron [req-91f1989e-94b6-4779-9ae9-ed13fcf1e7ec req-4893c95b-d735-4f81-a89a-b78ed4d8b50e service nova] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Refreshing network info cache for port d8727f3f-d838-442c-a9a8-278f38aa87d4 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:31:42 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Start _get_guest_xml network_info=[{"id": "d8727f3f-d838-442c-a9a8-278f38aa87d4", "address": "fa:16:3e:b4:8f:2e", "network": {"id": "626455de-9781-4ee8-b0be-c51ac124b189", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-1560904912-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0dd686dc07ca42f39aa898351ca281c2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd8727f3f-d8", "ovs_interfaceid": "d8727f3f-d838-442c-a9a8-278f38aa87d4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'size': 0, 'device_type': 'disk', 'encryption_secret_uuid': None, 'guest_format': None, 'encryption_options': None, 'disk_bus': 'virtio', 'encrypted': False, 'encryption_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'image_id': '18ccda7f-31d3-4978-ab9d-4ed2f74eff94'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:31:42 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:31:42 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:31:42 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:31:42 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:31:42 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:31:42 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:31:42 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:31:42 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:31:42 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:31:42 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:31:42 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:31:42 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:31:42 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:31:42 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:31:42 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:31:42 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:31:42 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:31:42 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:31:42 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:31:38Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-SnapshotDataIntegrityTests-server-1527375297',display_name='tempest-SnapshotDataIntegrityTests-server-1527375297',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-snapshotdataintegritytests-server-1527375297',id=39,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKb0wYqi2SGirp8rEcWOo1bwnUn0HV+KTTBhhMO9lQaJ37WHi7lGFbj8YPdn6+1lmH+ZUmXq/caH8VRWPRYOy7Ei7LeEYXXQj0+wJQPa35fNJDjJMxBmuiONaPaoN4KW9w==',key_name='tempest-keypair-1816826569',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='0dd686dc07ca42f39aa898351ca281c2',ramdisk_id='',reservation_id='r-j1j10mu7',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-SnapshotDataIntegrityTests-1458031403',owner_user_name='tempest-SnapshotDataIntegrityTests-1458031403-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:31:40Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='1d460843653049dda604faa051159936',uuid=7bef74f4-3bb3-4ad0-bff7-34156be31249,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "d8727f3f-d838-442c-a9a8-278f38aa87d4", "address": "fa:16:3e:b4:8f:2e", "network": {"id": "626455de-9781-4ee8-b0be-c51ac124b189", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-1560904912-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0dd686dc07ca42f39aa898351ca281c2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd8727f3f-d8", "ovs_interfaceid": "d8727f3f-d838-442c-a9a8-278f38aa87d4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:31:42 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Converting VIF {"id": "d8727f3f-d838-442c-a9a8-278f38aa87d4", "address": "fa:16:3e:b4:8f:2e", "network": {"id": "626455de-9781-4ee8-b0be-c51ac124b189", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-1560904912-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0dd686dc07ca42f39aa898351ca281c2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd8727f3f-d8", "ovs_interfaceid": "d8727f3f-d838-442c-a9a8-278f38aa87d4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:31:42 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:b4:8f:2e,bridge_name='br-int',has_traffic_filtering=True,id=d8727f3f-d838-442c-a9a8-278f38aa87d4,network=Network(626455de-9781-4ee8-b0be-c51ac124b189),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd8727f3f-d8') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:31:42 user nova-compute[72351]: DEBUG nova.objects.instance [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lazy-loading 'pci_devices' on Instance uuid 7bef74f4-3bb3-4ad0-bff7-34156be31249 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:31:42 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:31:42 user nova-compute[72351]:   <uuid>7bef74f4-3bb3-4ad0-bff7-34156be31249</uuid>
Aug 09 02:31:42 user nova-compute[72351]:   <name>instance-00000027</name>
Aug 09 02:31:42 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:31:42 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:31:42 user nova-compute[72351]:   <metadata>
Aug 09 02:31:42 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:31:42 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:31:42 user nova-compute[72351]:       <nova:name>tempest-SnapshotDataIntegrityTests-server-1527375297</nova:name>
Aug 09 02:31:42 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:31:42</nova:creationTime>
Aug 09 02:31:42 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:31:42 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:31:42 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:31:42 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:31:42 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:31:42 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:31:42 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:31:42 user nova-compute[72351]:       <nova:owner>
Aug 09 02:31:42 user nova-compute[72351]:         <nova:user uuid="1d460843653049dda604faa051159936">tempest-SnapshotDataIntegrityTests-1458031403-project-member</nova:user>
Aug 09 02:31:42 user nova-compute[72351]:         <nova:project uuid="0dd686dc07ca42f39aa898351ca281c2">tempest-SnapshotDataIntegrityTests-1458031403</nova:project>
Aug 09 02:31:42 user nova-compute[72351]:       </nova:owner>
Aug 09 02:31:42 user nova-compute[72351]:       <nova:root type="image" uuid="18ccda7f-31d3-4978-ab9d-4ed2f74eff94"/>
Aug 09 02:31:42 user nova-compute[72351]:       <nova:ports>
Aug 09 02:31:42 user nova-compute[72351]:         <nova:port uuid="d8727f3f-d838-442c-a9a8-278f38aa87d4">
Aug 09 02:31:42 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.11" ipVersion="4"/>
Aug 09 02:31:42 user nova-compute[72351]:         </nova:port>
Aug 09 02:31:42 user nova-compute[72351]:       </nova:ports>
Aug 09 02:31:42 user nova-compute[72351]:     </nova:instance>
Aug 09 02:31:42 user nova-compute[72351]:   </metadata>
Aug 09 02:31:42 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:31:42 user nova-compute[72351]:     <system>
Aug 09 02:31:42 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:31:42 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:31:42 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:31:42 user nova-compute[72351]:       <entry name="serial">7bef74f4-3bb3-4ad0-bff7-34156be31249</entry>
Aug 09 02:31:42 user nova-compute[72351]:       <entry name="uuid">7bef74f4-3bb3-4ad0-bff7-34156be31249</entry>
Aug 09 02:31:42 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:31:42 user nova-compute[72351]:     </system>
Aug 09 02:31:42 user nova-compute[72351]:   </sysinfo>
Aug 09 02:31:42 user nova-compute[72351]:   <os>
Aug 09 02:31:42 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:31:42 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:31:42 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:31:42 user nova-compute[72351]:   </os>
Aug 09 02:31:42 user nova-compute[72351]:   <features>
Aug 09 02:31:42 user nova-compute[72351]:     <acpi/>
Aug 09 02:31:42 user nova-compute[72351]:     <apic/>
Aug 09 02:31:42 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:31:42 user nova-compute[72351]:   </features>
Aug 09 02:31:42 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:31:42 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:31:42 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:31:42 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:31:42 user nova-compute[72351]:   </clock>
Aug 09 02:31:42 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:31:42 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:31:42 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:31:42 user nova-compute[72351]:   </cpu>
Aug 09 02:31:42 user nova-compute[72351]:   <devices>
Aug 09 02:31:42 user nova-compute[72351]:     <disk type="file" device="disk">
Aug 09 02:31:42 user nova-compute[72351]:       <driver name="qemu" type="qcow2" cache="none"/>
Aug 09 02:31:42 user nova-compute[72351]:       <source file="/opt/stack/data/nova/instances/7bef74f4-3bb3-4ad0-bff7-34156be31249/disk"/>
Aug 09 02:31:42 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:31:42 user nova-compute[72351]:     </disk>
Aug 09 02:31:42 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:31:42 user nova-compute[72351]:       <mac address="fa:16:3e:b4:8f:2e"/>
Aug 09 02:31:42 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:31:42 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:31:42 user nova-compute[72351]:       <target dev="tapd8727f3f-d8"/>
Aug 09 02:31:42 user nova-compute[72351]:     </interface>
Aug 09 02:31:42 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:31:42 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/7bef74f4-3bb3-4ad0-bff7-34156be31249/console.log" append="off"/>
Aug 09 02:31:42 user nova-compute[72351]:     </serial>
Aug 09 02:31:42 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:31:42 user nova-compute[72351]:     <video>
Aug 09 02:31:42 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:31:42 user nova-compute[72351]:     </video>
Aug 09 02:31:42 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:31:42 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:31:42 user nova-compute[72351]:     </rng>
Aug 09 02:31:42 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:31:42 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:31:42 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:31:42 user nova-compute[72351]:     </memballoon>
Aug 09 02:31:42 user nova-compute[72351]:   </devices>
Aug 09 02:31:42 user nova-compute[72351]: </domain>
Aug 09 02:31:42 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:31:42 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:31:38Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-SnapshotDataIntegrityTests-server-1527375297',display_name='tempest-SnapshotDataIntegrityTests-server-1527375297',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-snapshotdataintegritytests-server-1527375297',id=39,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKb0wYqi2SGirp8rEcWOo1bwnUn0HV+KTTBhhMO9lQaJ37WHi7lGFbj8YPdn6+1lmH+ZUmXq/caH8VRWPRYOy7Ei7LeEYXXQj0+wJQPa35fNJDjJMxBmuiONaPaoN4KW9w==',key_name='tempest-keypair-1816826569',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='0dd686dc07ca42f39aa898351ca281c2',ramdisk_id='',reservation_id='r-j1j10mu7',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-SnapshotDataIntegrityTests-1458031403',owner_user_name='tempest-SnapshotDataIntegrityTests-1458031403-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:31:40Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='1d460843653049dda604faa051159936',uuid=7bef74f4-3bb3-4ad0-bff7-34156be31249,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "d8727f3f-d838-442c-a9a8-278f38aa87d4", "address": "fa:16:3e:b4:8f:2e", "network": {"id": "626455de-9781-4ee8-b0be-c51ac124b189", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-1560904912-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0dd686dc07ca42f39aa898351ca281c2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd8727f3f-d8", "ovs_interfaceid": "d8727f3f-d838-442c-a9a8-278f38aa87d4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:31:42 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Converting VIF {"id": "d8727f3f-d838-442c-a9a8-278f38aa87d4", "address": "fa:16:3e:b4:8f:2e", "network": {"id": "626455de-9781-4ee8-b0be-c51ac124b189", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-1560904912-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0dd686dc07ca42f39aa898351ca281c2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd8727f3f-d8", "ovs_interfaceid": "d8727f3f-d838-442c-a9a8-278f38aa87d4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:31:42 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:b4:8f:2e,bridge_name='br-int',has_traffic_filtering=True,id=d8727f3f-d838-442c-a9a8-278f38aa87d4,network=Network(626455de-9781-4ee8-b0be-c51ac124b189),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd8727f3f-d8') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:31:42 user nova-compute[72351]: DEBUG os_vif [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:b4:8f:2e,bridge_name='br-int',has_traffic_filtering=True,id=d8727f3f-d838-442c-a9a8-278f38aa87d4,network=Network(626455de-9781-4ee8-b0be-c51ac124b189),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd8727f3f-d8') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:31:42 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:42 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:31:42 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:31:42 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:42 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapd8727f3f-d8, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:31:42 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapd8727f3f-d8, col_values=(('external_ids', {'iface-id': 'd8727f3f-d838-442c-a9a8-278f38aa87d4', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:b4:8f:2e', 'vm-uuid': '7bef74f4-3bb3-4ad0-bff7-34156be31249'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:31:42 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:42 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:31:42 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:42 user nova-compute[72351]: INFO os_vif [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:b4:8f:2e,bridge_name='br-int',has_traffic_filtering=True,id=d8727f3f-d838-442c-a9a8-278f38aa87d4,network=Network(626455de-9781-4ee8-b0be-c51ac124b189),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd8727f3f-d8')
Aug 09 02:31:42 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:31:42 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] No VIF found with MAC fa:16:3e:b4:8f:2e, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:31:42 user nova-compute[72351]: DEBUG nova.network.neutron [req-91f1989e-94b6-4779-9ae9-ed13fcf1e7ec req-4893c95b-d735-4f81-a89a-b78ed4d8b50e service nova] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Updated VIF entry in instance network info cache for port d8727f3f-d838-442c-a9a8-278f38aa87d4. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:31:42 user nova-compute[72351]: DEBUG nova.network.neutron [req-91f1989e-94b6-4779-9ae9-ed13fcf1e7ec req-4893c95b-d735-4f81-a89a-b78ed4d8b50e service nova] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Updating instance_info_cache with network_info: [{"id": "d8727f3f-d838-442c-a9a8-278f38aa87d4", "address": "fa:16:3e:b4:8f:2e", "network": {"id": "626455de-9781-4ee8-b0be-c51ac124b189", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-1560904912-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0dd686dc07ca42f39aa898351ca281c2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd8727f3f-d8", "ovs_interfaceid": "d8727f3f-d838-442c-a9a8-278f38aa87d4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:31:42 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-91f1989e-94b6-4779-9ae9-ed13fcf1e7ec req-4893c95b-d735-4f81-a89a-b78ed4d8b50e service nova] Releasing lock "refresh_cache-7bef74f4-3bb3-4ad0-bff7-34156be31249" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:31:43 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:43 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Acquiring lock "38a2b8e3-337d-416a-8a8c-da4921123db6" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:31:43 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:31:43 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Acquiring lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:31:43 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:31:43 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.004s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:31:43 user nova-compute[72351]: INFO nova.compute.manager [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Terminating instance
Aug 09 02:31:43 user nova-compute[72351]: DEBUG nova.compute.manager [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Start destroying the instance on the hypervisor. {{(pid=72351) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3124}}
Aug 09 02:31:43 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:43 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:43 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:43 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:44 user nova-compute[72351]: DEBUG nova.compute.manager [req-cce6d638-0121-40b8-87e8-402034cfcad3 req-71a26414-5d7f-4fd1-976e-d58bdc3b45cb service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received event network-vif-unplugged-0ebd83d0-5519-4945-8637-90c2ce024ac1 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:31:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-cce6d638-0121-40b8-87e8-402034cfcad3 req-71a26414-5d7f-4fd1-976e-d58bdc3b45cb service nova] Acquiring lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:31:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-cce6d638-0121-40b8-87e8-402034cfcad3 req-71a26414-5d7f-4fd1-976e-d58bdc3b45cb service nova] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:31:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-cce6d638-0121-40b8-87e8-402034cfcad3 req-71a26414-5d7f-4fd1-976e-d58bdc3b45cb service nova] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:31:44 user nova-compute[72351]: DEBUG nova.compute.manager [req-cce6d638-0121-40b8-87e8-402034cfcad3 req-71a26414-5d7f-4fd1-976e-d58bdc3b45cb service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] No waiting events found dispatching network-vif-unplugged-0ebd83d0-5519-4945-8637-90c2ce024ac1 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:31:44 user nova-compute[72351]: DEBUG nova.compute.manager [req-cce6d638-0121-40b8-87e8-402034cfcad3 req-71a26414-5d7f-4fd1-976e-d58bdc3b45cb service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received event network-vif-unplugged-0ebd83d0-5519-4945-8637-90c2ce024ac1 for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:31:44 user nova-compute[72351]: DEBUG nova.compute.manager [req-56b6d8e5-d3a3-473b-9736-226509045afa req-cf50cb6d-dac1-4f86-bc5d-9d27ea3c9c1f service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received event network-vif-unplugged-8b1245ae-85e9-49c3-9631-a288a172e4c1 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:31:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-56b6d8e5-d3a3-473b-9736-226509045afa req-cf50cb6d-dac1-4f86-bc5d-9d27ea3c9c1f service nova] Acquiring lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:31:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-56b6d8e5-d3a3-473b-9736-226509045afa req-cf50cb6d-dac1-4f86-bc5d-9d27ea3c9c1f service nova] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:31:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-56b6d8e5-d3a3-473b-9736-226509045afa req-cf50cb6d-dac1-4f86-bc5d-9d27ea3c9c1f service nova] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:31:44 user nova-compute[72351]: DEBUG nova.compute.manager [req-56b6d8e5-d3a3-473b-9736-226509045afa req-cf50cb6d-dac1-4f86-bc5d-9d27ea3c9c1f service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] No waiting events found dispatching network-vif-unplugged-8b1245ae-85e9-49c3-9631-a288a172e4c1 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:31:44 user nova-compute[72351]: DEBUG nova.compute.manager [req-56b6d8e5-d3a3-473b-9736-226509045afa req-cf50cb6d-dac1-4f86-bc5d-9d27ea3c9c1f service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received event network-vif-unplugged-8b1245ae-85e9-49c3-9631-a288a172e4c1 for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:31:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:44 user nova-compute[72351]: DEBUG nova.compute.manager [req-77993502-f616-42db-9226-6aefae67795f req-68791f48-e515-4688-9fe9-a6abe1e86084 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received event network-vif-unplugged-48dfbf61-0142-4478-9ec1-585d5fbe11da {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:31:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-77993502-f616-42db-9226-6aefae67795f req-68791f48-e515-4688-9fe9-a6abe1e86084 service nova] Acquiring lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:31:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-77993502-f616-42db-9226-6aefae67795f req-68791f48-e515-4688-9fe9-a6abe1e86084 service nova] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:31:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-77993502-f616-42db-9226-6aefae67795f req-68791f48-e515-4688-9fe9-a6abe1e86084 service nova] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:31:44 user nova-compute[72351]: DEBUG nova.compute.manager [req-77993502-f616-42db-9226-6aefae67795f req-68791f48-e515-4688-9fe9-a6abe1e86084 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] No waiting events found dispatching network-vif-unplugged-48dfbf61-0142-4478-9ec1-585d5fbe11da {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:31:44 user nova-compute[72351]: DEBUG nova.compute.manager [req-77993502-f616-42db-9226-6aefae67795f req-68791f48-e515-4688-9fe9-a6abe1e86084 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received event network-vif-unplugged-48dfbf61-0142-4478-9ec1-585d5fbe11da for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:31:45 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Instance destroyed successfully.
Aug 09 02:31:45 user nova-compute[72351]: DEBUG nova.objects.instance [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Lazy-loading 'resources' on Instance uuid 38a2b8e3-337d-416a-8a8c-da4921123db6 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:31:45 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-08-09T02:29:47Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-973345123',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-973345123',id=33,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBC5HMwUPTdKeiDcsJ4TkVOxRLOG1g/yzvRBKkBQKOQuDjCRYdbTKK7gH9+Y0CpEbv5BvHxw4RfGS5kTqYSc0L/jaTiuzJMVzMoXsxa5nneD0wyJcR62H0YaKv6Pgz9BCcQ==',key_name='tempest-keypair-2018857894',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:30:26Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='fb5ff9ee85084536b6479ed16b24293d',ramdisk_id='',reservation_id='r-oubsycpu',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest_v242-942602465',owner_user_name='tempest-TaggedBootDevicesTest_v242-942602465-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:30:27Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='76924d471e204a5ebee3c99280e62c91',uuid=38a2b8e3-337d-416a-8a8c-da4921123db6,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "0ebd83d0-5519-4945-8637-90c2ce024ac1", "address": "fa:16:3e:81:a2:62", "network": {"id": "c45d6719-95f9-4856-9b34-9d89c22ff899", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-927662772-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.149", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0ebd83d0-55", "ovs_interfaceid": "0ebd83d0-5519-4945-8637-90c2ce024ac1", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:31:45 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Converting VIF {"id": "0ebd83d0-5519-4945-8637-90c2ce024ac1", "address": "fa:16:3e:81:a2:62", "network": {"id": "c45d6719-95f9-4856-9b34-9d89c22ff899", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-927662772-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.149", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0ebd83d0-55", "ovs_interfaceid": "0ebd83d0-5519-4945-8637-90c2ce024ac1", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:31:45 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:81:a2:62,bridge_name='br-int',has_traffic_filtering=True,id=0ebd83d0-5519-4945-8637-90c2ce024ac1,network=Network(c45d6719-95f9-4856-9b34-9d89c22ff899),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap0ebd83d0-55') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:31:45 user nova-compute[72351]: DEBUG os_vif [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:81:a2:62,bridge_name='br-int',has_traffic_filtering=True,id=0ebd83d0-5519-4945-8637-90c2ce024ac1,network=Network(c45d6719-95f9-4856-9b34-9d89c22ff899),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap0ebd83d0-55') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:31:45 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:45 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap0ebd83d0-55, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:31:45 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:31:45 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:45 user nova-compute[72351]: INFO os_vif [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:81:a2:62,bridge_name='br-int',has_traffic_filtering=True,id=0ebd83d0-5519-4945-8637-90c2ce024ac1,network=Network(c45d6719-95f9-4856-9b34-9d89c22ff899),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap0ebd83d0-55')
Aug 09 02:31:45 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-08-09T02:29:47Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-973345123',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-973345123',id=33,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBC5HMwUPTdKeiDcsJ4TkVOxRLOG1g/yzvRBKkBQKOQuDjCRYdbTKK7gH9+Y0CpEbv5BvHxw4RfGS5kTqYSc0L/jaTiuzJMVzMoXsxa5nneD0wyJcR62H0YaKv6Pgz9BCcQ==',key_name='tempest-keypair-2018857894',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:30:26Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='fb5ff9ee85084536b6479ed16b24293d',ramdisk_id='',reservation_id='r-oubsycpu',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest_v242-942602465',owner_user_name='tempest-TaggedBootDevicesTest_v242-942602465-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:30:27Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='76924d471e204a5ebee3c99280e62c91',uuid=38a2b8e3-337d-416a-8a8c-da4921123db6,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "8b1245ae-85e9-49c3-9631-a288a172e4c1", "address": "fa:16:3e:a8:7b:e4", "network": {"id": "4617a3d0-dcb5-42a4-85d3-358087aeac69", "bridge": "br-int", "label": "tempest-device-tagging-net1-1844430962", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.34", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8b1245ae-85", "ovs_interfaceid": "8b1245ae-85e9-49c3-9631-a288a172e4c1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:31:45 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Converting VIF {"id": "8b1245ae-85e9-49c3-9631-a288a172e4c1", "address": "fa:16:3e:a8:7b:e4", "network": {"id": "4617a3d0-dcb5-42a4-85d3-358087aeac69", "bridge": "br-int", "label": "tempest-device-tagging-net1-1844430962", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.34", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8b1245ae-85", "ovs_interfaceid": "8b1245ae-85e9-49c3-9631-a288a172e4c1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:31:45 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:a8:7b:e4,bridge_name='br-int',has_traffic_filtering=True,id=8b1245ae-85e9-49c3-9631-a288a172e4c1,network=Network(4617a3d0-dcb5-42a4-85d3-358087aeac69),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap8b1245ae-85') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:31:45 user nova-compute[72351]: DEBUG os_vif [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:a8:7b:e4,bridge_name='br-int',has_traffic_filtering=True,id=8b1245ae-85e9-49c3-9631-a288a172e4c1,network=Network(4617a3d0-dcb5-42a4-85d3-358087aeac69),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap8b1245ae-85') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:31:45 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:45 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap8b1245ae-85, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:31:45 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:45 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:31:45 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:45 user nova-compute[72351]: INFO os_vif [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:a8:7b:e4,bridge_name='br-int',has_traffic_filtering=True,id=8b1245ae-85e9-49c3-9631-a288a172e4c1,network=Network(4617a3d0-dcb5-42a4-85d3-358087aeac69),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap8b1245ae-85')
Aug 09 02:31:45 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-08-09T02:29:47Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-973345123',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-973345123',id=33,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBC5HMwUPTdKeiDcsJ4TkVOxRLOG1g/yzvRBKkBQKOQuDjCRYdbTKK7gH9+Y0CpEbv5BvHxw4RfGS5kTqYSc0L/jaTiuzJMVzMoXsxa5nneD0wyJcR62H0YaKv6Pgz9BCcQ==',key_name='tempest-keypair-2018857894',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:30:26Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='fb5ff9ee85084536b6479ed16b24293d',ramdisk_id='',reservation_id='r-oubsycpu',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest_v242-942602465',owner_user_name='tempest-TaggedBootDevicesTest_v242-942602465-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:30:27Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='76924d471e204a5ebee3c99280e62c91',uuid=38a2b8e3-337d-416a-8a8c-da4921123db6,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "1be27a59-07a4-4b6a-b25a-ce6d7efac4a3", "address": "fa:16:3e:38:62:75", "network": {"id": "4617a3d0-dcb5-42a4-85d3-358087aeac69", "bridge": "br-int", "label": "tempest-device-tagging-net1-1844430962", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.226", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1be27a59-07", "ovs_interfaceid": "1be27a59-07a4-4b6a-b25a-ce6d7efac4a3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:31:45 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Converting VIF {"id": "1be27a59-07a4-4b6a-b25a-ce6d7efac4a3", "address": "fa:16:3e:38:62:75", "network": {"id": "4617a3d0-dcb5-42a4-85d3-358087aeac69", "bridge": "br-int", "label": "tempest-device-tagging-net1-1844430962", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.226", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1be27a59-07", "ovs_interfaceid": "1be27a59-07a4-4b6a-b25a-ce6d7efac4a3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:31:45 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:38:62:75,bridge_name='br-int',has_traffic_filtering=True,id=1be27a59-07a4-4b6a-b25a-ce6d7efac4a3,network=Network(4617a3d0-dcb5-42a4-85d3-358087aeac69),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap1be27a59-07') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:31:45 user nova-compute[72351]: DEBUG os_vif [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:38:62:75,bridge_name='br-int',has_traffic_filtering=True,id=1be27a59-07a4-4b6a-b25a-ce6d7efac4a3,network=Network(4617a3d0-dcb5-42a4-85d3-358087aeac69),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap1be27a59-07') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:31:45 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:45 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap1be27a59-07, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:31:45 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:45 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:31:45 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:45 user nova-compute[72351]: INFO os_vif [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:38:62:75,bridge_name='br-int',has_traffic_filtering=True,id=1be27a59-07a4-4b6a-b25a-ce6d7efac4a3,network=Network(4617a3d0-dcb5-42a4-85d3-358087aeac69),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap1be27a59-07')
Aug 09 02:31:45 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-08-09T02:29:47Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-973345123',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-973345123',id=33,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBC5HMwUPTdKeiDcsJ4TkVOxRLOG1g/yzvRBKkBQKOQuDjCRYdbTKK7gH9+Y0CpEbv5BvHxw4RfGS5kTqYSc0L/jaTiuzJMVzMoXsxa5nneD0wyJcR62H0YaKv6Pgz9BCcQ==',key_name='tempest-keypair-2018857894',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:30:26Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='fb5ff9ee85084536b6479ed16b24293d',ramdisk_id='',reservation_id='r-oubsycpu',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest_v242-942602465',owner_user_name='tempest-TaggedBootDevicesTest_v242-942602465-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:30:27Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='76924d471e204a5ebee3c99280e62c91',uuid=38a2b8e3-337d-416a-8a8c-da4921123db6,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "ccb930f6-9625-450b-b01b-c5009e610ae1", "address": "fa:16:3e:a9:0c:63", "network": {"id": "4617a3d0-dcb5-42a4-85d3-358087aeac69", "bridge": "br-int", "label": "tempest-device-tagging-net1-1844430962", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.237", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapccb930f6-96", "ovs_interfaceid": "ccb930f6-9625-450b-b01b-c5009e610ae1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:31:45 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Converting VIF {"id": "ccb930f6-9625-450b-b01b-c5009e610ae1", "address": "fa:16:3e:a9:0c:63", "network": {"id": "4617a3d0-dcb5-42a4-85d3-358087aeac69", "bridge": "br-int", "label": "tempest-device-tagging-net1-1844430962", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.237", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapccb930f6-96", "ovs_interfaceid": "ccb930f6-9625-450b-b01b-c5009e610ae1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:31:45 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:a9:0c:63,bridge_name='br-int',has_traffic_filtering=True,id=ccb930f6-9625-450b-b01b-c5009e610ae1,network=Network(4617a3d0-dcb5-42a4-85d3-358087aeac69),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapccb930f6-96') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:31:45 user nova-compute[72351]: DEBUG os_vif [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:a9:0c:63,bridge_name='br-int',has_traffic_filtering=True,id=ccb930f6-9625-450b-b01b-c5009e610ae1,network=Network(4617a3d0-dcb5-42a4-85d3-358087aeac69),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapccb930f6-96') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:31:45 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:45 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:45 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapccb930f6-96, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:31:45 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:45 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:31:45 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:45 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:45 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:45 user nova-compute[72351]: INFO os_vif [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:a9:0c:63,bridge_name='br-int',has_traffic_filtering=True,id=ccb930f6-9625-450b-b01b-c5009e610ae1,network=Network(4617a3d0-dcb5-42a4-85d3-358087aeac69),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapccb930f6-96')
Aug 09 02:31:45 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-08-09T02:29:47Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-973345123',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-973345123',id=33,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBC5HMwUPTdKeiDcsJ4TkVOxRLOG1g/yzvRBKkBQKOQuDjCRYdbTKK7gH9+Y0CpEbv5BvHxw4RfGS5kTqYSc0L/jaTiuzJMVzMoXsxa5nneD0wyJcR62H0YaKv6Pgz9BCcQ==',key_name='tempest-keypair-2018857894',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:30:26Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='fb5ff9ee85084536b6479ed16b24293d',ramdisk_id='',reservation_id='r-oubsycpu',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest_v242-942602465',owner_user_name='tempest-TaggedBootDevicesTest_v242-942602465-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:30:27Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='76924d471e204a5ebee3c99280e62c91',uuid=38a2b8e3-337d-416a-8a8c-da4921123db6,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "d083d9f2-dfe4-4a73-a624-3336f2f02b0d", "address": "fa:16:3e:a0:b1:db", "network": {"id": "4617a3d0-dcb5-42a4-85d3-358087aeac69", "bridge": "br-int", "label": "tempest-device-tagging-net1-1844430962", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.185", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd083d9f2-df", "ovs_interfaceid": "d083d9f2-dfe4-4a73-a624-3336f2f02b0d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:31:45 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Converting VIF {"id": "d083d9f2-dfe4-4a73-a624-3336f2f02b0d", "address": "fa:16:3e:a0:b1:db", "network": {"id": "4617a3d0-dcb5-42a4-85d3-358087aeac69", "bridge": "br-int", "label": "tempest-device-tagging-net1-1844430962", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.185", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd083d9f2-df", "ovs_interfaceid": "d083d9f2-dfe4-4a73-a624-3336f2f02b0d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:31:45 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:a0:b1:db,bridge_name='br-int',has_traffic_filtering=True,id=d083d9f2-dfe4-4a73-a624-3336f2f02b0d,network=Network(4617a3d0-dcb5-42a4-85d3-358087aeac69),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd083d9f2-df') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:31:45 user nova-compute[72351]: DEBUG os_vif [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:a0:b1:db,bridge_name='br-int',has_traffic_filtering=True,id=d083d9f2-dfe4-4a73-a624-3336f2f02b0d,network=Network(4617a3d0-dcb5-42a4-85d3-358087aeac69),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd083d9f2-df') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:31:45 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:45 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapd083d9f2-df, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:31:45 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:31:45 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:45 user nova-compute[72351]: INFO os_vif [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:a0:b1:db,bridge_name='br-int',has_traffic_filtering=True,id=d083d9f2-dfe4-4a73-a624-3336f2f02b0d,network=Network(4617a3d0-dcb5-42a4-85d3-358087aeac69),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd083d9f2-df')
Aug 09 02:31:45 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-08-09T02:29:47Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-973345123',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-973345123',id=33,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBC5HMwUPTdKeiDcsJ4TkVOxRLOG1g/yzvRBKkBQKOQuDjCRYdbTKK7gH9+Y0CpEbv5BvHxw4RfGS5kTqYSc0L/jaTiuzJMVzMoXsxa5nneD0wyJcR62H0YaKv6Pgz9BCcQ==',key_name='tempest-keypair-2018857894',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:30:26Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='fb5ff9ee85084536b6479ed16b24293d',ramdisk_id='',reservation_id='r-oubsycpu',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest_v242-942602465',owner_user_name='tempest-TaggedBootDevicesTest_v242-942602465-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:30:27Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='76924d471e204a5ebee3c99280e62c91',uuid=38a2b8e3-337d-416a-8a8c-da4921123db6,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "a88b872b-9340-4597-8ab4-ea6173dc4e4c", "address": "fa:16:3e:cf:3d:24", "network": {"id": "2984e5a6-dc0d-40ee-8064-f78a7686663e", "bridge": "br-int", "label": "tempest-device-tagging-net2-1630262622", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa88b872b-93", "ovs_interfaceid": "a88b872b-9340-4597-8ab4-ea6173dc4e4c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:31:45 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Converting VIF {"id": "a88b872b-9340-4597-8ab4-ea6173dc4e4c", "address": "fa:16:3e:cf:3d:24", "network": {"id": "2984e5a6-dc0d-40ee-8064-f78a7686663e", "bridge": "br-int", "label": "tempest-device-tagging-net2-1630262622", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa88b872b-93", "ovs_interfaceid": "a88b872b-9340-4597-8ab4-ea6173dc4e4c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:31:45 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:cf:3d:24,bridge_name='br-int',has_traffic_filtering=True,id=a88b872b-9340-4597-8ab4-ea6173dc4e4c,network=Network(2984e5a6-dc0d-40ee-8064-f78a7686663e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa88b872b-93') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:31:45 user nova-compute[72351]: DEBUG os_vif [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:cf:3d:24,bridge_name='br-int',has_traffic_filtering=True,id=a88b872b-9340-4597-8ab4-ea6173dc4e4c,network=Network(2984e5a6-dc0d-40ee-8064-f78a7686663e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa88b872b-93') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:31:45 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:45 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapa88b872b-93, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:31:45 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:45 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:46 user nova-compute[72351]: INFO os_vif [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:cf:3d:24,bridge_name='br-int',has_traffic_filtering=True,id=a88b872b-9340-4597-8ab4-ea6173dc4e4c,network=Network(2984e5a6-dc0d-40ee-8064-f78a7686663e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa88b872b-93')
Aug 09 02:31:46 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-08-09T02:29:47Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-973345123',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-973345123',id=33,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBC5HMwUPTdKeiDcsJ4TkVOxRLOG1g/yzvRBKkBQKOQuDjCRYdbTKK7gH9+Y0CpEbv5BvHxw4RfGS5kTqYSc0L/jaTiuzJMVzMoXsxa5nneD0wyJcR62H0YaKv6Pgz9BCcQ==',key_name='tempest-keypair-2018857894',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:30:26Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='fb5ff9ee85084536b6479ed16b24293d',ramdisk_id='',reservation_id='r-oubsycpu',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest_v242-942602465',owner_user_name='tempest-TaggedBootDevicesTest_v242-942602465-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:30:27Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='76924d471e204a5ebee3c99280e62c91',uuid=38a2b8e3-337d-416a-8a8c-da4921123db6,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "48dfbf61-0142-4478-9ec1-585d5fbe11da", "address": "fa:16:3e:5b:d8:08", "network": {"id": "2984e5a6-dc0d-40ee-8064-f78a7686663e", "bridge": "br-int", "label": "tempest-device-tagging-net2-1630262622", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap48dfbf61-01", "ovs_interfaceid": "48dfbf61-0142-4478-9ec1-585d5fbe11da", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Converting VIF {"id": "48dfbf61-0142-4478-9ec1-585d5fbe11da", "address": "fa:16:3e:5b:d8:08", "network": {"id": "2984e5a6-dc0d-40ee-8064-f78a7686663e", "bridge": "br-int", "label": "tempest-device-tagging-net2-1630262622", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap48dfbf61-01", "ovs_interfaceid": "48dfbf61-0142-4478-9ec1-585d5fbe11da", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:5b:d8:08,bridge_name='br-int',has_traffic_filtering=True,id=48dfbf61-0142-4478-9ec1-585d5fbe11da,network=Network(2984e5a6-dc0d-40ee-8064-f78a7686663e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap48dfbf61-01') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG os_vif [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:5b:d8:08,bridge_name='br-int',has_traffic_filtering=True,id=48dfbf61-0142-4478-9ec1-585d5fbe11da,network=Network(2984e5a6-dc0d-40ee-8064-f78a7686663e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap48dfbf61-01') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap48dfbf61-01, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:46 user nova-compute[72351]: INFO os_vif [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:5b:d8:08,bridge_name='br-int',has_traffic_filtering=True,id=48dfbf61-0142-4478-9ec1-585d5fbe11da,network=Network(2984e5a6-dc0d-40ee-8064-f78a7686663e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap48dfbf61-01')
Aug 09 02:31:46 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'UYXtqndm', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdd'}, None), 'kwargs': True}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:31:46 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:31:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[c3f61b29-9be6-403e-a126-3214c883d750]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.037s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[2bfacfe2-fd21-4b2d-8be5-ec48b547295d]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e (non-flash)\ntcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70 (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)\ntcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)\ntcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:31:46 user nova-compute[72351]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 (non-flash)
Aug 09 02:31:46 user nova-compute[72351]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e (non-flash)
Aug 09 02:31:46 user nova-compute[72351]: tcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70 (non-flash)
Aug 09 02:31:46 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:31:46 user nova-compute[72351]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)
Aug 09 02:31:46 user nova-compute[72351]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:31:46 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:31:46 user nova-compute[72351]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 (non-flash)
Aug 09 02:31:46 user nova-compute[72351]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e (non-flash)
Aug 09 02:31:46 user nova-compute[72351]: tcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70 (non-flash)
Aug 09 02:31:46 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:31:46 user nova-compute[72351]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)
Aug 09 02:31:46 user nova-compute[72351]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:31:46 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fcfdc1a7130>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70'): ({'sdd'}, set())}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Removing single pathed devices sdd {{(pid=72351) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.026s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[7e554bc2-f15e-4feb-b3ac-fc09b427e7e2]: (4, ('path checker states:\nup                  7\n\npaths: 0\nbusy: False\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdd {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdd" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[f203ebe1-bc28-439c-9eea-d2dad73e169e]: (4, ('ok\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Flushing IO for device /dev/sdd {{(pid=72351) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdd {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdd" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[90680325-d0cd-4df7-9c60-aee1b1a48907]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Remove SCSI device /dev/sdd with /sys/block/sdd/device/delete {{(pid=72351) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdd/device/delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdd/device/delete" returned: 0 in 0.045s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[8488266e-df10-4ea2-8247-6928075bea91]: (4, ('1', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Checking to see if SCSI volumes sdd have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] SCSI volumes sdd have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70')] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.025s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[356c374a-0c14-4ba1-b05f-5b620142c5f0]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70 -p 172.16.0.220:3260 --logout {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG nova.compute.manager [req-febf46a9-f0c3-43df-8769-a14e43188802 req-1a6515d1-b9d3-4665-bb34-fe655425ef8e service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received event network-vif-plugged-0ebd83d0-5519-4945-8637-90c2ce024ac1 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-febf46a9-f0c3-43df-8769-a14e43188802 req-1a6515d1-b9d3-4665-bb34-fe655425ef8e service nova] Acquiring lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-febf46a9-f0c3-43df-8769-a14e43188802 req-1a6515d1-b9d3-4665-bb34-fe655425ef8e service nova] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-febf46a9-f0c3-43df-8769-a14e43188802 req-1a6515d1-b9d3-4665-bb34-fe655425ef8e service nova] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG nova.compute.manager [req-febf46a9-f0c3-43df-8769-a14e43188802 req-1a6515d1-b9d3-4665-bb34-fe655425ef8e service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] No waiting events found dispatching network-vif-plugged-0ebd83d0-5519-4945-8637-90c2ce024ac1 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:31:46 user nova-compute[72351]: WARNING nova.compute.manager [req-febf46a9-f0c3-43df-8769-a14e43188802 req-1a6515d1-b9d3-4665-bb34-fe655425ef8e service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received unexpected event network-vif-plugged-0ebd83d0-5519-4945-8637-90c2ce024ac1 for instance with vm_state active and task_state deleting.
Aug 09 02:31:46 user nova-compute[72351]: DEBUG nova.compute.manager [req-febf46a9-f0c3-43df-8769-a14e43188802 req-1a6515d1-b9d3-4665-bb34-fe655425ef8e service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received event network-vif-unplugged-1be27a59-07a4-4b6a-b25a-ce6d7efac4a3 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-febf46a9-f0c3-43df-8769-a14e43188802 req-1a6515d1-b9d3-4665-bb34-fe655425ef8e service nova] Acquiring lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-febf46a9-f0c3-43df-8769-a14e43188802 req-1a6515d1-b9d3-4665-bb34-fe655425ef8e service nova] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-febf46a9-f0c3-43df-8769-a14e43188802 req-1a6515d1-b9d3-4665-bb34-fe655425ef8e service nova] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG nova.compute.manager [req-febf46a9-f0c3-43df-8769-a14e43188802 req-1a6515d1-b9d3-4665-bb34-fe655425ef8e service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] No waiting events found dispatching network-vif-unplugged-1be27a59-07a4-4b6a-b25a-ce6d7efac4a3 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG nova.compute.manager [req-febf46a9-f0c3-43df-8769-a14e43188802 req-1a6515d1-b9d3-4665-bb34-fe655425ef8e service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received event network-vif-unplugged-1be27a59-07a4-4b6a-b25a-ce6d7efac4a3 for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG nova.compute.manager [req-febf46a9-f0c3-43df-8769-a14e43188802 req-1a6515d1-b9d3-4665-bb34-fe655425ef8e service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received event network-vif-plugged-1be27a59-07a4-4b6a-b25a-ce6d7efac4a3 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-febf46a9-f0c3-43df-8769-a14e43188802 req-1a6515d1-b9d3-4665-bb34-fe655425ef8e service nova] Acquiring lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-febf46a9-f0c3-43df-8769-a14e43188802 req-1a6515d1-b9d3-4665-bb34-fe655425ef8e service nova] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-febf46a9-f0c3-43df-8769-a14e43188802 req-1a6515d1-b9d3-4665-bb34-fe655425ef8e service nova] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG nova.compute.manager [req-febf46a9-f0c3-43df-8769-a14e43188802 req-1a6515d1-b9d3-4665-bb34-fe655425ef8e service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] No waiting events found dispatching network-vif-plugged-1be27a59-07a4-4b6a-b25a-ce6d7efac4a3 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:31:46 user nova-compute[72351]: WARNING nova.compute.manager [req-febf46a9-f0c3-43df-8769-a14e43188802 req-1a6515d1-b9d3-4665-bb34-fe655425ef8e service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received unexpected event network-vif-plugged-1be27a59-07a4-4b6a-b25a-ce6d7efac4a3 for instance with vm_state active and task_state deleting.
Aug 09 02:31:46 user nova-compute[72351]: DEBUG nova.compute.manager [req-febf46a9-f0c3-43df-8769-a14e43188802 req-1a6515d1-b9d3-4665-bb34-fe655425ef8e service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received event network-vif-unplugged-ccb930f6-9625-450b-b01b-c5009e610ae1 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-febf46a9-f0c3-43df-8769-a14e43188802 req-1a6515d1-b9d3-4665-bb34-fe655425ef8e service nova] Acquiring lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-febf46a9-f0c3-43df-8769-a14e43188802 req-1a6515d1-b9d3-4665-bb34-fe655425ef8e service nova] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.010s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-febf46a9-f0c3-43df-8769-a14e43188802 req-1a6515d1-b9d3-4665-bb34-fe655425ef8e service nova] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG nova.compute.manager [req-febf46a9-f0c3-43df-8769-a14e43188802 req-1a6515d1-b9d3-4665-bb34-fe655425ef8e service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] No waiting events found dispatching network-vif-unplugged-ccb930f6-9625-450b-b01b-c5009e610ae1 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG nova.compute.manager [req-febf46a9-f0c3-43df-8769-a14e43188802 req-1a6515d1-b9d3-4665-bb34-fe655425ef8e service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received event network-vif-unplugged-ccb930f6-9625-450b-b01b-c5009e610ae1 for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG nova.compute.manager [req-febf46a9-f0c3-43df-8769-a14e43188802 req-1a6515d1-b9d3-4665-bb34-fe655425ef8e service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received event network-vif-plugged-ccb930f6-9625-450b-b01b-c5009e610ae1 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-febf46a9-f0c3-43df-8769-a14e43188802 req-1a6515d1-b9d3-4665-bb34-fe655425ef8e service nova] Acquiring lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-febf46a9-f0c3-43df-8769-a14e43188802 req-1a6515d1-b9d3-4665-bb34-fe655425ef8e service nova] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-febf46a9-f0c3-43df-8769-a14e43188802 req-1a6515d1-b9d3-4665-bb34-fe655425ef8e service nova] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG nova.compute.manager [req-febf46a9-f0c3-43df-8769-a14e43188802 req-1a6515d1-b9d3-4665-bb34-fe655425ef8e service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] No waiting events found dispatching network-vif-plugged-ccb930f6-9625-450b-b01b-c5009e610ae1 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:31:46 user nova-compute[72351]: WARNING nova.compute.manager [req-febf46a9-f0c3-43df-8769-a14e43188802 req-1a6515d1-b9d3-4665-bb34-fe655425ef8e service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received unexpected event network-vif-plugged-ccb930f6-9625-450b-b01b-c5009e610ae1 for instance with vm_state active and task_state deleting.
Aug 09 02:31:46 user nova-compute[72351]: DEBUG nova.compute.manager [req-febf46a9-f0c3-43df-8769-a14e43188802 req-1a6515d1-b9d3-4665-bb34-fe655425ef8e service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received event network-vif-unplugged-d083d9f2-dfe4-4a73-a624-3336f2f02b0d {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-febf46a9-f0c3-43df-8769-a14e43188802 req-1a6515d1-b9d3-4665-bb34-fe655425ef8e service nova] Acquiring lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-febf46a9-f0c3-43df-8769-a14e43188802 req-1a6515d1-b9d3-4665-bb34-fe655425ef8e service nova] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-febf46a9-f0c3-43df-8769-a14e43188802 req-1a6515d1-b9d3-4665-bb34-fe655425ef8e service nova] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG nova.compute.manager [req-febf46a9-f0c3-43df-8769-a14e43188802 req-1a6515d1-b9d3-4665-bb34-fe655425ef8e service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] No waiting events found dispatching network-vif-unplugged-d083d9f2-dfe4-4a73-a624-3336f2f02b0d {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG nova.compute.manager [req-febf46a9-f0c3-43df-8769-a14e43188802 req-1a6515d1-b9d3-4665-bb34-fe655425ef8e service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received event network-vif-unplugged-d083d9f2-dfe4-4a73-a624-3336f2f02b0d for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70 -p 172.16.0.220:3260 --logout" returned: 0 in 0.083s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ee9c13f9-32c8-42f9-9379-747c0af01f63]: (4, ('Logging out of session [sid: 35, target: iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70, portal: 172.16.0.220,3260]\nLogout of [sid: 35, target: iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 35, target: iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70, portal: 172.16.0.220,3260]
Aug 09 02:31:46 user nova-compute[72351]: Logout of [sid: 35, target: iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70, portal: 172.16.0.220,3260] successful.
Aug 09 02:31:46 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70 -p 172.16.0.220:3260 --op delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG nova.compute.manager [req-a9e8498b-f23b-4a90-8bfb-a08e3457c029 req-7eefc5b8-996f-462e-a09e-30734f544144 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received event network-vif-plugged-8b1245ae-85e9-49c3-9631-a288a172e4c1 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a9e8498b-f23b-4a90-8bfb-a08e3457c029 req-7eefc5b8-996f-462e-a09e-30734f544144 service nova] Acquiring lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a9e8498b-f23b-4a90-8bfb-a08e3457c029 req-7eefc5b8-996f-462e-a09e-30734f544144 service nova] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a9e8498b-f23b-4a90-8bfb-a08e3457c029 req-7eefc5b8-996f-462e-a09e-30734f544144 service nova] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG nova.compute.manager [req-a9e8498b-f23b-4a90-8bfb-a08e3457c029 req-7eefc5b8-996f-462e-a09e-30734f544144 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] No waiting events found dispatching network-vif-plugged-8b1245ae-85e9-49c3-9631-a288a172e4c1 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:31:46 user nova-compute[72351]: WARNING nova.compute.manager [req-a9e8498b-f23b-4a90-8bfb-a08e3457c029 req-7eefc5b8-996f-462e-a09e-30734f544144 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received unexpected event network-vif-plugged-8b1245ae-85e9-49c3-9631-a288a172e4c1 for instance with vm_state active and task_state deleting.
Aug 09 02:31:46 user nova-compute[72351]: DEBUG nova.compute.manager [req-a9e8498b-f23b-4a90-8bfb-a08e3457c029 req-7eefc5b8-996f-462e-a09e-30734f544144 service nova] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Received event network-vif-plugged-d8727f3f-d838-442c-a9a8-278f38aa87d4 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a9e8498b-f23b-4a90-8bfb-a08e3457c029 req-7eefc5b8-996f-462e-a09e-30734f544144 service nova] Acquiring lock "7bef74f4-3bb3-4ad0-bff7-34156be31249-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a9e8498b-f23b-4a90-8bfb-a08e3457c029 req-7eefc5b8-996f-462e-a09e-30734f544144 service nova] Lock "7bef74f4-3bb3-4ad0-bff7-34156be31249-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a9e8498b-f23b-4a90-8bfb-a08e3457c029 req-7eefc5b8-996f-462e-a09e-30734f544144 service nova] Lock "7bef74f4-3bb3-4ad0-bff7-34156be31249-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG nova.compute.manager [req-a9e8498b-f23b-4a90-8bfb-a08e3457c029 req-7eefc5b8-996f-462e-a09e-30734f544144 service nova] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] No waiting events found dispatching network-vif-plugged-d8727f3f-d838-442c-a9a8-278f38aa87d4 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:31:46 user nova-compute[72351]: WARNING nova.compute.manager [req-a9e8498b-f23b-4a90-8bfb-a08e3457c029 req-7eefc5b8-996f-462e-a09e-30734f544144 service nova] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Received unexpected event network-vif-plugged-d8727f3f-d838-442c-a9a8-278f38aa87d4 for instance with vm_state building and task_state spawning.
Aug 09 02:31:46 user nova-compute[72351]: DEBUG nova.compute.manager [req-a9e8498b-f23b-4a90-8bfb-a08e3457c029 req-7eefc5b8-996f-462e-a09e-30734f544144 service nova] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Received event network-vif-plugged-d8727f3f-d838-442c-a9a8-278f38aa87d4 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a9e8498b-f23b-4a90-8bfb-a08e3457c029 req-7eefc5b8-996f-462e-a09e-30734f544144 service nova] Acquiring lock "7bef74f4-3bb3-4ad0-bff7-34156be31249-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a9e8498b-f23b-4a90-8bfb-a08e3457c029 req-7eefc5b8-996f-462e-a09e-30734f544144 service nova] Lock "7bef74f4-3bb3-4ad0-bff7-34156be31249-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a9e8498b-f23b-4a90-8bfb-a08e3457c029 req-7eefc5b8-996f-462e-a09e-30734f544144 service nova] Lock "7bef74f4-3bb3-4ad0-bff7-34156be31249-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG nova.compute.manager [req-a9e8498b-f23b-4a90-8bfb-a08e3457c029 req-7eefc5b8-996f-462e-a09e-30734f544144 service nova] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] No waiting events found dispatching network-vif-plugged-d8727f3f-d838-442c-a9a8-278f38aa87d4 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:31:46 user nova-compute[72351]: WARNING nova.compute.manager [req-a9e8498b-f23b-4a90-8bfb-a08e3457c029 req-7eefc5b8-996f-462e-a09e-30734f544144 service nova] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Received unexpected event network-vif-plugged-d8727f3f-d838-442c-a9a8-278f38aa87d4 for instance with vm_state building and task_state spawning.
Aug 09 02:31:46 user nova-compute[72351]: DEBUG nova.compute.manager [req-a9e8498b-f23b-4a90-8bfb-a08e3457c029 req-7eefc5b8-996f-462e-a09e-30734f544144 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received event network-vif-unplugged-a88b872b-9340-4597-8ab4-ea6173dc4e4c {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a9e8498b-f23b-4a90-8bfb-a08e3457c029 req-7eefc5b8-996f-462e-a09e-30734f544144 service nova] Acquiring lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a9e8498b-f23b-4a90-8bfb-a08e3457c029 req-7eefc5b8-996f-462e-a09e-30734f544144 service nova] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a9e8498b-f23b-4a90-8bfb-a08e3457c029 req-7eefc5b8-996f-462e-a09e-30734f544144 service nova] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG nova.compute.manager [req-a9e8498b-f23b-4a90-8bfb-a08e3457c029 req-7eefc5b8-996f-462e-a09e-30734f544144 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] No waiting events found dispatching network-vif-unplugged-a88b872b-9340-4597-8ab4-ea6173dc4e4c {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG nova.compute.manager [req-a9e8498b-f23b-4a90-8bfb-a08e3457c029 req-7eefc5b8-996f-462e-a09e-30734f544144 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received event network-vif-unplugged-a88b872b-9340-4597-8ab4-ea6173dc4e4c for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG nova.compute.manager [req-a9e8498b-f23b-4a90-8bfb-a08e3457c029 req-7eefc5b8-996f-462e-a09e-30734f544144 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received event network-vif-plugged-a88b872b-9340-4597-8ab4-ea6173dc4e4c {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a9e8498b-f23b-4a90-8bfb-a08e3457c029 req-7eefc5b8-996f-462e-a09e-30734f544144 service nova] Acquiring lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a9e8498b-f23b-4a90-8bfb-a08e3457c029 req-7eefc5b8-996f-462e-a09e-30734f544144 service nova] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a9e8498b-f23b-4a90-8bfb-a08e3457c029 req-7eefc5b8-996f-462e-a09e-30734f544144 service nova] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG nova.compute.manager [req-a9e8498b-f23b-4a90-8bfb-a08e3457c029 req-7eefc5b8-996f-462e-a09e-30734f544144 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] No waiting events found dispatching network-vif-plugged-a88b872b-9340-4597-8ab4-ea6173dc4e4c {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:31:46 user nova-compute[72351]: WARNING nova.compute.manager [req-a9e8498b-f23b-4a90-8bfb-a08e3457c029 req-7eefc5b8-996f-462e-a09e-30734f544144 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received unexpected event network-vif-plugged-a88b872b-9340-4597-8ab4-ea6173dc4e4c for instance with vm_state active and task_state deleting.
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5aa01207-8de0-4c53-9662-6771d109fe70 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.034s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[7f74d05a-a228-4347-8292-f8a0a3a4087d]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.381s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] <== disconnect_volume: return (384ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'NEfWCzIA', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb'}, None), 'kwargs': True}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:31:46 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:31:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.003s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.024s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[3cd19100-b797-41c0-bd85-46bdeabb0af3]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.032s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[614e3298-7fcc-4c7e-8ce1-d329e0c6d1d1]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)\ntcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)\ntcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:31:46 user nova-compute[72351]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 (non-flash)
Aug 09 02:31:46 user nova-compute[72351]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e (non-flash)
Aug 09 02:31:46 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:31:46 user nova-compute[72351]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)
Aug 09 02:31:46 user nova-compute[72351]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:31:46 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:31:46 user nova-compute[72351]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 (non-flash)
Aug 09 02:31:46 user nova-compute[72351]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e (non-flash)
Aug 09 02:31:46 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:31:46 user nova-compute[72351]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)
Aug 09 02:31:46 user nova-compute[72351]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:31:46 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fcfdc1dcca0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3'): ({'sdb'}, set())}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Removing single pathed devices sdb {{(pid=72351) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.025s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[1e4e21bd-aab7-4f9e-8f77-fc1172fde7fa]: (4, ('path checker states:\nup                  6\n\npaths: 0\nbusy: False\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[3b8d1cac-ce73-4c50-8637-01a22d86a025]: (4, ('ok\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Flushing IO for device /dev/sdb {{(pid=72351) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.015s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[6da22351-d9ba-4e3c-938b-8e68565f59e4]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=72351) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.055s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a372f941-313d-4efe-8677-cfcb9f9a6d7d]: (4, ('1', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] SCSI volumes sdb have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3')] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[0e19af02-04a6-423b-bea8-25f80de5c5d3]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 -p 172.16.0.220:3260 --logout {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548306.743835, 7bef74f4-3bb3-4ad0-bff7-34156be31249 => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:31:46 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] VM Resumed (Lifecycle Event)
Aug 09 02:31:46 user nova-compute[72351]: DEBUG nova.compute.manager [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Guest created on hypervisor {{(pid=72351) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4411}}
Aug 09 02:31:46 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Instance spawned successfully.
Aug 09 02:31:46 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:894}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 -p 172.16.0.220:3260 --logout" returned: 0 in 0.073s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[bde58c5a-4e4e-47ce-bbe0-13c71cb016e3]: (4, ('Logging out of session [sid: 33, target: iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3, portal: 172.16.0.220,3260]\nLogout of [sid: 33, target: iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 33, target: iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3, portal: 172.16.0.220,3260]
Aug 09 02:31:46 user nova-compute[72351]: Logout of [sid: 33, target: iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3, portal: 172.16.0.220,3260] successful.
Aug 09 02:31:46 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 -p 172.16.0.220:3260 --op delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9d91e65d-30f6-4d64-a186-cbd8f80000b3 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.039s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[0bdd53df-fc22-4299-8286-a44fb0b4f2f6]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.356s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] <== disconnect_volume: return (360ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Found default for hw_cdrom_bus of ide {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Found default for hw_disk_bus of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Found default for hw_input_bus of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Found default for hw_pointer_model of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Found default for hw_video_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Found default for hw_vif_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'pefgySaV', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc'}, None), 'kwargs': True}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:31:46 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:31:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[5249295a-09fd-4b62-a50b-3d06a7952683]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:46 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:31:46 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548306.7472425, 7bef74f4-3bb3-4ad0-bff7-34156be31249 => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:31:46 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] VM Started (Lifecycle Event)
Aug 09 02:31:46 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG nova.compute.manager [req-6cb1cc0b-6a48-415e-ae7d-79a261e915ba req-1b1b0dc9-9dc0-4e95-9fd3-8fd7f1e01122 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received event network-vif-plugged-48dfbf61-0142-4478-9ec1-585d5fbe11da {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-6cb1cc0b-6a48-415e-ae7d-79a261e915ba req-1b1b0dc9-9dc0-4e95-9fd3-8fd7f1e01122 service nova] Acquiring lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-6cb1cc0b-6a48-415e-ae7d-79a261e915ba req-1b1b0dc9-9dc0-4e95-9fd3-8fd7f1e01122 service nova] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-6cb1cc0b-6a48-415e-ae7d-79a261e915ba req-1b1b0dc9-9dc0-4e95-9fd3-8fd7f1e01122 service nova] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:31:46 user nova-compute[72351]: DEBUG nova.compute.manager [req-6cb1cc0b-6a48-415e-ae7d-79a261e915ba req-1b1b0dc9-9dc0-4e95-9fd3-8fd7f1e01122 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] No waiting events found dispatching network-vif-plugged-48dfbf61-0142-4478-9ec1-585d5fbe11da {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:31:46 user nova-compute[72351]: WARNING nova.compute.manager [req-6cb1cc0b-6a48-415e-ae7d-79a261e915ba req-1b1b0dc9-9dc0-4e95-9fd3-8fd7f1e01122 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received unexpected event network-vif-plugged-48dfbf61-0142-4478-9ec1-585d5fbe11da for instance with vm_state active and task_state deleting.
Aug 09 02:31:47 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.044s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:47 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[bc595d24-c099-4d37-8022-a8268eae39a0]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)\ntcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)\ntcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:47 user nova-compute[72351]: INFO nova.compute.manager [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Took 6.86 seconds to spawn the instance on the hypervisor.
Aug 09 02:31:47 user nova-compute[72351]: DEBUG nova.compute.manager [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:31:47 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:31:47 user nova-compute[72351]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e (non-flash)
Aug 09 02:31:47 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:31:47 user nova-compute[72351]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)
Aug 09 02:31:47 user nova-compute[72351]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:31:47 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:31:47 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:31:47 user nova-compute[72351]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e (non-flash)
Aug 09 02:31:47 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:31:47 user nova-compute[72351]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)
Aug 09 02:31:47 user nova-compute[72351]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:31:47 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:31:47 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fd00b3d7d90>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e'): ({'sdc'}, set())}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 02:31:47 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Removing single pathed devices sdc {{(pid=72351) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Aug 09 02:31:47 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:31:47 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:47 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.025s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:47 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[0148265d-8289-4841-a6fa-757784c24558]: (4, ('path checker states:\nup                  5\n\npaths: 0\nbusy: False\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:47 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:31:47 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdc {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:47 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdc" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:47 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[7cf51ee6-ddcc-45f8-b306-b21eb67b6f54]: (4, ('ok\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:47 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Flushing IO for device /dev/sdc {{(pid=72351) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Aug 09 02:31:47 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdc {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:47 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdc" returned: 0 in 0.016s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:47 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[914d1414-1726-40b8-bd5b-af0614b5c35f]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:47 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Remove SCSI device /dev/sdc with /sys/block/sdc/device/delete {{(pid=72351) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Aug 09 02:31:47 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdc/device/delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:47 user nova-compute[72351]: INFO nova.compute.manager [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Took 7.97 seconds to build instance.
Aug 09 02:31:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1bbaecce-94d4-4941-a2ab-a18bf95d6f02 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "7bef74f4-3bb3-4ad0-bff7-34156be31249" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 8.065s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:31:47 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdc/device/delete" returned: 0 in 0.047s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:47 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[c15fd9f1-7bd1-4cfe-a980-936f7fb8c64c]: (4, ('1', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:47 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Checking to see if SCSI volumes sdc have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Aug 09 02:31:47 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] SCSI volumes sdc have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Aug 09 02:31:47 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e')] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 02:31:47 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:47 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.026s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:47 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[0b31e64f-1ee6-4bf4-a303-b57a1019fca4]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:47 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:31:47 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e -p 172.16.0.220:3260 --logout {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:47 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e -p 172.16.0.220:3260 --logout" returned: 0 in 0.062s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:47 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[3ba15b8a-5dfb-4080-aa9a-ac2e0aaa0f3e]: (4, ('Logging out of session [sid: 34, target: iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e, portal: 172.16.0.220,3260]\nLogout of [sid: 34, target: iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:47 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 34, target: iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e, portal: 172.16.0.220,3260]
Aug 09 02:31:47 user nova-compute[72351]: Logout of [sid: 34, target: iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e, portal: 172.16.0.220,3260] successful.
Aug 09 02:31:47 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:31:47 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e -p 172.16.0.220:3260 --op delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:31:47 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:aa951fc7-aa57-4fc9-9433-de522d77537e -p 172.16.0.220:3260 --op delete" returned: 0 in 0.034s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:31:47 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[3301b83f-c7e6-4ea3-af24-04b58a6a8083]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:31:47 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:31:47 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.364s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:31:47 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] <== disconnect_volume: return (365ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:31:47 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 02:31:47 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Deleting instance files /opt/stack/data/nova/instances/38a2b8e3-337d-416a-8a8c-da4921123db6_del
Aug 09 02:31:47 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Deletion of /opt/stack/data/nova/instances/38a2b8e3-337d-416a-8a8c-da4921123db6_del complete
Aug 09 02:31:47 user nova-compute[72351]: INFO nova.compute.manager [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Took 3.68 seconds to destroy the instance on the hypervisor.
Aug 09 02:31:47 user nova-compute[72351]: DEBUG oslo.service.loopingcall [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72351) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Aug 09 02:31:47 user nova-compute[72351]: DEBUG nova.compute.manager [-] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Deallocating network for instance {{(pid=72351) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2263}}
Aug 09 02:31:47 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] deallocate_for_instance() {{(pid=72351) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1795}}
Aug 09 02:31:48 user nova-compute[72351]: DEBUG nova.compute.manager [req-0413f98c-5ebf-4477-a07a-b860647978ae req-a2918ced-60a9-4740-a927-663454824fff service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received event network-vif-plugged-d083d9f2-dfe4-4a73-a624-3336f2f02b0d {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:31:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-0413f98c-5ebf-4477-a07a-b860647978ae req-a2918ced-60a9-4740-a927-663454824fff service nova] Acquiring lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:31:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-0413f98c-5ebf-4477-a07a-b860647978ae req-a2918ced-60a9-4740-a927-663454824fff service nova] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:31:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-0413f98c-5ebf-4477-a07a-b860647978ae req-a2918ced-60a9-4740-a927-663454824fff service nova] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:31:48 user nova-compute[72351]: DEBUG nova.compute.manager [req-0413f98c-5ebf-4477-a07a-b860647978ae req-a2918ced-60a9-4740-a927-663454824fff service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] No waiting events found dispatching network-vif-plugged-d083d9f2-dfe4-4a73-a624-3336f2f02b0d {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:31:48 user nova-compute[72351]: WARNING nova.compute.manager [req-0413f98c-5ebf-4477-a07a-b860647978ae req-a2918ced-60a9-4740-a927-663454824fff service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received unexpected event network-vif-plugged-d083d9f2-dfe4-4a73-a624-3336f2f02b0d for instance with vm_state active and task_state deleting.
Aug 09 02:31:48 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:50 user nova-compute[72351]: DEBUG nova.compute.manager [req-989149cf-8282-486e-a21b-8f6b038acbd8 req-2f833cde-b1e6-47e2-bf5e-6020de1b089b service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received event network-vif-deleted-ccb930f6-9625-450b-b01b-c5009e610ae1 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:31:50 user nova-compute[72351]: INFO nova.compute.manager [req-989149cf-8282-486e-a21b-8f6b038acbd8 req-2f833cde-b1e6-47e2-bf5e-6020de1b089b service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Neutron deleted interface ccb930f6-9625-450b-b01b-c5009e610ae1; detaching it from the instance and deleting it from the info cache
Aug 09 02:31:50 user nova-compute[72351]: DEBUG nova.network.neutron [req-989149cf-8282-486e-a21b-8f6b038acbd8 req-2f833cde-b1e6-47e2-bf5e-6020de1b089b service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Updating instance_info_cache with network_info: [{"id": "0ebd83d0-5519-4945-8637-90c2ce024ac1", "address": "fa:16:3e:81:a2:62", "network": {"id": "c45d6719-95f9-4856-9b34-9d89c22ff899", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-927662772-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.149", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0ebd83d0-55", "ovs_interfaceid": "0ebd83d0-5519-4945-8637-90c2ce024ac1", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "8b1245ae-85e9-49c3-9631-a288a172e4c1", "address": "fa:16:3e:a8:7b:e4", "network": {"id": "4617a3d0-dcb5-42a4-85d3-358087aeac69", "bridge": "br-int", "label": "tempest-device-tagging-net1-1844430962", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.34", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8b1245ae-85", "ovs_interfaceid": "8b1245ae-85e9-49c3-9631-a288a172e4c1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "1be27a59-07a4-4b6a-b25a-ce6d7efac4a3", "address": "fa:16:3e:38:62:75", "network": {"id": "4617a3d0-dcb5-42a4-85d3-358087aeac69", "bridge": "br-int", "label": "tempest-device-tagging-net1-1844430962", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.226", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1be27a59-07", "ovs_interfaceid": "1be27a59-07a4-4b6a-b25a-ce6d7efac4a3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "d083d9f2-dfe4-4a73-a624-3336f2f02b0d", "address": "fa:16:3e:a0:b1:db", "network": {"id": "4617a3d0-dcb5-42a4-85d3-358087aeac69", "bridge": "br-int", "label": "tempest-device-tagging-net1-1844430962", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.185", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd083d9f2-df", "ovs_interfaceid": "d083d9f2-dfe4-4a73-a624-3336f2f02b0d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "a88b872b-9340-4597-8ab4-ea6173dc4e4c", "address": "fa:16:3e:cf:3d:24", "network": {"id": "2984e5a6-dc0d-40ee-8064-f78a7686663e", "bridge": "br-int", "label": "tempest-device-tagging-net2-1630262622", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa88b872b-93", "ovs_interfaceid": "a88b872b-9340-4597-8ab4-ea6173dc4e4c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "48dfbf61-0142-4478-9ec1-585d5fbe11da", "address": "fa:16:3e:5b:d8:08", "network": {"id": "2984e5a6-dc0d-40ee-8064-f78a7686663e", "bridge": "br-int", "label": "tempest-device-tagging-net2-1630262622", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "fb5ff9ee85084536b6479ed16b24293d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap48dfbf61-01", "ovs_interfaceid": "48dfbf61-0142-4478-9ec1-585d5fbe11da", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:31:50 user nova-compute[72351]: DEBUG nova.compute.manager [req-989149cf-8282-486e-a21b-8f6b038acbd8 req-2f833cde-b1e6-47e2-bf5e-6020de1b089b service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Detach interface failed, port_id=ccb930f6-9625-450b-b01b-c5009e610ae1, reason: Instance 38a2b8e3-337d-416a-8a8c-da4921123db6 could not be found. {{(pid=72351) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10866}}
Aug 09 02:31:51 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:52 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:31:52 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Took 4.85 seconds to deallocate network for instance.
Aug 09 02:31:52 user nova-compute[72351]: DEBUG nova.compute.manager [req-68eee97b-3ce7-4c9d-b711-e2df7c051f07 req-faaca531-6060-460d-b9f1-2fdd08140786 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received event network-vif-deleted-d083d9f2-dfe4-4a73-a624-3336f2f02b0d {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:31:52 user nova-compute[72351]: DEBUG nova.compute.manager [req-68eee97b-3ce7-4c9d-b711-e2df7c051f07 req-faaca531-6060-460d-b9f1-2fdd08140786 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received event network-vif-deleted-a88b872b-9340-4597-8ab4-ea6173dc4e4c {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:31:52 user nova-compute[72351]: DEBUG nova.compute.manager [req-68eee97b-3ce7-4c9d-b711-e2df7c051f07 req-faaca531-6060-460d-b9f1-2fdd08140786 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received event network-vif-deleted-0ebd83d0-5519-4945-8637-90c2ce024ac1 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:31:52 user nova-compute[72351]: DEBUG nova.compute.manager [req-68eee97b-3ce7-4c9d-b711-e2df7c051f07 req-faaca531-6060-460d-b9f1-2fdd08140786 service nova] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Received event network-vif-deleted-48dfbf61-0142-4478-9ec1-585d5fbe11da {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:31:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:54 user nova-compute[72351]: INFO nova.compute.manager [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Took 2.47 seconds to detach 3 volumes for instance.
Aug 09 02:31:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:31:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:31:55 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:31:55 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:31:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.512s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:31:55 user nova-compute[72351]: INFO nova.scheduler.client.report [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Deleted allocations for instance 38a2b8e3-337d-416a-8a8c-da4921123db6
Aug 09 02:31:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-97aa2c73-7ee0-4c2f-b082-5922ebbee6fe tempest-TaggedBootDevicesTest_v242-942602465 tempest-TaggedBootDevicesTest_v242-942602465-project-member] Lock "38a2b8e3-337d-416a-8a8c-da4921123db6" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 11.750s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:31:56 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:58 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:31:59 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Acquiring lock "41b5e305-3973-4937-b262-d32db1091e20" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:31:59 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "41b5e305-3973-4937-b262-d32db1091e20" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:31:59 user nova-compute[72351]: DEBUG nova.compute.manager [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Starting instance... {{(pid=72351) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2406}}
Aug 09 02:31:59 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:31:59 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:31:59 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72351) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Aug 09 02:31:59 user nova-compute[72351]: INFO nova.compute.claims [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Claim successful on node user
Aug 09 02:31:59 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:31:59 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:31:59 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.512s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:31:59 user nova-compute[72351]: DEBUG nova.compute.manager [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Start building networks asynchronously for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2803}}
Aug 09 02:31:59 user nova-compute[72351]: DEBUG nova.compute.manager [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Allocating IP information in the background. {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1956}}
Aug 09 02:31:59 user nova-compute[72351]: DEBUG nova.network.neutron [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 41b5e305-3973-4937-b262-d32db1091e20] allocate_for_instance() {{(pid=72351) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Aug 09 02:31:59 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Aug 09 02:31:59 user nova-compute[72351]: DEBUG nova.compute.manager [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Start building block device mappings for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2838}}
Aug 09 02:32:00 user nova-compute[72351]: DEBUG nova.compute.manager [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Start spawning the instance on the hypervisor. {{(pid=72351) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2612}}
Aug 09 02:32:00 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Creating instance directory {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4717}}
Aug 09 02:32:00 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Creating image(s)
Aug 09 02:32:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Acquiring lock "/opt/stack/data/nova/instances/41b5e305-3973-4937-b262-d32db1091e20/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:32:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "/opt/stack/data/nova/instances/41b5e305-3973-4937-b262-d32db1091e20/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:32:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "/opt/stack/data/nova/instances/41b5e305-3973-4937-b262-d32db1091e20/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:32:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.207s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:32:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Acquiring lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:32:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:32:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:00 user nova-compute[72351]: DEBUG nova.policy [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '73d05c48be5640c28455e8ba8b05ef31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '321111f935dd4e8aab4a083206ea9a7e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72351) authorize /opt/stack/nova/nova/policy.py:203}}
Aug 09 02:32:00 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691548305.4247096, 38a2b8e3-337d-416a-8a8c-da4921123db6 => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:32:00 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] VM Stopped (Lifecycle Event)
Aug 09 02:32:00 user nova-compute[72351]: DEBUG nova.compute.manager [None req-e447a43f-1fd1-4d31-bf3b-56ba9efe11cb None None] [instance: 38a2b8e3-337d-416a-8a8c-da4921123db6] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:32:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.182s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:32:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/41b5e305-3973-4937-b262-d32db1091e20/disk 1073741824 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/41b5e305-3973-4937-b262-d32db1091e20/disk 1073741824" returned: 0 in 0.149s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:32:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.337s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:32:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.143s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:32:00 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Checking if we can resize image /opt/stack/data/nova/instances/41b5e305-3973-4937-b262-d32db1091e20/disk. size=1073741824 {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Aug 09 02:32:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/41b5e305-3973-4937-b262-d32db1091e20/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:00 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:32:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/41b5e305-3973-4937-b262-d32db1091e20/disk --force-share --output=json" returned: 0 in 0.160s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:32:00 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Cannot resize image /opt/stack/data/nova/instances/41b5e305-3973-4937-b262-d32db1091e20/disk to a smaller size. {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Aug 09 02:32:00 user nova-compute[72351]: DEBUG nova.objects.instance [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lazy-loading 'migration_context' on Instance uuid 41b5e305-3973-4937-b262-d32db1091e20 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:32:01 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Created local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4851}}
Aug 09 02:32:01 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Ensure instance console log exists: /opt/stack/data/nova/instances/41b5e305-3973-4937-b262-d32db1091e20/console.log {{(pid=72351) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4603}}
Aug 09 02:32:01 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:32:01 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:32:01 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:32:01 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:32:01 user nova-compute[72351]: DEBUG nova.compute.manager [req-b3cd94a5-62ba-46f5-bc29-7c011cf527d1 req-7ba6d76c-4055-48dc-bfd2-93f7e6c10dad service nova] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Received event network-changed-c4a56310-1841-46ae-9a97-0421861e62c0 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:32:01 user nova-compute[72351]: DEBUG nova.compute.manager [req-b3cd94a5-62ba-46f5-bc29-7c011cf527d1 req-7ba6d76c-4055-48dc-bfd2-93f7e6c10dad service nova] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Refreshing instance network info cache due to event network-changed-c4a56310-1841-46ae-9a97-0421861e62c0. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:32:01 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-b3cd94a5-62ba-46f5-bc29-7c011cf527d1 req-7ba6d76c-4055-48dc-bfd2-93f7e6c10dad service nova] Acquiring lock "refresh_cache-7b6f5d60-5f23-4ef1-b972-9ba7086f56cb" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:32:01 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-b3cd94a5-62ba-46f5-bc29-7c011cf527d1 req-7ba6d76c-4055-48dc-bfd2-93f7e6c10dad service nova] Acquired lock "refresh_cache-7b6f5d60-5f23-4ef1-b972-9ba7086f56cb" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:32:01 user nova-compute[72351]: DEBUG nova.network.neutron [req-b3cd94a5-62ba-46f5-bc29-7c011cf527d1 req-7ba6d76c-4055-48dc-bfd2-93f7e6c10dad service nova] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Refreshing network info cache for port c4a56310-1841-46ae-9a97-0421861e62c0 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:32:01 user nova-compute[72351]: DEBUG nova.network.neutron [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Successfully created port: e0032717-884b-4439-b7fc-42803aa7a802 {{(pid=72351) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Aug 09 02:32:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5b7bbcb0-ec99-4c06-aa93-d59a28ff91eb tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Acquiring lock "7b6f5d60-5f23-4ef1-b972-9ba7086f56cb" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:32:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5b7bbcb0-ec99-4c06-aa93-d59a28ff91eb tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "7b6f5d60-5f23-4ef1-b972-9ba7086f56cb" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:32:02 user nova-compute[72351]: DEBUG nova.objects.instance [None req-5b7bbcb0-ec99-4c06-aa93-d59a28ff91eb tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lazy-loading 'flavor' on Instance uuid 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:32:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5b7bbcb0-ec99-4c06-aa93-d59a28ff91eb tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "7b6f5d60-5f23-4ef1-b972-9ba7086f56cb" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.058s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:32:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5b7bbcb0-ec99-4c06-aa93-d59a28ff91eb tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Acquiring lock "7b6f5d60-5f23-4ef1-b972-9ba7086f56cb" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:32:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5b7bbcb0-ec99-4c06-aa93-d59a28ff91eb tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "7b6f5d60-5f23-4ef1-b972-9ba7086f56cb" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:32:02 user nova-compute[72351]: INFO nova.compute.manager [None req-5b7bbcb0-ec99-4c06-aa93-d59a28ff91eb tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Attaching volume 8ad3397f-79b8-4b2e-b253-fc2c09dafe22 to /dev/vdb
Aug 09 02:32:02 user nova-compute[72351]: DEBUG os_brick.utils [None req-5b7bbcb0-ec99-4c06-aa93-d59a28ff91eb tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:32:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:32:02 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ce032201-09c3-46a8-b6e2-14ebb8bc0617]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:a52269602ff4\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:32:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.016s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:32:02 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[3fdffec5-202c-4994-ae16-476298dc382f]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:32:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:32:02 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[2e767797-d998-4e05-8e82-030c326f71ea]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:32:02 user nova-compute[72351]: DEBUG nova.network.neutron [req-b3cd94a5-62ba-46f5-bc29-7c011cf527d1 req-7ba6d76c-4055-48dc-bfd2-93f7e6c10dad service nova] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Updated VIF entry in instance network info cache for port c4a56310-1841-46ae-9a97-0421861e62c0. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:32:02 user nova-compute[72351]: DEBUG nova.network.neutron [req-b3cd94a5-62ba-46f5-bc29-7c011cf527d1 req-7ba6d76c-4055-48dc-bfd2-93f7e6c10dad service nova] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Updating instance_info_cache with network_info: [{"id": "c4a56310-1841-46ae-9a97-0421861e62c0", "address": "fa:16:3e:75:cf:ba", "network": {"id": "49e8751c-e119-430f-9110-3dc6069b1589", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-899275135-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.125", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3aea1e66805146f7a8cd1d97e64a24af", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc4a56310-18", "ovs_interfaceid": "c4a56310-1841-46ae-9a97-0421861e62c0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:32:02 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[bbde8b1b-17ea-44c4-8fd3-c98ea83cc69c]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:32:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-5b7bbcb0-ec99-4c06-aa93-d59a28ff91eb tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Running cmd (subprocess): nvme version {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-5b7bbcb0-ec99-4c06-aa93-d59a28ff91eb tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] 'nvme version' failed. Not Retrying. {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:32:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-b3cd94a5-62ba-46f5-bc29-7c011cf527d1 req-7ba6d76c-4055-48dc-bfd2-93f7e6c10dad service nova] Releasing lock "refresh_cache-7b6f5d60-5f23-4ef1-b972-9ba7086f56cb" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:32:02 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.nvmeof [None req-5b7bbcb0-ec99-4c06-aa93-d59a28ff91eb tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] nvme not present on system {{(pid=72351) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Aug 09 02:32:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:02 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:32:02 user nova-compute[72351]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Aug 09 02:32:02 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[b7e2237d-25e3-4ed2-9a9e-a07d5705ec42]: (4, '') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:32:02 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-5b7bbcb0-ec99-4c06-aa93-d59a28ff91eb tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72351) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Aug 09 02:32:02 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-5b7bbcb0-ec99-4c06-aa93-d59a28ff91eb tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Aug 09 02:32:02 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-5b7bbcb0-ec99-4c06-aa93-d59a28ff91eb tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] LIGHTOS: no hostnqn found. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Aug 09 02:32:02 user nova-compute[72351]: DEBUG os_brick.utils [None req-5b7bbcb0-ec99-4c06-aa93-d59a28ff91eb tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] <== get_connector_properties: return (151ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:a52269602ff4', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:32:02 user nova-compute[72351]: DEBUG nova.virt.block_device [None req-5b7bbcb0-ec99-4c06-aa93-d59a28ff91eb tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Updating existing volume attachment record: 87abca94-1595-46dc-b0df-d0b2f4967d63 {{(pid=72351) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Aug 09 02:32:02 user nova-compute[72351]: DEBUG nova.network.neutron [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Successfully updated port: e0032717-884b-4439-b7fc-42803aa7a802 {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:32:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Acquiring lock "refresh_cache-41b5e305-3973-4937-b262-d32db1091e20" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:32:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Acquired lock "refresh_cache-41b5e305-3973-4937-b262-d32db1091e20" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:32:02 user nova-compute[72351]: DEBUG nova.network.neutron [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:32:02 user nova-compute[72351]: DEBUG nova.network.neutron [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:32:02 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:32:02 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72351) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10461}}
Aug 09 02:32:03 user nova-compute[72351]: DEBUG nova.network.neutron [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Updating instance_info_cache with network_info: [{"id": "e0032717-884b-4439-b7fc-42803aa7a802", "address": "fa:16:3e:49:57:14", "network": {"id": "9a452414-d61f-4045-a6cc-9be93c892c4c", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "321111f935dd4e8aab4a083206ea9a7e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape0032717-88", "ovs_interfaceid": "e0032717-884b-4439-b7fc-42803aa7a802", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:32:03 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Releasing lock "refresh_cache-41b5e305-3973-4937-b262-d32db1091e20" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:32:03 user nova-compute[72351]: DEBUG nova.compute.manager [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Instance network_info: |[{"id": "e0032717-884b-4439-b7fc-42803aa7a802", "address": "fa:16:3e:49:57:14", "network": {"id": "9a452414-d61f-4045-a6cc-9be93c892c4c", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "321111f935dd4e8aab4a083206ea9a7e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape0032717-88", "ovs_interfaceid": "e0032717-884b-4439-b7fc-42803aa7a802", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1971}}
Aug 09 02:32:03 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Start _get_guest_xml network_info=[{"id": "e0032717-884b-4439-b7fc-42803aa7a802", "address": "fa:16:3e:49:57:14", "network": {"id": "9a452414-d61f-4045-a6cc-9be93c892c4c", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "321111f935dd4e8aab4a083206ea9a7e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape0032717-88", "ovs_interfaceid": "e0032717-884b-4439-b7fc-42803aa7a802", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'size': 0, 'device_type': 'disk', 'encryption_secret_uuid': None, 'guest_format': None, 'encryption_options': None, 'disk_bus': 'virtio', 'encrypted': False, 'encryption_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'image_id': '18ccda7f-31d3-4978-ab9d-4ed2f74eff94'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:32:03 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:32:03 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:32:03 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:32:03 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:32:03 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:32:03 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:32:03 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:32:03 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:32:03 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:32:03 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:32:03 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:32:03 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:32:03 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:32:03 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:32:03 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:32:03 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:32:03 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:32:03 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:32:03 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:31:58Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-427830094',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-427830094',id=40,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='321111f935dd4e8aab4a083206ea9a7e',ramdisk_id='',reservation_id='r-iurnxcqs',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-2108329302',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:32:00Z,user_data=None,user_id='73d05c48be5640c28455e8ba8b05ef31',uuid=41b5e305-3973-4937-b262-d32db1091e20,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "e0032717-884b-4439-b7fc-42803aa7a802", "address": "fa:16:3e:49:57:14", "network": {"id": "9a452414-d61f-4045-a6cc-9be93c892c4c", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "321111f935dd4e8aab4a083206ea9a7e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape0032717-88", "ovs_interfaceid": "e0032717-884b-4439-b7fc-42803aa7a802", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:32:03 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Converting VIF {"id": "e0032717-884b-4439-b7fc-42803aa7a802", "address": "fa:16:3e:49:57:14", "network": {"id": "9a452414-d61f-4045-a6cc-9be93c892c4c", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "321111f935dd4e8aab4a083206ea9a7e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape0032717-88", "ovs_interfaceid": "e0032717-884b-4439-b7fc-42803aa7a802", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:32:03 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:49:57:14,bridge_name='br-int',has_traffic_filtering=True,id=e0032717-884b-4439-b7fc-42803aa7a802,network=Network(9a452414-d61f-4045-a6cc-9be93c892c4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape0032717-88') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:32:03 user nova-compute[72351]: DEBUG nova.objects.instance [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lazy-loading 'pci_devices' on Instance uuid 41b5e305-3973-4937-b262-d32db1091e20 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:32:03 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 41b5e305-3973-4937-b262-d32db1091e20] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:32:03 user nova-compute[72351]:   <uuid>41b5e305-3973-4937-b262-d32db1091e20</uuid>
Aug 09 02:32:03 user nova-compute[72351]:   <name>instance-00000028</name>
Aug 09 02:32:03 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:32:03 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:32:03 user nova-compute[72351]:   <metadata>
Aug 09 02:32:03 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:32:03 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:32:03 user nova-compute[72351]:       <nova:name>tempest-ServerBootFromVolumeStableRescueTest-server-427830094</nova:name>
Aug 09 02:32:03 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:32:03</nova:creationTime>
Aug 09 02:32:03 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:32:03 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:32:03 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:32:03 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:32:03 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:32:03 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:32:03 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:32:03 user nova-compute[72351]:       <nova:owner>
Aug 09 02:32:03 user nova-compute[72351]:         <nova:user uuid="73d05c48be5640c28455e8ba8b05ef31">tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member</nova:user>
Aug 09 02:32:03 user nova-compute[72351]:         <nova:project uuid="321111f935dd4e8aab4a083206ea9a7e">tempest-ServerBootFromVolumeStableRescueTest-2108329302</nova:project>
Aug 09 02:32:03 user nova-compute[72351]:       </nova:owner>
Aug 09 02:32:03 user nova-compute[72351]:       <nova:root type="image" uuid="18ccda7f-31d3-4978-ab9d-4ed2f74eff94"/>
Aug 09 02:32:03 user nova-compute[72351]:       <nova:ports>
Aug 09 02:32:03 user nova-compute[72351]:         <nova:port uuid="e0032717-884b-4439-b7fc-42803aa7a802">
Aug 09 02:32:03 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.9" ipVersion="4"/>
Aug 09 02:32:03 user nova-compute[72351]:         </nova:port>
Aug 09 02:32:03 user nova-compute[72351]:       </nova:ports>
Aug 09 02:32:03 user nova-compute[72351]:     </nova:instance>
Aug 09 02:32:03 user nova-compute[72351]:   </metadata>
Aug 09 02:32:03 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:32:03 user nova-compute[72351]:     <system>
Aug 09 02:32:03 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:32:03 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:32:03 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:32:03 user nova-compute[72351]:       <entry name="serial">41b5e305-3973-4937-b262-d32db1091e20</entry>
Aug 09 02:32:03 user nova-compute[72351]:       <entry name="uuid">41b5e305-3973-4937-b262-d32db1091e20</entry>
Aug 09 02:32:03 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:32:03 user nova-compute[72351]:     </system>
Aug 09 02:32:03 user nova-compute[72351]:   </sysinfo>
Aug 09 02:32:03 user nova-compute[72351]:   <os>
Aug 09 02:32:03 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:32:03 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:32:03 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:32:03 user nova-compute[72351]:   </os>
Aug 09 02:32:03 user nova-compute[72351]:   <features>
Aug 09 02:32:03 user nova-compute[72351]:     <acpi/>
Aug 09 02:32:03 user nova-compute[72351]:     <apic/>
Aug 09 02:32:03 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:32:03 user nova-compute[72351]:   </features>
Aug 09 02:32:03 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:32:03 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:32:03 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:32:03 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:32:03 user nova-compute[72351]:   </clock>
Aug 09 02:32:03 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:32:03 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:32:03 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:32:03 user nova-compute[72351]:   </cpu>
Aug 09 02:32:03 user nova-compute[72351]:   <devices>
Aug 09 02:32:03 user nova-compute[72351]:     <disk type="file" device="disk">
Aug 09 02:32:03 user nova-compute[72351]:       <driver name="qemu" type="qcow2" cache="none"/>
Aug 09 02:32:03 user nova-compute[72351]:       <source file="/opt/stack/data/nova/instances/41b5e305-3973-4937-b262-d32db1091e20/disk"/>
Aug 09 02:32:03 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:32:03 user nova-compute[72351]:     </disk>
Aug 09 02:32:03 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:32:03 user nova-compute[72351]:       <mac address="fa:16:3e:49:57:14"/>
Aug 09 02:32:03 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:32:03 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:32:03 user nova-compute[72351]:       <target dev="tape0032717-88"/>
Aug 09 02:32:03 user nova-compute[72351]:     </interface>
Aug 09 02:32:03 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:32:03 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/41b5e305-3973-4937-b262-d32db1091e20/console.log" append="off"/>
Aug 09 02:32:03 user nova-compute[72351]:     </serial>
Aug 09 02:32:03 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:32:03 user nova-compute[72351]:     <video>
Aug 09 02:32:03 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:32:03 user nova-compute[72351]:     </video>
Aug 09 02:32:03 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:32:03 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:32:03 user nova-compute[72351]:     </rng>
Aug 09 02:32:03 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:32:03 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:32:03 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:32:03 user nova-compute[72351]:     </memballoon>
Aug 09 02:32:03 user nova-compute[72351]:   </devices>
Aug 09 02:32:03 user nova-compute[72351]: </domain>
Aug 09 02:32:03 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:32:03 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:31:58Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-427830094',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-427830094',id=40,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='321111f935dd4e8aab4a083206ea9a7e',ramdisk_id='',reservation_id='r-iurnxcqs',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-2108329302',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:32:00Z,user_data=None,user_id='73d05c48be5640c28455e8ba8b05ef31',uuid=41b5e305-3973-4937-b262-d32db1091e20,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "e0032717-884b-4439-b7fc-42803aa7a802", "address": "fa:16:3e:49:57:14", "network": {"id": "9a452414-d61f-4045-a6cc-9be93c892c4c", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "321111f935dd4e8aab4a083206ea9a7e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape0032717-88", "ovs_interfaceid": "e0032717-884b-4439-b7fc-42803aa7a802", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:32:03 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Converting VIF {"id": "e0032717-884b-4439-b7fc-42803aa7a802", "address": "fa:16:3e:49:57:14", "network": {"id": "9a452414-d61f-4045-a6cc-9be93c892c4c", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "321111f935dd4e8aab4a083206ea9a7e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape0032717-88", "ovs_interfaceid": "e0032717-884b-4439-b7fc-42803aa7a802", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:32:03 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:49:57:14,bridge_name='br-int',has_traffic_filtering=True,id=e0032717-884b-4439-b7fc-42803aa7a802,network=Network(9a452414-d61f-4045-a6cc-9be93c892c4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape0032717-88') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:32:03 user nova-compute[72351]: DEBUG os_vif [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:49:57:14,bridge_name='br-int',has_traffic_filtering=True,id=e0032717-884b-4439-b7fc-42803aa7a802,network=Network(9a452414-d61f-4045-a6cc-9be93c892c4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape0032717-88') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:32:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:32:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:32:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:32:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:32:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tape0032717-88, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:32:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tape0032717-88, col_values=(('external_ids', {'iface-id': 'e0032717-884b-4439-b7fc-42803aa7a802', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:49:57:14', 'vm-uuid': '41b5e305-3973-4937-b262-d32db1091e20'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:32:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:32:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:32:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:32:03 user nova-compute[72351]: INFO os_vif [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:49:57:14,bridge_name='br-int',has_traffic_filtering=True,id=e0032717-884b-4439-b7fc-42803aa7a802,network=Network(9a452414-d61f-4045-a6cc-9be93c892c4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape0032717-88')
Aug 09 02:32:03 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:32:03 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] No VIF found with MAC fa:16:3e:49:57:14, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:32:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:32:03 user nova-compute[72351]: DEBUG nova.compute.manager [req-3fd826c7-2167-4848-aaa3-6151586cd436 req-35e62983-8fc2-43c7-9165-bdc453a38cfc service nova] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Received event network-changed-e0032717-884b-4439-b7fc-42803aa7a802 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:32:03 user nova-compute[72351]: DEBUG nova.compute.manager [req-3fd826c7-2167-4848-aaa3-6151586cd436 req-35e62983-8fc2-43c7-9165-bdc453a38cfc service nova] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Refreshing instance network info cache due to event network-changed-e0032717-884b-4439-b7fc-42803aa7a802. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:32:03 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-3fd826c7-2167-4848-aaa3-6151586cd436 req-35e62983-8fc2-43c7-9165-bdc453a38cfc service nova] Acquiring lock "refresh_cache-41b5e305-3973-4937-b262-d32db1091e20" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:32:03 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-3fd826c7-2167-4848-aaa3-6151586cd436 req-35e62983-8fc2-43c7-9165-bdc453a38cfc service nova] Acquired lock "refresh_cache-41b5e305-3973-4937-b262-d32db1091e20" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:32:03 user nova-compute[72351]: DEBUG nova.network.neutron [req-3fd826c7-2167-4848-aaa3-6151586cd436 req-35e62983-8fc2-43c7-9165-bdc453a38cfc service nova] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Refreshing network info cache for port e0032717-884b-4439-b7fc-42803aa7a802 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:32:04 user nova-compute[72351]: DEBUG nova.network.neutron [req-3fd826c7-2167-4848-aaa3-6151586cd436 req-35e62983-8fc2-43c7-9165-bdc453a38cfc service nova] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Updated VIF entry in instance network info cache for port e0032717-884b-4439-b7fc-42803aa7a802. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:32:04 user nova-compute[72351]: DEBUG nova.network.neutron [req-3fd826c7-2167-4848-aaa3-6151586cd436 req-35e62983-8fc2-43c7-9165-bdc453a38cfc service nova] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Updating instance_info_cache with network_info: [{"id": "e0032717-884b-4439-b7fc-42803aa7a802", "address": "fa:16:3e:49:57:14", "network": {"id": "9a452414-d61f-4045-a6cc-9be93c892c4c", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "321111f935dd4e8aab4a083206ea9a7e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape0032717-88", "ovs_interfaceid": "e0032717-884b-4439-b7fc-42803aa7a802", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:32:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-3fd826c7-2167-4848-aaa3-6151586cd436 req-35e62983-8fc2-43c7-9165-bdc453a38cfc service nova] Releasing lock "refresh_cache-41b5e305-3973-4937-b262-d32db1091e20" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:32:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:32:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:32:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:32:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:32:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:32:05 user nova-compute[72351]: DEBUG nova.compute.manager [req-980be7c7-8705-4d5a-bcb7-cdf0869b57d8 req-00f369c0-5602-4966-9076-dab301845d9b service nova] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Received event network-vif-plugged-e0032717-884b-4439-b7fc-42803aa7a802 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:32:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-980be7c7-8705-4d5a-bcb7-cdf0869b57d8 req-00f369c0-5602-4966-9076-dab301845d9b service nova] Acquiring lock "41b5e305-3973-4937-b262-d32db1091e20-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:32:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-980be7c7-8705-4d5a-bcb7-cdf0869b57d8 req-00f369c0-5602-4966-9076-dab301845d9b service nova] Lock "41b5e305-3973-4937-b262-d32db1091e20-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:32:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-980be7c7-8705-4d5a-bcb7-cdf0869b57d8 req-00f369c0-5602-4966-9076-dab301845d9b service nova] Lock "41b5e305-3973-4937-b262-d32db1091e20-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:32:05 user nova-compute[72351]: DEBUG nova.compute.manager [req-980be7c7-8705-4d5a-bcb7-cdf0869b57d8 req-00f369c0-5602-4966-9076-dab301845d9b service nova] [instance: 41b5e305-3973-4937-b262-d32db1091e20] No waiting events found dispatching network-vif-plugged-e0032717-884b-4439-b7fc-42803aa7a802 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:32:05 user nova-compute[72351]: WARNING nova.compute.manager [req-980be7c7-8705-4d5a-bcb7-cdf0869b57d8 req-00f369c0-5602-4966-9076-dab301845d9b service nova] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Received unexpected event network-vif-plugged-e0032717-884b-4439-b7fc-42803aa7a802 for instance with vm_state building and task_state spawning.
Aug 09 02:32:05 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:32:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:32:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:32:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:32:05 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Auditing locally available compute resources for user (node: user) {{(pid=72351) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Aug 09 02:32:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json" returned: 0 in 0.143s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:32:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json" returned: 0 in 0.133s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:32:06 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdg (vda) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:32:06 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdh (vda) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:32:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/47afce95-e5f7-4703-a99f-7ef8b28c4bd2/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/47afce95-e5f7-4703-a99f-7ef8b28c4bd2/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:32:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/47afce95-e5f7-4703-a99f-7ef8b28c4bd2/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/47afce95-e5f7-4703-a99f-7ef8b28c4bd2/disk --force-share --output=json" returned: 0 in 0.142s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:32:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.rescue --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.rescue --force-share --output=json" returned: 0 in 0.134s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:32:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.rescue --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.rescue --force-share --output=json" returned: 0 in 0.141s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:32:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:07 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548327.02756, 41b5e305-3973-4937-b262-d32db1091e20 => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:32:07 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 41b5e305-3973-4937-b262-d32db1091e20] VM Resumed (Lifecycle Event)
Aug 09 02:32:07 user nova-compute[72351]: DEBUG nova.compute.manager [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:32:07 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Guest created on hypervisor {{(pid=72351) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4411}}
Aug 09 02:32:07 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Instance spawned successfully.
Aug 09 02:32:07 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:894}}
Aug 09 02:32:07 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:32:07 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:32:07 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Found default for hw_cdrom_bus of ide {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:32:07 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Found default for hw_disk_bus of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:32:07 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Found default for hw_input_bus of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:32:07 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Found default for hw_pointer_model of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:32:07 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Found default for hw_video_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:32:07 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Found default for hw_vif_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:32:07 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 41b5e305-3973-4937-b262-d32db1091e20] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:32:07 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548327.0286984, 41b5e305-3973-4937-b262-d32db1091e20 => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:32:07 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 41b5e305-3973-4937-b262-d32db1091e20] VM Started (Lifecycle Event)
Aug 09 02:32:07 user nova-compute[72351]: DEBUG nova.compute.manager [req-dabeebb3-4a7b-4f36-aa1c-39023e970e00 req-2c63fcf8-d8b3-41ed-ad62-71d9d56917bc service nova] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Received event network-vif-plugged-e0032717-884b-4439-b7fc-42803aa7a802 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:32:07 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-dabeebb3-4a7b-4f36-aa1c-39023e970e00 req-2c63fcf8-d8b3-41ed-ad62-71d9d56917bc service nova] Acquiring lock "41b5e305-3973-4937-b262-d32db1091e20-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:32:07 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-dabeebb3-4a7b-4f36-aa1c-39023e970e00 req-2c63fcf8-d8b3-41ed-ad62-71d9d56917bc service nova] Lock "41b5e305-3973-4937-b262-d32db1091e20-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:32:07 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-dabeebb3-4a7b-4f36-aa1c-39023e970e00 req-2c63fcf8-d8b3-41ed-ad62-71d9d56917bc service nova] Lock "41b5e305-3973-4937-b262-d32db1091e20-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:32:07 user nova-compute[72351]: DEBUG nova.compute.manager [req-dabeebb3-4a7b-4f36-aa1c-39023e970e00 req-2c63fcf8-d8b3-41ed-ad62-71d9d56917bc service nova] [instance: 41b5e305-3973-4937-b262-d32db1091e20] No waiting events found dispatching network-vif-plugged-e0032717-884b-4439-b7fc-42803aa7a802 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:32:07 user nova-compute[72351]: WARNING nova.compute.manager [req-dabeebb3-4a7b-4f36-aa1c-39023e970e00 req-2c63fcf8-d8b3-41ed-ad62-71d9d56917bc service nova] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Received unexpected event network-vif-plugged-e0032717-884b-4439-b7fc-42803aa7a802 for instance with vm_state building and task_state spawning.
Aug 09 02:32:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json" returned: 0 in 0.160s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:32:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:07 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:32:07 user nova-compute[72351]: INFO nova.compute.manager [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Took 7.10 seconds to spawn the instance on the hypervisor.
Aug 09 02:32:07 user nova-compute[72351]: DEBUG nova.compute.manager [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:32:07 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:32:07 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 41b5e305-3973-4937-b262-d32db1091e20] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:32:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json" returned: 0 in 0.147s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:32:07 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sde (vda) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:32:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:07 user nova-compute[72351]: INFO nova.compute.manager [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Took 8.07 seconds to build instance.
Aug 09 02:32:07 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b16f19c3-3a4b-45ee-a077-a7955b54c9b6 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "41b5e305-3973-4937-b262-d32db1091e20" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 8.175s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:32:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json" returned: 0 in 0.143s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:32:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json" returned: 0 in 0.145s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:32:07 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdf (vda) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:32:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/41b5e305-3973-4937-b262-d32db1091e20/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/41b5e305-3973-4937-b262-d32db1091e20/disk --force-share --output=json" returned: 0 in 0.147s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:32:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/41b5e305-3973-4937-b262-d32db1091e20/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/41b5e305-3973-4937-b262-d32db1091e20/disk --force-share --output=json" returned: 0 in 0.164s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:32:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json" returned: 0 in 0.164s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:32:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json" returned: 0 in 0.155s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:32:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:32:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7b6f5d60-5f23-4ef1-b972-9ba7086f56cb/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:32:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7b6f5d60-5f23-4ef1-b972-9ba7086f56cb/disk --force-share --output=json" returned: 0 in 0.161s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:32:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7b6f5d60-5f23-4ef1-b972-9ba7086f56cb/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7b6f5d60-5f23-4ef1-b972-9ba7086f56cb/disk --force-share --output=json" returned: 0 in 0.153s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:32:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7bef74f4-3bb3-4ad0-bff7-34156be31249/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7bef74f4-3bb3-4ad0-bff7-34156be31249/disk --force-share --output=json" returned: 0 in 0.167s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:32:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7bef74f4-3bb3-4ad0-bff7-34156be31249/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7bef74f4-3bb3-4ad0-bff7-34156be31249/disk --force-share --output=json" returned: 0 in 0.158s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:32:09 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a03fbded-23ad-440b-97e7-bea07425e873/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:09 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a03fbded-23ad-440b-97e7-bea07425e873/disk --force-share --output=json" returned: 0 in 0.200s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:32:09 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a03fbded-23ad-440b-97e7-bea07425e873/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:09 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a03fbded-23ad-440b-97e7-bea07425e873/disk --force-share --output=json" returned: 0 in 0.164s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:32:09 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f37b7fb7-1b94-421b-be79-7066010efabf/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:09 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f37b7fb7-1b94-421b-be79-7066010efabf/disk --force-share --output=json" returned: 0 in 0.158s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:32:09 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f37b7fb7-1b94-421b-be79-7066010efabf/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:09 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f37b7fb7-1b94-421b-be79-7066010efabf/disk --force-share --output=json" returned: 0 in 0.158s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:32:10 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:32:10 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:32:10 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Hypervisor/Node resource view: name=user free_ram=6770MB free_disk=25.101844787597656GB free_vcpus=-2 pci_devices=[{"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72351) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Aug 09 02:32:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:32:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:32:10 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:32:10 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance eb86cb49-2018-42c2-bd97-c662b829775d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:32:10 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:32:10 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance a7f3d076-864b-4b68-9207-214ce2c21444 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:32:10 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 47afce95-e5f7-4703-a99f-7ef8b28c4bd2 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:32:10 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 957c1510-1320-4492-9052-a58ff540319d actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:32:10 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance a03fbded-23ad-440b-97e7-bea07425e873 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:32:10 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:32:10 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 49a10075-0d3c-4cd6-bb43-10238cad3f37 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:32:10 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance f37b7fb7-1b94-421b-be79-7066010efabf actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:32:10 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 8e970bdd-54a1-4058-b09d-784cb6b5b1a4 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:32:10 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 80e87454-90b9-4168-bf00-b7d93b245526 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:32:10 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 7bef74f4-3bb3-4ad0-bff7-34156be31249 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:32:10 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 41b5e305-3973-4937-b262-d32db1091e20 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:32:10 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Total usable vcpus: 12, total allocated vcpus: 14 {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Aug 09 02:32:10 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Final resource view: name=user phys_ram=16011MB used_ram=7680MB phys_disk=40GB used_disk=10GB total_vcpus=12 used_vcpus=14 pci_stats=[] {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Aug 09 02:32:11 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:32:11 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:32:11 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Compute_service record updated for user:user {{(pid=72351) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Aug 09 02:32:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.875s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:32:12 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:32:12 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:32:12 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Starting heal instance info cache {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9842}}
Aug 09 02:32:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "refresh_cache-585c81a5-ac24-4d12-a8a6-1e7b12a2d53a" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:32:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquired lock "refresh_cache-585c81a5-ac24-4d12-a8a6-1e7b12a2d53a" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:32:12 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Forcefully refreshing network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1996}}
Aug 09 02:32:13 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Updating instance_info_cache with network_info: [{"id": "e521fae4-093d-4c92-a32d-684c1a5137f5", "address": "fa:16:3e:e5:b0:10", "network": {"id": "b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-210689817-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.41", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "69d584c230b94ff8aae722eb2f364c5c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape521fae4-09", "ovs_interfaceid": "e521fae4-093d-4c92-a32d-684c1a5137f5", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:32:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Releasing lock "refresh_cache-585c81a5-ac24-4d12-a8a6-1e7b12a2d53a" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:32:13 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Updated the network info_cache for instance {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9913}}
Aug 09 02:32:13 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:32:13 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:32:13 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:32:13 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:32:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:32:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:32:18 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-5b7bbcb0-ec99-4c06-aa93-d59a28ff91eb tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 02:32:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5b7bbcb0-ec99-4c06-aa93-d59a28ff91eb tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'vlwMEpQJ', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:32:18 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-5b7bbcb0-ec99-4c06-aa93-d59a28ff91eb tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:32:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-5b7bbcb0-ec99-4c06-aa93-d59a28ff91eb tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:32:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-5b7bbcb0-ec99-4c06-aa93-d59a28ff91eb tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:32:18 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-5b7bbcb0-ec99-4c06-aa93-d59a28ff91eb tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:32:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-5b7bbcb0-ec99-4c06-aa93-d59a28ff91eb tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:32:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-5b7bbcb0-ec99-4c06-aa93-d59a28ff91eb tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:32:18 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-5b7bbcb0-ec99-4c06-aa93-d59a28ff91eb tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 02:32:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 -p 172.16.0.220:3260" returned: 21 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:32:18 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[f35b1ec2-7aca-4323-908c-3dc93832b018]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:32:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5b7bbcb0-ec99-4c06-aa93-d59a28ff91eb tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Aug 09 02:32:18 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:32:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 -p 172.16.0.220:3260 --interface default --op new {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:32:18 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[28fe4da4-e70c-49c3-a417-88598d1f4cad]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22] added\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:32:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5b7bbcb0-ec99-4c06-aa93-d59a28ff91eb tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22] added
Aug 09 02:32:18 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:32:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:32:18 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[6897d08b-aa09-4adb-87af-7e85bfbb3a20]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:32:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5b7bbcb0-ec99-4c06-aa93-d59a28ff91eb tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:32:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:32:18 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[381c57ae-9179-4224-8ebb-922ed424cabe]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:32:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5b7bbcb0-ec99-4c06-aa93-d59a28ff91eb tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:32:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v vlwMEpQJ {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v vlwMEpQJ" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:32:18 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[3ca53511-fa4c-4ec4-bce8-a1cf4ddde62c]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:32:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5b7bbcb0-ec99-4c06-aa93-d59a28ff91eb tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'vlwMEpQJ'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:32:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:32:18 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[cb2cd0e8-b585-46ec-b525-245afe691428]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:32:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5b7bbcb0-ec99-4c06-aa93-d59a28ff91eb tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:32:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.026s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:32:18 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[078c5473-78f5-45d3-83a9-d72cad9e69b4]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)\ntcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)\ntcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:32:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5b7bbcb0-ec99-4c06-aa93-d59a28ff91eb tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:32:18 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:32:18 user nova-compute[72351]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)
Aug 09 02:32:18 user nova-compute[72351]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:32:18 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:32:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5b7bbcb0-ec99-4c06-aa93-d59a28ff91eb tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:32:18 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:32:18 user nova-compute[72351]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)
Aug 09 02:32:18 user nova-compute[72351]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:32:18 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:32:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 -p 172.16.0.220:3260 --login {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 -p 172.16.0.220:3260 --login" returned: 0 in 0.079s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:32:18 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[7ee331ed-686b-4d45-8a1d-b528cb3c5d16]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:32:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5b7bbcb0-ec99-4c06-aa93-d59a28ff91eb tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22, portal: 172.16.0.220,3260]
Aug 09 02:32:18 user nova-compute[72351]: Login to [iface: default, target: iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22, portal: 172.16.0.220,3260] successful.
Aug 09 02:32:18 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:32:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:32:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.024s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:32:18 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[3b8e8a3b-b6ad-462f-9c73-f077bae02f4b]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:32:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5b7bbcb0-ec99-4c06-aa93-d59a28ff91eb tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:32:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.029s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:32:18 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[993b9534-e00a-459b-adec-29267a796b37]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)\ntcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)\ntcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:32:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5b7bbcb0-ec99-4c06-aa93-d59a28ff91eb tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:32:18 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:32:18 user nova-compute[72351]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)
Aug 09 02:32:18 user nova-compute[72351]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:32:18 user nova-compute[72351]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)
Aug 09 02:32:18 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:32:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5b7bbcb0-ec99-4c06-aa93-d59a28ff91eb tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:32:18 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:32:18 user nova-compute[72351]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)
Aug 09 02:32:18 user nova-compute[72351]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:32:18 user nova-compute[72351]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)
Aug 09 02:32:18 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:32:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-5b7bbcb0-ec99-4c06-aa93-d59a28ff91eb tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.323s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:32:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5b7bbcb0-ec99-4c06-aa93-d59a28ff91eb tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 02:32:18 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-5b7bbcb0-ec99-4c06-aa93-d59a28ff91eb tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] HCTL ('33', '-', '-', 0) found on session 40 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 02:32:18 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-5b7bbcb0-ec99-4c06-aa93-d59a28ff91eb tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 02:32:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:32:18 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[22db40aa-06f2-4f9e-b19f-9988b8411c05]: (4, ('- - 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:32:18 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-5b7bbcb0-ec99-4c06-aa93-d59a28ff91eb tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Searching for a device in session 40 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:32:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5b7bbcb0-ec99-4c06-aa93-d59a28ff91eb tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Connected to sdb using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'vlwMEpQJ', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 02:32:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:32:19 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-5b7bbcb0-ec99-4c06-aa93-d59a28ff91eb tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.372s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:32:19 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5b7bbcb0-ec99-4c06-aa93-d59a28ff91eb tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] <== connect_volume: return (1375ms) {'type': 'block', 'scsi_wwn': '23964653262616131', 'path': '/dev/sdb'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:32:19 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-5b7bbcb0-ec99-4c06-aa93-d59a28ff91eb tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23964653262616131', 'path': '/dev/sdb'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 02:32:19 user nova-compute[72351]: DEBUG nova.objects.instance [None req-5b7bbcb0-ec99-4c06-aa93-d59a28ff91eb tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lazy-loading 'flavor' on Instance uuid 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:32:19 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-5b7bbcb0-ec99-4c06-aa93-d59a28ff91eb tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] attach device xml: <disk type="block" device="disk">
Aug 09 02:32:19 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:32:19 user nova-compute[72351]:   <source dev="/dev/sdb"/>
Aug 09 02:32:19 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:32:19 user nova-compute[72351]:   <serial>8ad3397f-79b8-4b2e-b253-fc2c09dafe22</serial>
Aug 09 02:32:19 user nova-compute[72351]: </disk>
Aug 09 02:32:19 user nova-compute[72351]:  {{(pid=72351) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Aug 09 02:32:19 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-5b7bbcb0-ec99-4c06-aa93-d59a28ff91eb tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:32:19 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-5b7bbcb0-ec99-4c06-aa93-d59a28ff91eb tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:32:19 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-5b7bbcb0-ec99-4c06-aa93-d59a28ff91eb tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] No VIF found with MAC fa:16:3e:75:cf:ba, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:32:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5b7bbcb0-ec99-4c06-aa93-d59a28ff91eb tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "7b6f5d60-5f23-4ef1-b972-9ba7086f56cb" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 17.764s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:32:21 user nova-compute[72351]: INFO nova.compute.manager [None req-e1d79873-9f6b-4543-ab5f-745286aa3d0a tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Rescuing
Aug 09 02:32:21 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e1d79873-9f6b-4543-ab5f-745286aa3d0a tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Acquiring lock "refresh_cache-7b6f5d60-5f23-4ef1-b972-9ba7086f56cb" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:32:21 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e1d79873-9f6b-4543-ab5f-745286aa3d0a tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Acquired lock "refresh_cache-7b6f5d60-5f23-4ef1-b972-9ba7086f56cb" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:32:21 user nova-compute[72351]: DEBUG nova.network.neutron [None req-e1d79873-9f6b-4543-ab5f-745286aa3d0a tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:32:21 user nova-compute[72351]: DEBUG nova.network.neutron [None req-e1d79873-9f6b-4543-ab5f-745286aa3d0a tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Updating instance_info_cache with network_info: [{"id": "c4a56310-1841-46ae-9a97-0421861e62c0", "address": "fa:16:3e:75:cf:ba", "network": {"id": "49e8751c-e119-430f-9110-3dc6069b1589", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-899275135-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.125", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3aea1e66805146f7a8cd1d97e64a24af", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc4a56310-18", "ovs_interfaceid": "c4a56310-1841-46ae-9a97-0421861e62c0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:32:21 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e1d79873-9f6b-4543-ab5f-745286aa3d0a tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Releasing lock "refresh_cache-7b6f5d60-5f23-4ef1-b972-9ba7086f56cb" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:32:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:32:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:32:22 user nova-compute[72351]: DEBUG nova.compute.manager [req-ff2947ec-6d6a-47da-a13a-87fa9076f3ee req-e64a5362-3e84-4e3e-8a4a-9d3f4ecd296c service nova] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Received event network-vif-unplugged-c4a56310-1841-46ae-9a97-0421861e62c0 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:32:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ff2947ec-6d6a-47da-a13a-87fa9076f3ee req-e64a5362-3e84-4e3e-8a4a-9d3f4ecd296c service nova] Acquiring lock "7b6f5d60-5f23-4ef1-b972-9ba7086f56cb-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:32:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ff2947ec-6d6a-47da-a13a-87fa9076f3ee req-e64a5362-3e84-4e3e-8a4a-9d3f4ecd296c service nova] Lock "7b6f5d60-5f23-4ef1-b972-9ba7086f56cb-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:32:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ff2947ec-6d6a-47da-a13a-87fa9076f3ee req-e64a5362-3e84-4e3e-8a4a-9d3f4ecd296c service nova] Lock "7b6f5d60-5f23-4ef1-b972-9ba7086f56cb-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:32:22 user nova-compute[72351]: DEBUG nova.compute.manager [req-ff2947ec-6d6a-47da-a13a-87fa9076f3ee req-e64a5362-3e84-4e3e-8a4a-9d3f4ecd296c service nova] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] No waiting events found dispatching network-vif-unplugged-c4a56310-1841-46ae-9a97-0421861e62c0 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:32:22 user nova-compute[72351]: WARNING nova.compute.manager [req-ff2947ec-6d6a-47da-a13a-87fa9076f3ee req-e64a5362-3e84-4e3e-8a4a-9d3f4ecd296c service nova] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Received unexpected event network-vif-unplugged-c4a56310-1841-46ae-9a97-0421861e62c0 for instance with vm_state active and task_state rescuing.
Aug 09 02:32:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:32:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:32:22 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Instance destroyed successfully.
Aug 09 02:32:22 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-e1d79873-9f6b-4543-ab5f-745286aa3d0a tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Attempting rescue
Aug 09 02:32:22 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-e1d79873-9f6b-4543-ab5f-745286aa3d0a tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] rescue generated disk_info: {'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'disk.rescue': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} {{(pid=72351) rescue /opt/stack/nova/nova/virt/libvirt/driver.py:4308}}
Aug 09 02:32:22 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-e1d79873-9f6b-4543-ab5f-745286aa3d0a tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Instance directory exists: not creating {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4713}}
Aug 09 02:32:22 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-e1d79873-9f6b-4543-ab5f-745286aa3d0a tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Creating image(s)
Aug 09 02:32:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e1d79873-9f6b-4543-ab5f-745286aa3d0a tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Acquiring lock "/opt/stack/data/nova/instances/7b6f5d60-5f23-4ef1-b972-9ba7086f56cb/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:32:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e1d79873-9f6b-4543-ab5f-745286aa3d0a tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "/opt/stack/data/nova/instances/7b6f5d60-5f23-4ef1-b972-9ba7086f56cb/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:32:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e1d79873-9f6b-4543-ab5f-745286aa3d0a tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "/opt/stack/data/nova/instances/7b6f5d60-5f23-4ef1-b972-9ba7086f56cb/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.004s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:32:22 user nova-compute[72351]: DEBUG nova.objects.instance [None req-e1d79873-9f6b-4543-ab5f-745286aa3d0a tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lazy-loading 'trusted_certs' on Instance uuid 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:32:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e1d79873-9f6b-4543-ab5f-745286aa3d0a tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Acquiring lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:32:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e1d79873-9f6b-4543-ab5f-745286aa3d0a tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:32:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-e1d79873-9f6b-4543-ab5f-745286aa3d0a tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-e1d79873-9f6b-4543-ab5f-745286aa3d0a tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.157s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:32:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-e1d79873-9f6b-4543-ab5f-745286aa3d0a tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/7b6f5d60-5f23-4ef1-b972-9ba7086f56cb/disk.rescue {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-e1d79873-9f6b-4543-ab5f-745286aa3d0a tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/7b6f5d60-5f23-4ef1-b972-9ba7086f56cb/disk.rescue" returned: 0 in 0.058s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:32:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e1d79873-9f6b-4543-ab5f-745286aa3d0a tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.222s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:32:22 user nova-compute[72351]: DEBUG nova.objects.instance [None req-e1d79873-9f6b-4543-ab5f-745286aa3d0a tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lazy-loading 'migration_context' on Instance uuid 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:32:22 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-e1d79873-9f6b-4543-ab5f-745286aa3d0a tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Created local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4851}}
Aug 09 02:32:22 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-e1d79873-9f6b-4543-ab5f-745286aa3d0a tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Start _get_guest_xml network_info=[{"id": "c4a56310-1841-46ae-9a97-0421861e62c0", "address": "fa:16:3e:75:cf:ba", "network": {"id": "49e8751c-e119-430f-9110-3dc6069b1589", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-899275135-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.125", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerRescueNegativeTestJSON-899275135-network", "vif_mac": "fa:16:3e:75:cf:ba"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3aea1e66805146f7a8cd1d97e64a24af", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc4a56310-18", "ovs_interfaceid": "c4a56310-1841-46ae-9a97-0421861e62c0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'disk.rescue': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>) rescue={'image_id': '18ccda7f-31d3-4978-ab9d-4ed2f74eff94', 'kernel_id': '', 'ramdisk_id': ''} block_device_info=None {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:32:22 user nova-compute[72351]: DEBUG nova.objects.instance [None req-e1d79873-9f6b-4543-ab5f-745286aa3d0a tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lazy-loading 'resources' on Instance uuid 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:32:22 user nova-compute[72351]: DEBUG nova.objects.instance [None req-e1d79873-9f6b-4543-ab5f-745286aa3d0a tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lazy-loading 'numa_topology' on Instance uuid 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:32:22 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-e1d79873-9f6b-4543-ab5f-745286aa3d0a tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:32:22 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-e1d79873-9f6b-4543-ab5f-745286aa3d0a tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:32:22 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-e1d79873-9f6b-4543-ab5f-745286aa3d0a tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:32:22 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-e1d79873-9f6b-4543-ab5f-745286aa3d0a tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:32:22 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-e1d79873-9f6b-4543-ab5f-745286aa3d0a tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:32:22 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-e1d79873-9f6b-4543-ab5f-745286aa3d0a tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:32:22 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-e1d79873-9f6b-4543-ab5f-745286aa3d0a tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:32:22 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-e1d79873-9f6b-4543-ab5f-745286aa3d0a tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:32:22 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-e1d79873-9f6b-4543-ab5f-745286aa3d0a tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:32:22 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-e1d79873-9f6b-4543-ab5f-745286aa3d0a tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:32:22 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-e1d79873-9f6b-4543-ab5f-745286aa3d0a tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:32:22 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-e1d79873-9f6b-4543-ab5f-745286aa3d0a tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:32:22 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-e1d79873-9f6b-4543-ab5f-745286aa3d0a tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:32:22 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-e1d79873-9f6b-4543-ab5f-745286aa3d0a tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:32:22 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-e1d79873-9f6b-4543-ab5f-745286aa3d0a tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:32:22 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-e1d79873-9f6b-4543-ab5f-745286aa3d0a tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:32:22 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-e1d79873-9f6b-4543-ab5f-745286aa3d0a tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:32:22 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-e1d79873-9f6b-4543-ab5f-745286aa3d0a tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:32:22 user nova-compute[72351]: DEBUG nova.objects.instance [None req-e1d79873-9f6b-4543-ab5f-745286aa3d0a tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lazy-loading 'vcpu_model' on Instance uuid 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:32:22 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-e1d79873-9f6b-4543-ab5f-745286aa3d0a tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:30:09Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerRescueNegativeTestJSON-server-2047297087',display_name='tempest-ServerRescueNegativeTestJSON-server-2047297087',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serverrescuenegativetestjson-server-2047297087',id=34,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBPLsNIutXj2fb507aP4WngpUN1jl7pet8vFZkz28Sl/BaBQhyC41kBOXIZAx0v3md9QJVdNaIzkTSV1yroT5SZJzTuTaLLebnfImCGnkD5mtQK+8liTnNCRPuxSfIElfWA==',key_name='tempest-keypair-694850304',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:30:17Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='3aea1e66805146f7a8cd1d97e64a24af',ramdisk_id='',reservation_id='r-02psy90j',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerRescueNegativeTestJSON-289364438',owner_user_name='tempest-ServerRescueNegativeTestJSON-289364438-project-member'},tags=<?>,task_state='rescuing',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:30:18Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='419b28d5a46c481db435715a30ab2614',uuid=7b6f5d60-5f23-4ef1-b972-9ba7086f56cb,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "c4a56310-1841-46ae-9a97-0421861e62c0", "address": "fa:16:3e:75:cf:ba", "network": {"id": "49e8751c-e119-430f-9110-3dc6069b1589", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-899275135-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.125", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerRescueNegativeTestJSON-899275135-network", "vif_mac": "fa:16:3e:75:cf:ba"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3aea1e66805146f7a8cd1d97e64a24af", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc4a56310-18", "ovs_interfaceid": "c4a56310-1841-46ae-9a97-0421861e62c0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:32:22 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-e1d79873-9f6b-4543-ab5f-745286aa3d0a tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Converting VIF {"id": "c4a56310-1841-46ae-9a97-0421861e62c0", "address": "fa:16:3e:75:cf:ba", "network": {"id": "49e8751c-e119-430f-9110-3dc6069b1589", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-899275135-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.125", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerRescueNegativeTestJSON-899275135-network", "vif_mac": "fa:16:3e:75:cf:ba"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3aea1e66805146f7a8cd1d97e64a24af", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc4a56310-18", "ovs_interfaceid": "c4a56310-1841-46ae-9a97-0421861e62c0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:32:22 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-e1d79873-9f6b-4543-ab5f-745286aa3d0a tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:75:cf:ba,bridge_name='br-int',has_traffic_filtering=True,id=c4a56310-1841-46ae-9a97-0421861e62c0,network=Network(49e8751c-e119-430f-9110-3dc6069b1589),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc4a56310-18') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:32:22 user nova-compute[72351]: DEBUG nova.objects.instance [None req-e1d79873-9f6b-4543-ab5f-745286aa3d0a tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lazy-loading 'pci_devices' on Instance uuid 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:32:22 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-e1d79873-9f6b-4543-ab5f-745286aa3d0a tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:32:22 user nova-compute[72351]:   <uuid>7b6f5d60-5f23-4ef1-b972-9ba7086f56cb</uuid>
Aug 09 02:32:22 user nova-compute[72351]:   <name>instance-00000022</name>
Aug 09 02:32:22 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:32:22 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:32:22 user nova-compute[72351]:   <metadata>
Aug 09 02:32:22 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:32:22 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:32:22 user nova-compute[72351]:       <nova:name>tempest-ServerRescueNegativeTestJSON-server-2047297087</nova:name>
Aug 09 02:32:22 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:32:22</nova:creationTime>
Aug 09 02:32:22 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:32:22 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:32:22 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:32:22 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:32:22 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:32:22 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:32:22 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:32:22 user nova-compute[72351]:       <nova:owner>
Aug 09 02:32:22 user nova-compute[72351]:         <nova:user uuid="419b28d5a46c481db435715a30ab2614">tempest-ServerRescueNegativeTestJSON-289364438-project-member</nova:user>
Aug 09 02:32:22 user nova-compute[72351]:         <nova:project uuid="3aea1e66805146f7a8cd1d97e64a24af">tempest-ServerRescueNegativeTestJSON-289364438</nova:project>
Aug 09 02:32:22 user nova-compute[72351]:       </nova:owner>
Aug 09 02:32:22 user nova-compute[72351]:       <nova:root type="image" uuid="18ccda7f-31d3-4978-ab9d-4ed2f74eff94"/>
Aug 09 02:32:22 user nova-compute[72351]:       <nova:ports>
Aug 09 02:32:22 user nova-compute[72351]:         <nova:port uuid="c4a56310-1841-46ae-9a97-0421861e62c0">
Aug 09 02:32:22 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.7" ipVersion="4"/>
Aug 09 02:32:22 user nova-compute[72351]:         </nova:port>
Aug 09 02:32:22 user nova-compute[72351]:       </nova:ports>
Aug 09 02:32:22 user nova-compute[72351]:     </nova:instance>
Aug 09 02:32:22 user nova-compute[72351]:   </metadata>
Aug 09 02:32:22 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:32:22 user nova-compute[72351]:     <system>
Aug 09 02:32:22 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:32:22 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:32:22 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:32:22 user nova-compute[72351]:       <entry name="serial">7b6f5d60-5f23-4ef1-b972-9ba7086f56cb</entry>
Aug 09 02:32:22 user nova-compute[72351]:       <entry name="uuid">7b6f5d60-5f23-4ef1-b972-9ba7086f56cb</entry>
Aug 09 02:32:22 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:32:22 user nova-compute[72351]:     </system>
Aug 09 02:32:22 user nova-compute[72351]:   </sysinfo>
Aug 09 02:32:22 user nova-compute[72351]:   <os>
Aug 09 02:32:22 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:32:22 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:32:22 user nova-compute[72351]:   </os>
Aug 09 02:32:22 user nova-compute[72351]:   <features>
Aug 09 02:32:22 user nova-compute[72351]:     <acpi/>
Aug 09 02:32:22 user nova-compute[72351]:     <apic/>
Aug 09 02:32:22 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:32:22 user nova-compute[72351]:   </features>
Aug 09 02:32:22 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:32:22 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:32:22 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:32:22 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:32:22 user nova-compute[72351]:   </clock>
Aug 09 02:32:22 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:32:22 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:32:22 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:32:22 user nova-compute[72351]:   </cpu>
Aug 09 02:32:22 user nova-compute[72351]:   <devices>
Aug 09 02:32:22 user nova-compute[72351]:     <disk type="file" device="disk">
Aug 09 02:32:22 user nova-compute[72351]:       <driver name="qemu" type="qcow2" cache="none"/>
Aug 09 02:32:22 user nova-compute[72351]:       <source file="/opt/stack/data/nova/instances/7b6f5d60-5f23-4ef1-b972-9ba7086f56cb/disk.rescue"/>
Aug 09 02:32:22 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:32:22 user nova-compute[72351]:     </disk>
Aug 09 02:32:22 user nova-compute[72351]:     <disk type="file" device="disk">
Aug 09 02:32:22 user nova-compute[72351]:       <driver name="qemu" type="qcow2" cache="none"/>
Aug 09 02:32:22 user nova-compute[72351]:       <source file="/opt/stack/data/nova/instances/7b6f5d60-5f23-4ef1-b972-9ba7086f56cb/disk"/>
Aug 09 02:32:22 user nova-compute[72351]:       <target dev="vdb" bus="virtio"/>
Aug 09 02:32:22 user nova-compute[72351]:     </disk>
Aug 09 02:32:22 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:32:22 user nova-compute[72351]:       <mac address="fa:16:3e:75:cf:ba"/>
Aug 09 02:32:22 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:32:22 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:32:22 user nova-compute[72351]:       <target dev="tapc4a56310-18"/>
Aug 09 02:32:22 user nova-compute[72351]:     </interface>
Aug 09 02:32:22 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:32:22 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/7b6f5d60-5f23-4ef1-b972-9ba7086f56cb/console.log" append="off"/>
Aug 09 02:32:22 user nova-compute[72351]:     </serial>
Aug 09 02:32:22 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:32:22 user nova-compute[72351]:     <video>
Aug 09 02:32:22 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:32:22 user nova-compute[72351]:     </video>
Aug 09 02:32:22 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:32:22 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:32:22 user nova-compute[72351]:     </rng>
Aug 09 02:32:22 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:32:22 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:32:22 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:32:22 user nova-compute[72351]:     </memballoon>
Aug 09 02:32:22 user nova-compute[72351]:   </devices>
Aug 09 02:32:22 user nova-compute[72351]: </domain>
Aug 09 02:32:22 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:32:22 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Instance destroyed successfully.
Aug 09 02:32:23 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-e1d79873-9f6b-4543-ab5f-745286aa3d0a tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:32:23 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-e1d79873-9f6b-4543-ab5f-745286aa3d0a tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:32:23 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-e1d79873-9f6b-4543-ab5f-745286aa3d0a tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] No VIF found with MAC fa:16:3e:75:cf:ba, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:32:23 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:32:23 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:32:24 user nova-compute[72351]: DEBUG nova.compute.manager [req-6f8f850d-9d2b-4845-932d-ed384f553e34 req-6c6a17ca-6d6e-4f4a-8b6b-f849975271a5 service nova] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Received event network-vif-plugged-c4a56310-1841-46ae-9a97-0421861e62c0 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:32:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-6f8f850d-9d2b-4845-932d-ed384f553e34 req-6c6a17ca-6d6e-4f4a-8b6b-f849975271a5 service nova] Acquiring lock "7b6f5d60-5f23-4ef1-b972-9ba7086f56cb-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:32:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-6f8f850d-9d2b-4845-932d-ed384f553e34 req-6c6a17ca-6d6e-4f4a-8b6b-f849975271a5 service nova] Lock "7b6f5d60-5f23-4ef1-b972-9ba7086f56cb-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:32:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-6f8f850d-9d2b-4845-932d-ed384f553e34 req-6c6a17ca-6d6e-4f4a-8b6b-f849975271a5 service nova] Lock "7b6f5d60-5f23-4ef1-b972-9ba7086f56cb-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:32:24 user nova-compute[72351]: DEBUG nova.compute.manager [req-6f8f850d-9d2b-4845-932d-ed384f553e34 req-6c6a17ca-6d6e-4f4a-8b6b-f849975271a5 service nova] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] No waiting events found dispatching network-vif-plugged-c4a56310-1841-46ae-9a97-0421861e62c0 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:32:24 user nova-compute[72351]: WARNING nova.compute.manager [req-6f8f850d-9d2b-4845-932d-ed384f553e34 req-6c6a17ca-6d6e-4f4a-8b6b-f849975271a5 service nova] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Received unexpected event network-vif-plugged-c4a56310-1841-46ae-9a97-0421861e62c0 for instance with vm_state active and task_state rescuing.
Aug 09 02:32:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:32:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:32:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:32:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:32:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:32:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:32:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:32:26 user nova-compute[72351]: DEBUG nova.compute.manager [req-35abc969-c357-4a50-87bd-7147e4693143 req-2192067e-5483-4a1a-a5c7-1e27946074b1 service nova] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Received event network-vif-plugged-c4a56310-1841-46ae-9a97-0421861e62c0 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:32:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-35abc969-c357-4a50-87bd-7147e4693143 req-2192067e-5483-4a1a-a5c7-1e27946074b1 service nova] Acquiring lock "7b6f5d60-5f23-4ef1-b972-9ba7086f56cb-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:32:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-35abc969-c357-4a50-87bd-7147e4693143 req-2192067e-5483-4a1a-a5c7-1e27946074b1 service nova] Lock "7b6f5d60-5f23-4ef1-b972-9ba7086f56cb-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:32:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-35abc969-c357-4a50-87bd-7147e4693143 req-2192067e-5483-4a1a-a5c7-1e27946074b1 service nova] Lock "7b6f5d60-5f23-4ef1-b972-9ba7086f56cb-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:32:26 user nova-compute[72351]: DEBUG nova.compute.manager [req-35abc969-c357-4a50-87bd-7147e4693143 req-2192067e-5483-4a1a-a5c7-1e27946074b1 service nova] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] No waiting events found dispatching network-vif-plugged-c4a56310-1841-46ae-9a97-0421861e62c0 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:32:26 user nova-compute[72351]: WARNING nova.compute.manager [req-35abc969-c357-4a50-87bd-7147e4693143 req-2192067e-5483-4a1a-a5c7-1e27946074b1 service nova] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Received unexpected event network-vif-plugged-c4a56310-1841-46ae-9a97-0421861e62c0 for instance with vm_state active and task_state rescuing.
Aug 09 02:32:26 user nova-compute[72351]: DEBUG nova.compute.manager [req-35abc969-c357-4a50-87bd-7147e4693143 req-2192067e-5483-4a1a-a5c7-1e27946074b1 service nova] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Received event network-vif-plugged-c4a56310-1841-46ae-9a97-0421861e62c0 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:32:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-35abc969-c357-4a50-87bd-7147e4693143 req-2192067e-5483-4a1a-a5c7-1e27946074b1 service nova] Acquiring lock "7b6f5d60-5f23-4ef1-b972-9ba7086f56cb-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:32:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-35abc969-c357-4a50-87bd-7147e4693143 req-2192067e-5483-4a1a-a5c7-1e27946074b1 service nova] Lock "7b6f5d60-5f23-4ef1-b972-9ba7086f56cb-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:32:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-35abc969-c357-4a50-87bd-7147e4693143 req-2192067e-5483-4a1a-a5c7-1e27946074b1 service nova] Lock "7b6f5d60-5f23-4ef1-b972-9ba7086f56cb-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:32:26 user nova-compute[72351]: DEBUG nova.compute.manager [req-35abc969-c357-4a50-87bd-7147e4693143 req-2192067e-5483-4a1a-a5c7-1e27946074b1 service nova] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] No waiting events found dispatching network-vif-plugged-c4a56310-1841-46ae-9a97-0421861e62c0 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:32:26 user nova-compute[72351]: WARNING nova.compute.manager [req-35abc969-c357-4a50-87bd-7147e4693143 req-2192067e-5483-4a1a-a5c7-1e27946074b1 service nova] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Received unexpected event network-vif-plugged-c4a56310-1841-46ae-9a97-0421861e62c0 for instance with vm_state active and task_state rescuing.
Aug 09 02:32:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Removed pending event for 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb due to event {{(pid=72351) _event_emit_delayed /opt/stack/nova/nova/virt/libvirt/host.py:438}}
Aug 09 02:32:26 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548346.63204, 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:32:26 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] VM Resumed (Lifecycle Event)
Aug 09 02:32:26 user nova-compute[72351]: DEBUG nova.compute.manager [None req-e1d79873-9f6b-4543-ab5f-745286aa3d0a tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:32:26 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:32:26 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: active, current task_state: rescuing, current DB power_state: 1, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:32:26 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] During sync_power_state the instance has a pending task (rescuing). Skip.
Aug 09 02:32:26 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548346.632306, 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:32:26 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] VM Started (Lifecycle Event)
Aug 09 02:32:26 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:32:26 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Synchronizing instance power state after lifecycle event "Started"; current vm_state: active, current task_state: rescuing, current DB power_state: 1, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:32:28 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:32:28 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:32:29 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:32:31 user nova-compute[72351]: DEBUG nova.compute.manager [req-4bec3cea-9196-479e-849c-d36fc8c09f05 req-8f4876eb-45d2-4902-8b9e-72d2ad80a4e2 service nova] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Received event network-changed-2dcfe7b8-34fe-4860-b820-f3de7c00df62 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:32:31 user nova-compute[72351]: DEBUG nova.compute.manager [req-4bec3cea-9196-479e-849c-d36fc8c09f05 req-8f4876eb-45d2-4902-8b9e-72d2ad80a4e2 service nova] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Refreshing instance network info cache due to event network-changed-2dcfe7b8-34fe-4860-b820-f3de7c00df62. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:32:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-4bec3cea-9196-479e-849c-d36fc8c09f05 req-8f4876eb-45d2-4902-8b9e-72d2ad80a4e2 service nova] Acquiring lock "refresh_cache-49a10075-0d3c-4cd6-bb43-10238cad3f37" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:32:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-4bec3cea-9196-479e-849c-d36fc8c09f05 req-8f4876eb-45d2-4902-8b9e-72d2ad80a4e2 service nova] Acquired lock "refresh_cache-49a10075-0d3c-4cd6-bb43-10238cad3f37" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:32:31 user nova-compute[72351]: DEBUG nova.network.neutron [req-4bec3cea-9196-479e-849c-d36fc8c09f05 req-8f4876eb-45d2-4902-8b9e-72d2ad80a4e2 service nova] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Refreshing network info cache for port 2dcfe7b8-34fe-4860-b820-f3de7c00df62 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:32:31 user nova-compute[72351]: DEBUG nova.network.neutron [req-4bec3cea-9196-479e-849c-d36fc8c09f05 req-8f4876eb-45d2-4902-8b9e-72d2ad80a4e2 service nova] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Updated VIF entry in instance network info cache for port 2dcfe7b8-34fe-4860-b820-f3de7c00df62. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:32:31 user nova-compute[72351]: DEBUG nova.network.neutron [req-4bec3cea-9196-479e-849c-d36fc8c09f05 req-8f4876eb-45d2-4902-8b9e-72d2ad80a4e2 service nova] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Updating instance_info_cache with network_info: [{"id": "2dcfe7b8-34fe-4860-b820-f3de7c00df62", "address": "fa:16:3e:01:0f:c9", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.21", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2dcfe7b8-34", "ovs_interfaceid": "2dcfe7b8-34fe-4860-b820-f3de7c00df62", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:32:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-4bec3cea-9196-479e-849c-d36fc8c09f05 req-8f4876eb-45d2-4902-8b9e-72d2ad80a4e2 service nova] Releasing lock "refresh_cache-49a10075-0d3c-4cd6-bb43-10238cad3f37" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:32:33 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:32:33 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:32:36 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:32:38 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:32:38 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:32:43 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:32:43 user nova-compute[72351]: DEBUG nova.compute.manager [req-c4936996-eee0-4d07-ab2e-aebbb5985f12 req-a15110ba-df0d-4f94-a32e-52a62e4a5ed8 service nova] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Received event network-changed-1f2da7a0-a0f0-4715-8c5d-1df2a85fd295 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:32:43 user nova-compute[72351]: DEBUG nova.compute.manager [req-c4936996-eee0-4d07-ab2e-aebbb5985f12 req-a15110ba-df0d-4f94-a32e-52a62e4a5ed8 service nova] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Refreshing instance network info cache due to event network-changed-1f2da7a0-a0f0-4715-8c5d-1df2a85fd295. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:32:43 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c4936996-eee0-4d07-ab2e-aebbb5985f12 req-a15110ba-df0d-4f94-a32e-52a62e4a5ed8 service nova] Acquiring lock "refresh_cache-f37b7fb7-1b94-421b-be79-7066010efabf" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:32:43 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c4936996-eee0-4d07-ab2e-aebbb5985f12 req-a15110ba-df0d-4f94-a32e-52a62e4a5ed8 service nova] Acquired lock "refresh_cache-f37b7fb7-1b94-421b-be79-7066010efabf" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:32:43 user nova-compute[72351]: DEBUG nova.network.neutron [req-c4936996-eee0-4d07-ab2e-aebbb5985f12 req-a15110ba-df0d-4f94-a32e-52a62e4a5ed8 service nova] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Refreshing network info cache for port 1f2da7a0-a0f0-4715-8c5d-1df2a85fd295 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:32:43 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:32:43 user nova-compute[72351]: DEBUG nova.network.neutron [req-c4936996-eee0-4d07-ab2e-aebbb5985f12 req-a15110ba-df0d-4f94-a32e-52a62e4a5ed8 service nova] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Updated VIF entry in instance network info cache for port 1f2da7a0-a0f0-4715-8c5d-1df2a85fd295. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:32:43 user nova-compute[72351]: DEBUG nova.network.neutron [req-c4936996-eee0-4d07-ab2e-aebbb5985f12 req-a15110ba-df0d-4f94-a32e-52a62e4a5ed8 service nova] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Updating instance_info_cache with network_info: [{"id": "1f2da7a0-a0f0-4715-8c5d-1df2a85fd295", "address": "fa:16:3e:0a:ba:2e", "network": {"id": "49bfc70e-93c7-4607-9507-4711bda1a686", "bridge": "br-int", "label": "tempest-TestStampPattern-1656229455-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.112", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fc12f9037cd54bb6a2fdff3071132dd4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1f2da7a0-a0", "ovs_interfaceid": "1f2da7a0-a0f0-4715-8c5d-1df2a85fd295", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:32:43 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c4936996-eee0-4d07-ab2e-aebbb5985f12 req-a15110ba-df0d-4f94-a32e-52a62e4a5ed8 service nova] Releasing lock "refresh_cache-f37b7fb7-1b94-421b-be79-7066010efabf" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:32:43 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-2f09b9f9-8c23-4f8c-a198-a7e42ee9aa38 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Acquiring lock "f37b7fb7-1b94-421b-be79-7066010efabf" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:32:43 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-2f09b9f9-8c23-4f8c-a198-a7e42ee9aa38 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lock "f37b7fb7-1b94-421b-be79-7066010efabf" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:32:43 user nova-compute[72351]: DEBUG nova.objects.instance [None req-2f09b9f9-8c23-4f8c-a198-a7e42ee9aa38 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lazy-loading 'flavor' on Instance uuid f37b7fb7-1b94-421b-be79-7066010efabf {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:32:43 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-2f09b9f9-8c23-4f8c-a198-a7e42ee9aa38 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lock "f37b7fb7-1b94-421b-be79-7066010efabf" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.059s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:32:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-2f09b9f9-8c23-4f8c-a198-a7e42ee9aa38 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Acquiring lock "f37b7fb7-1b94-421b-be79-7066010efabf" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:32:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-2f09b9f9-8c23-4f8c-a198-a7e42ee9aa38 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lock "f37b7fb7-1b94-421b-be79-7066010efabf" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:32:44 user nova-compute[72351]: INFO nova.compute.manager [None req-2f09b9f9-8c23-4f8c-a198-a7e42ee9aa38 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Attaching volume 075fce6b-3efd-452d-886d-22cd05c7e146 to /dev/vdb
Aug 09 02:32:44 user nova-compute[72351]: DEBUG os_brick.utils [None req-2f09b9f9-8c23-4f8c-a198-a7e42ee9aa38 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:32:44 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:44 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:32:44 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[c53c03fb-88b4-4e48-a522-a62a0d171ca1]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:a52269602ff4\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:32:44 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:44 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:32:44 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[5f83ef5b-b440-46f4-93a9-31b040f9b471]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:32:44 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:44 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.016s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:32:44 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[28c7d58b-8165-481e-8bce-867f6c6dcd06]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:32:44 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[8a568d9a-bebf-4bcd-98ae-14dc2cc623d1]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:32:44 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-2f09b9f9-8c23-4f8c-a198-a7e42ee9aa38 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Running cmd (subprocess): nvme version {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:44 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-2f09b9f9-8c23-4f8c-a198-a7e42ee9aa38 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] 'nvme version' failed. Not Retrying. {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:32:44 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.nvmeof [None req-2f09b9f9-8c23-4f8c-a198-a7e42ee9aa38 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] nvme not present on system {{(pid=72351) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Aug 09 02:32:44 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:44 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:32:44 user nova-compute[72351]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Aug 09 02:32:44 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ebac4c68-1243-409e-aea7-cf75da5428c9]: (4, '') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:32:44 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-2f09b9f9-8c23-4f8c-a198-a7e42ee9aa38 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72351) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Aug 09 02:32:44 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-2f09b9f9-8c23-4f8c-a198-a7e42ee9aa38 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Aug 09 02:32:44 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-2f09b9f9-8c23-4f8c-a198-a7e42ee9aa38 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] LIGHTOS: no hostnqn found. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Aug 09 02:32:44 user nova-compute[72351]: DEBUG os_brick.utils [None req-2f09b9f9-8c23-4f8c-a198-a7e42ee9aa38 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] <== get_connector_properties: return (136ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:a52269602ff4', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:32:44 user nova-compute[72351]: DEBUG nova.virt.block_device [None req-2f09b9f9-8c23-4f8c-a198-a7e42ee9aa38 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Updating existing volume attachment record: 9c392f8f-d1ec-4a3f-a754-a18ea93d7b54 {{(pid=72351) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Aug 09 02:32:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquiring lock "8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:32:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:32:44 user nova-compute[72351]: DEBUG nova.compute.manager [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Starting instance... {{(pid=72351) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2406}}
Aug 09 02:32:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:32:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:32:44 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72351) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Aug 09 02:32:44 user nova-compute[72351]: INFO nova.compute.claims [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Claim successful on node user
Aug 09 02:32:45 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:32:45 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:32:45 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.600s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:32:45 user nova-compute[72351]: DEBUG nova.compute.manager [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Start building networks asynchronously for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2803}}
Aug 09 02:32:45 user nova-compute[72351]: DEBUG nova.compute.manager [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Allocating IP information in the background. {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1956}}
Aug 09 02:32:45 user nova-compute[72351]: DEBUG nova.network.neutron [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] allocate_for_instance() {{(pid=72351) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Aug 09 02:32:45 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Aug 09 02:32:45 user nova-compute[72351]: DEBUG nova.compute.manager [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Start building block device mappings for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2838}}
Aug 09 02:32:45 user nova-compute[72351]: INFO nova.virt.block_device [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Booting with volume bc935627-c165-4eca-b63b-394ec763b6fc at /dev/vda
Aug 09 02:32:45 user nova-compute[72351]: DEBUG nova.policy [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '07d8d5a634274df19c2db769d2d27b29', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4c6bbfd8d904ba1ac69ba5da4564513', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72351) authorize /opt/stack/nova/nova/policy.py:203}}
Aug 09 02:32:45 user nova-compute[72351]: DEBUG os_brick.utils [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:32:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.016s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:32:45 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[2d12fb6c-5843-4a4c-9c32-5823f0882307]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:a52269602ff4\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:32:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:32:45 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[900a7a18-1916-43ce-a3b7-b47ab408694d]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:32:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.016s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:32:45 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e7426943-2a17-4859-8ead-4ea1d23d83c1]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:32:45 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[02efc0d1-4508-4143-ae61-952f26de363a]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:32:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Running cmd (subprocess): nvme version {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] 'nvme version' failed. Not Retrying. {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:32:45 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.nvmeof [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] nvme not present on system {{(pid=72351) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Aug 09 02:32:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:32:45 user nova-compute[72351]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Aug 09 02:32:45 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[4aa7f55b-04be-46c1-a4f8-6c089e5b9c35]: (4, '') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:32:45 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72351) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Aug 09 02:32:45 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Aug 09 02:32:45 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] LIGHTOS: no hostnqn found. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Aug 09 02:32:45 user nova-compute[72351]: DEBUG os_brick.utils [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] <== get_connector_properties: return (129ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:a52269602ff4', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:32:45 user nova-compute[72351]: DEBUG nova.virt.block_device [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Updating existing volume attachment record: afefdbdc-7d67-4b3d-9923-828e34529cf1 {{(pid=72351) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Aug 09 02:32:46 user nova-compute[72351]: DEBUG nova.network.neutron [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Successfully created port: 00868cee-ead4-4f80-a2c7-1ec86fd0d78f {{(pid=72351) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Aug 09 02:32:47 user nova-compute[72351]: DEBUG nova.network.neutron [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Successfully updated port: 00868cee-ead4-4f80-a2c7-1ec86fd0d78f {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:32:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquiring lock "refresh_cache-8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:32:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquired lock "refresh_cache-8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:32:47 user nova-compute[72351]: DEBUG nova.network.neutron [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:32:47 user nova-compute[72351]: DEBUG nova.network.neutron [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:32:47 user nova-compute[72351]: DEBUG nova.compute.manager [req-f52fa068-2aa6-4e8a-a096-0c804bc89e20 req-f5af3815-6935-42d2-a53c-bd0f37ab7804 service nova] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Received event network-changed-00868cee-ead4-4f80-a2c7-1ec86fd0d78f {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:32:47 user nova-compute[72351]: DEBUG nova.compute.manager [req-f52fa068-2aa6-4e8a-a096-0c804bc89e20 req-f5af3815-6935-42d2-a53c-bd0f37ab7804 service nova] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Refreshing instance network info cache due to event network-changed-00868cee-ead4-4f80-a2c7-1ec86fd0d78f. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:32:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-f52fa068-2aa6-4e8a-a096-0c804bc89e20 req-f5af3815-6935-42d2-a53c-bd0f37ab7804 service nova] Acquiring lock "refresh_cache-8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:32:47 user nova-compute[72351]: DEBUG nova.network.neutron [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Updating instance_info_cache with network_info: [{"id": "00868cee-ead4-4f80-a2c7-1ec86fd0d78f", "address": "fa:16:3e:20:ad:e5", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap00868cee-ea", "ovs_interfaceid": "00868cee-ead4-4f80-a2c7-1ec86fd0d78f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:32:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Releasing lock "refresh_cache-8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:32:47 user nova-compute[72351]: DEBUG nova.compute.manager [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Instance network_info: |[{"id": "00868cee-ead4-4f80-a2c7-1ec86fd0d78f", "address": "fa:16:3e:20:ad:e5", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap00868cee-ea", "ovs_interfaceid": "00868cee-ead4-4f80-a2c7-1ec86fd0d78f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1971}}
Aug 09 02:32:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-f52fa068-2aa6-4e8a-a096-0c804bc89e20 req-f5af3815-6935-42d2-a53c-bd0f37ab7804 service nova] Acquired lock "refresh_cache-8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:32:47 user nova-compute[72351]: DEBUG nova.network.neutron [req-f52fa068-2aa6-4e8a-a096-0c804bc89e20 req-f5af3815-6935-42d2-a53c-bd0f37ab7804 service nova] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Refreshing network info cache for port 00868cee-ead4-4f80-a2c7-1ec86fd0d78f {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:32:48 user nova-compute[72351]: DEBUG nova.network.neutron [req-f52fa068-2aa6-4e8a-a096-0c804bc89e20 req-f5af3815-6935-42d2-a53c-bd0f37ab7804 service nova] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Updated VIF entry in instance network info cache for port 00868cee-ead4-4f80-a2c7-1ec86fd0d78f. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:32:48 user nova-compute[72351]: DEBUG nova.network.neutron [req-f52fa068-2aa6-4e8a-a096-0c804bc89e20 req-f5af3815-6935-42d2-a53c-bd0f37ab7804 service nova] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Updating instance_info_cache with network_info: [{"id": "00868cee-ead4-4f80-a2c7-1ec86fd0d78f", "address": "fa:16:3e:20:ad:e5", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap00868cee-ea", "ovs_interfaceid": "00868cee-ead4-4f80-a2c7-1ec86fd0d78f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:32:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-f52fa068-2aa6-4e8a-a096-0c804bc89e20 req-f5af3815-6935-42d2-a53c-bd0f37ab7804 service nova] Releasing lock "refresh_cache-8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:32:48 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:32:48 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:32:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:32:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG nova.compute.manager [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Start spawning the instance on the hypervisor. {{(pid=72351) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2612}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Creating instance directory {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4717}}
Aug 09 02:32:55 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Creating image(s)
Aug 09 02:32:55 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Did not create local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4853}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Ensure instance console log exists: /opt/stack/data/nova/instances/8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb/console.log {{(pid=72351) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4603}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Start _get_guest_xml network_info=[{"id": "00868cee-ead4-4f80-a2c7-1ec86fd0d78f", "address": "fa:16:3e:20:ad:e5", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap00868cee-ea", "ovs_interfaceid": "00868cee-ead4-4f80-a2c7-1ec86fd0d78f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'device_type': 'disk', 'guest_format': None, 'disk_bus': 'virtio', 'mount_device': '/dev/vda', 'boot_index': 0, 'attachment_id': 'afefdbdc-7d67-4b3d-9923-828e34529cf1', 'delete_on_termination': False, 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqns': ['iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'fHEsehNU', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb', 'attached_at': '', 'detached_at': '', 'volume_id': 'bc935627-c165-4eca-b63b-394ec763b6fc', 'serial': 'bc935627-c165-4eca-b63b-394ec763b6fc'}, 'volume_type': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:32:55 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:32:55 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:32:55 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'fHEsehNU', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:32:55 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:32:55 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:32:55 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:32:55 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:32:55 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 02:32:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc -p 172.16.0.220:3260" returned: 21 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[9cec0632-5190-4e58-b2ac-a72c67d96a6d]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Aug 09 02:32:55 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc -p 172.16.0.220:3260 --interface default --op new {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[1b880be8-fe1a-4875-a5ed-222b9b073e16]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc] added\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc] added
Aug 09 02:32:55 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a1c74a2f-de6e-45aa-8f0d-f80d744fb30f]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.104s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[faffec52-9094-4a8e-824a-f4ace091d46e]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc -p 172.16.0.220:3260 --op update -n node.session.auth.username -v fHEsehNU {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc -p 172.16.0.220:3260 --op update -n node.session.auth.username -v fHEsehNU" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[b774021c-4d19-4456-a2ed-3bbf6f049060]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'fHEsehNU'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e8e9f446-e475-434e-88f1-61c2b516beba]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e6f8cf3a-0b99-4666-9908-a21fa677cc50]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)\ntcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)\ntcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:32:55 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:32:55 user nova-compute[72351]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)
Aug 09 02:32:55 user nova-compute[72351]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:32:55 user nova-compute[72351]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)
Aug 09 02:32:55 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:32:55 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:32:55 user nova-compute[72351]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)
Aug 09 02:32:55 user nova-compute[72351]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:32:55 user nova-compute[72351]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)
Aug 09 02:32:55 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc -p 172.16.0.220:3260 --login {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc -p 172.16.0.220:3260 --login" returned: 0 in 0.077s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ee63974b-c1e0-4826-b57c-8b89822b9c21]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc, portal: 172.16.0.220,3260]
Aug 09 02:32:55 user nova-compute[72351]: Login to [iface: default, target: iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc, portal: 172.16.0.220,3260] successful.
Aug 09 02:32:55 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ae2bc87c-a0f4-4733-aca5-207bb3881259]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.027s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[20ec22bb-68de-4513-85bd-c4cc8d9bfcfb]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)\ntcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)\ntcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)\ntcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:32:55 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:32:55 user nova-compute[72351]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)
Aug 09 02:32:55 user nova-compute[72351]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:32:55 user nova-compute[72351]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)
Aug 09 02:32:55 user nova-compute[72351]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc (non-flash)
Aug 09 02:32:55 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:32:55 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:32:55 user nova-compute[72351]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)
Aug 09 02:32:55 user nova-compute[72351]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:32:55 user nova-compute[72351]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)
Aug 09 02:32:55 user nova-compute[72351]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc (non-flash)
Aug 09 02:32:55 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.389s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] HCTL ('34', '-', '-', 0) found on session 41 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Scanning host 34 c: -, t: -, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[3b64d8d0-f614-43d8-b3b9-cb6ce0d797fa]: (4, ('- - 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Searching for a device in session 41 and hctl ['34', '*', '*', 0] yield: sdc {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:32:55 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Connected to sdc using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'fHEsehNU', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 02:32:56 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.425s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:32:56 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] <== connect_volume: return (1428ms) {'type': 'block', 'scsi_wwn': '26336373433656239', 'path': '/dev/sdc'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:32:56 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26336373433656239', 'path': '/dev/sdc'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 02:32:56 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:32:44Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-server-2049659607',display_name='tempest-TestVolumeBootPattern-server-2049659607',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testvolumebootpattern-server-2049659607',id=41,image_ref='',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBM+1oHdu/+hmVTCW73FChAt+AvG2bL++YbEuGYMvrUaUYAyZQrt9Vwv7j34dhFmoVuCI0Rk18KgJ1X07X8pPcTOnOW0XFALfY2BbRHpJcmwARDVbYh8uPoNVxqSRoIR2gA==',key_name='tempest-TestVolumeBootPattern-805107831',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='a4c6bbfd8d904ba1ac69ba5da4564513',ramdisk_id='',reservation_id='r-ng07e3t8',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestVolumeBootPattern-350743460',owner_user_name='tempest-TestVolumeBootPattern-350743460-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:32:45Z,user_data=None,user_id='07d8d5a634274df19c2db769d2d27b29',uuid=8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "00868cee-ead4-4f80-a2c7-1ec86fd0d78f", "address": "fa:16:3e:20:ad:e5", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap00868cee-ea", "ovs_interfaceid": "00868cee-ead4-4f80-a2c7-1ec86fd0d78f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:32:56 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Converting VIF {"id": "00868cee-ead4-4f80-a2c7-1ec86fd0d78f", "address": "fa:16:3e:20:ad:e5", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap00868cee-ea", "ovs_interfaceid": "00868cee-ead4-4f80-a2c7-1ec86fd0d78f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:32:56 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:20:ad:e5,bridge_name='br-int',has_traffic_filtering=True,id=00868cee-ead4-4f80-a2c7-1ec86fd0d78f,network=Network(5ef428cf-e408-490c-8f60-70934f760f4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap00868cee-ea') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:32:56 user nova-compute[72351]: DEBUG nova.objects.instance [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lazy-loading 'pci_devices' on Instance uuid 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:32:56 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:32:56 user nova-compute[72351]:   <uuid>8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb</uuid>
Aug 09 02:32:56 user nova-compute[72351]:   <name>instance-00000029</name>
Aug 09 02:32:56 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:32:56 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:32:56 user nova-compute[72351]:   <metadata>
Aug 09 02:32:56 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:32:56 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:32:56 user nova-compute[72351]:       <nova:name>tempest-TestVolumeBootPattern-server-2049659607</nova:name>
Aug 09 02:32:56 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:32:55</nova:creationTime>
Aug 09 02:32:56 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:32:56 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:32:56 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:32:56 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:32:56 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:32:56 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:32:56 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:32:56 user nova-compute[72351]:       <nova:owner>
Aug 09 02:32:56 user nova-compute[72351]:         <nova:user uuid="07d8d5a634274df19c2db769d2d27b29">tempest-TestVolumeBootPattern-350743460-project-member</nova:user>
Aug 09 02:32:56 user nova-compute[72351]:         <nova:project uuid="a4c6bbfd8d904ba1ac69ba5da4564513">tempest-TestVolumeBootPattern-350743460</nova:project>
Aug 09 02:32:56 user nova-compute[72351]:       </nova:owner>
Aug 09 02:32:56 user nova-compute[72351]:       <nova:ports>
Aug 09 02:32:56 user nova-compute[72351]:         <nova:port uuid="00868cee-ead4-4f80-a2c7-1ec86fd0d78f">
Aug 09 02:32:56 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.8" ipVersion="4"/>
Aug 09 02:32:56 user nova-compute[72351]:         </nova:port>
Aug 09 02:32:56 user nova-compute[72351]:       </nova:ports>
Aug 09 02:32:56 user nova-compute[72351]:     </nova:instance>
Aug 09 02:32:56 user nova-compute[72351]:   </metadata>
Aug 09 02:32:56 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:32:56 user nova-compute[72351]:     <system>
Aug 09 02:32:56 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:32:56 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:32:56 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:32:56 user nova-compute[72351]:       <entry name="serial">8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb</entry>
Aug 09 02:32:56 user nova-compute[72351]:       <entry name="uuid">8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb</entry>
Aug 09 02:32:56 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:32:56 user nova-compute[72351]:     </system>
Aug 09 02:32:56 user nova-compute[72351]:   </sysinfo>
Aug 09 02:32:56 user nova-compute[72351]:   <os>
Aug 09 02:32:56 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:32:56 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:32:56 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:32:56 user nova-compute[72351]:   </os>
Aug 09 02:32:56 user nova-compute[72351]:   <features>
Aug 09 02:32:56 user nova-compute[72351]:     <acpi/>
Aug 09 02:32:56 user nova-compute[72351]:     <apic/>
Aug 09 02:32:56 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:32:56 user nova-compute[72351]:   </features>
Aug 09 02:32:56 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:32:56 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:32:56 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:32:56 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:32:56 user nova-compute[72351]:   </clock>
Aug 09 02:32:56 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:32:56 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:32:56 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:32:56 user nova-compute[72351]:   </cpu>
Aug 09 02:32:56 user nova-compute[72351]:   <devices>
Aug 09 02:32:56 user nova-compute[72351]:     <disk type="block" device="disk">
Aug 09 02:32:56 user nova-compute[72351]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:32:56 user nova-compute[72351]:       <source dev="/dev/sdc"/>
Aug 09 02:32:56 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:32:56 user nova-compute[72351]:       <serial>bc935627-c165-4eca-b63b-394ec763b6fc</serial>
Aug 09 02:32:56 user nova-compute[72351]:     </disk>
Aug 09 02:32:56 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:32:56 user nova-compute[72351]:       <mac address="fa:16:3e:20:ad:e5"/>
Aug 09 02:32:56 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:32:56 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:32:56 user nova-compute[72351]:       <target dev="tap00868cee-ea"/>
Aug 09 02:32:56 user nova-compute[72351]:     </interface>
Aug 09 02:32:56 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:32:56 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb/console.log" append="off"/>
Aug 09 02:32:56 user nova-compute[72351]:     </serial>
Aug 09 02:32:56 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:32:56 user nova-compute[72351]:     <video>
Aug 09 02:32:56 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:32:56 user nova-compute[72351]:     </video>
Aug 09 02:32:56 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:32:56 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:32:56 user nova-compute[72351]:     </rng>
Aug 09 02:32:56 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:32:56 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:32:56 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:32:56 user nova-compute[72351]:     </memballoon>
Aug 09 02:32:56 user nova-compute[72351]:   </devices>
Aug 09 02:32:56 user nova-compute[72351]: </domain>
Aug 09 02:32:56 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:32:56 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:32:44Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-server-2049659607',display_name='tempest-TestVolumeBootPattern-server-2049659607',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testvolumebootpattern-server-2049659607',id=41,image_ref='',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBM+1oHdu/+hmVTCW73FChAt+AvG2bL++YbEuGYMvrUaUYAyZQrt9Vwv7j34dhFmoVuCI0Rk18KgJ1X07X8pPcTOnOW0XFALfY2BbRHpJcmwARDVbYh8uPoNVxqSRoIR2gA==',key_name='tempest-TestVolumeBootPattern-805107831',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='a4c6bbfd8d904ba1ac69ba5da4564513',ramdisk_id='',reservation_id='r-ng07e3t8',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestVolumeBootPattern-350743460',owner_user_name='tempest-TestVolumeBootPattern-350743460-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:32:45Z,user_data=None,user_id='07d8d5a634274df19c2db769d2d27b29',uuid=8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "00868cee-ead4-4f80-a2c7-1ec86fd0d78f", "address": "fa:16:3e:20:ad:e5", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap00868cee-ea", "ovs_interfaceid": "00868cee-ead4-4f80-a2c7-1ec86fd0d78f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:32:56 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Converting VIF {"id": "00868cee-ead4-4f80-a2c7-1ec86fd0d78f", "address": "fa:16:3e:20:ad:e5", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap00868cee-ea", "ovs_interfaceid": "00868cee-ead4-4f80-a2c7-1ec86fd0d78f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:32:56 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:20:ad:e5,bridge_name='br-int',has_traffic_filtering=True,id=00868cee-ead4-4f80-a2c7-1ec86fd0d78f,network=Network(5ef428cf-e408-490c-8f60-70934f760f4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap00868cee-ea') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:32:56 user nova-compute[72351]: DEBUG os_vif [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:20:ad:e5,bridge_name='br-int',has_traffic_filtering=True,id=00868cee-ead4-4f80-a2c7-1ec86fd0d78f,network=Network(5ef428cf-e408-490c-8f60-70934f760f4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap00868cee-ea') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:32:56 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:32:56 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:32:56 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:32:56 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:32:56 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap00868cee-ea, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:32:56 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap00868cee-ea, col_values=(('external_ids', {'iface-id': '00868cee-ead4-4f80-a2c7-1ec86fd0d78f', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:20:ad:e5', 'vm-uuid': '8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:32:56 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:32:56 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:32:56 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:32:56 user nova-compute[72351]: INFO os_vif [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:20:ad:e5,bridge_name='br-int',has_traffic_filtering=True,id=00868cee-ead4-4f80-a2c7-1ec86fd0d78f,network=Network(5ef428cf-e408-490c-8f60-70934f760f4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap00868cee-ea')
Aug 09 02:32:56 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:32:56 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] No VIF found with MAC fa:16:3e:20:ad:e5, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:32:58 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:32:58 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:32:58 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:32:58 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:32:58 user nova-compute[72351]: DEBUG nova.compute.manager [req-134ac386-b83e-49a3-bdbe-38f00dd9e6d5 req-26917f60-e29e-483a-926a-0753a49dc23a service nova] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Received event network-vif-plugged-00868cee-ead4-4f80-a2c7-1ec86fd0d78f {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:32:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-134ac386-b83e-49a3-bdbe-38f00dd9e6d5 req-26917f60-e29e-483a-926a-0753a49dc23a service nova] Acquiring lock "8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:32:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-134ac386-b83e-49a3-bdbe-38f00dd9e6d5 req-26917f60-e29e-483a-926a-0753a49dc23a service nova] Lock "8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:32:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-134ac386-b83e-49a3-bdbe-38f00dd9e6d5 req-26917f60-e29e-483a-926a-0753a49dc23a service nova] Lock "8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:32:58 user nova-compute[72351]: DEBUG nova.compute.manager [req-134ac386-b83e-49a3-bdbe-38f00dd9e6d5 req-26917f60-e29e-483a-926a-0753a49dc23a service nova] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] No waiting events found dispatching network-vif-plugged-00868cee-ead4-4f80-a2c7-1ec86fd0d78f {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:32:58 user nova-compute[72351]: WARNING nova.compute.manager [req-134ac386-b83e-49a3-bdbe-38f00dd9e6d5 req-26917f60-e29e-483a-926a-0753a49dc23a service nova] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Received unexpected event network-vif-plugged-00868cee-ead4-4f80-a2c7-1ec86fd0d78f for instance with vm_state building and task_state spawning.
Aug 09 02:32:58 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:32:58 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:00 user nova-compute[72351]: DEBUG nova.compute.manager [req-51602ec6-4860-491e-b739-f23da8daf39b req-07e2723e-70aa-4793-8cb1-8c372dea5044 service nova] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Received event network-vif-plugged-00868cee-ead4-4f80-a2c7-1ec86fd0d78f {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:33:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-51602ec6-4860-491e-b739-f23da8daf39b req-07e2723e-70aa-4793-8cb1-8c372dea5044 service nova] Acquiring lock "8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:33:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-51602ec6-4860-491e-b739-f23da8daf39b req-07e2723e-70aa-4793-8cb1-8c372dea5044 service nova] Lock "8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:33:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-51602ec6-4860-491e-b739-f23da8daf39b req-07e2723e-70aa-4793-8cb1-8c372dea5044 service nova] Lock "8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:33:00 user nova-compute[72351]: DEBUG nova.compute.manager [req-51602ec6-4860-491e-b739-f23da8daf39b req-07e2723e-70aa-4793-8cb1-8c372dea5044 service nova] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] No waiting events found dispatching network-vif-plugged-00868cee-ead4-4f80-a2c7-1ec86fd0d78f {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:33:00 user nova-compute[72351]: WARNING nova.compute.manager [req-51602ec6-4860-491e-b739-f23da8daf39b req-07e2723e-70aa-4793-8cb1-8c372dea5044 service nova] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Received unexpected event network-vif-plugged-00868cee-ead4-4f80-a2c7-1ec86fd0d78f for instance with vm_state building and task_state spawning.
Aug 09 02:33:00 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548380.648196, 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:33:00 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] VM Resumed (Lifecycle Event)
Aug 09 02:33:00 user nova-compute[72351]: DEBUG nova.compute.manager [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:33:00 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Guest created on hypervisor {{(pid=72351) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4411}}
Aug 09 02:33:00 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Instance spawned successfully.
Aug 09 02:33:00 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:894}}
Aug 09 02:33:00 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:33:00 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:33:00 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Found default for hw_cdrom_bus of ide {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:33:00 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Found default for hw_disk_bus of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:33:00 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Found default for hw_input_bus of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:33:00 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Found default for hw_pointer_model of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:33:00 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Found default for hw_video_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:33:00 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Found default for hw_vif_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:33:00 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:33:00 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548380.6502857, 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:33:00 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] VM Started (Lifecycle Event)
Aug 09 02:33:00 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:33:00 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:33:00 user nova-compute[72351]: INFO nova.compute.manager [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Took 5.44 seconds to spawn the instance on the hypervisor.
Aug 09 02:33:00 user nova-compute[72351]: DEBUG nova.compute.manager [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:33:00 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:33:00 user nova-compute[72351]: INFO nova.compute.manager [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Took 16.19 seconds to build instance.
Aug 09 02:33:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-28025223-545e-4fdf-86d2-326dda1aaa44 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 16.300s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:33:01 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:01 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-bfb0caa3-0059-49d2-b0de-3737426601f8 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Acquiring lock "8e970bdd-54a1-4058-b09d-784cb6b5b1a4" by "nova.compute.manager.ComputeManager.shelve_offload_instance.<locals>.do_shelve_offload_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:33:01 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-bfb0caa3-0059-49d2-b0de-3737426601f8 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Lock "8e970bdd-54a1-4058-b09d-784cb6b5b1a4" acquired by "nova.compute.manager.ComputeManager.shelve_offload_instance.<locals>.do_shelve_offload_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:33:01 user nova-compute[72351]: INFO nova.compute.manager [None req-bfb0caa3-0059-49d2-b0de-3737426601f8 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Shelve offloading
Aug 09 02:33:01 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:01 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:01 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:01 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:01 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:02 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Instance destroyed successfully.
Aug 09 02:33:02 user nova-compute[72351]: DEBUG nova.compute.manager [None req-bfb0caa3-0059-49d2-b0de-3737426601f8 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:33:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-bfb0caa3-0059-49d2-b0de-3737426601f8 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Acquiring lock "refresh_cache-8e970bdd-54a1-4058-b09d-784cb6b5b1a4" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:33:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-bfb0caa3-0059-49d2-b0de-3737426601f8 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Acquired lock "refresh_cache-8e970bdd-54a1-4058-b09d-784cb6b5b1a4" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:33:02 user nova-compute[72351]: DEBUG nova.network.neutron [None req-bfb0caa3-0059-49d2-b0de-3737426601f8 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:33:02 user nova-compute[72351]: DEBUG nova.compute.manager [req-c4d35d8e-df42-42fa-a0fe-4e6e3332fe28 req-fe875017-ec87-4472-91f5-2913b139cc8b service nova] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Received event network-changed-a83afb1f-511f-406c-b925-eb70b32bed90 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:33:02 user nova-compute[72351]: DEBUG nova.compute.manager [req-c4d35d8e-df42-42fa-a0fe-4e6e3332fe28 req-fe875017-ec87-4472-91f5-2913b139cc8b service nova] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Refreshing instance network info cache due to event network-changed-a83afb1f-511f-406c-b925-eb70b32bed90. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:33:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c4d35d8e-df42-42fa-a0fe-4e6e3332fe28 req-fe875017-ec87-4472-91f5-2913b139cc8b service nova] Acquiring lock "refresh_cache-8e970bdd-54a1-4058-b09d-784cb6b5b1a4" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:33:02 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-2f09b9f9-8c23-4f8c-a198-a7e42ee9aa38 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-2f09b9f9-8c23-4f8c-a198-a7e42ee9aa38 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:075fce6b-3efd-452d-886d-22cd05c7e146'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'BJOXsaHo', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:33:03 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-2f09b9f9-8c23-4f8c-a198-a7e42ee9aa38 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:33:03 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-2f09b9f9-8c23-4f8c-a198-a7e42ee9aa38 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-2f09b9f9-8c23-4f8c-a198-a7e42ee9aa38 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:33:03 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-2f09b9f9-8c23-4f8c-a198-a7e42ee9aa38 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:33:03 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-2f09b9f9-8c23-4f8c-a198-a7e42ee9aa38 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:075fce6b-3efd-452d-886d-22cd05c7e146" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-2f09b9f9-8c23-4f8c-a198-a7e42ee9aa38 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:075fce6b-3efd-452d-886d-22cd05c7e146" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:33:03 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-2f09b9f9-8c23-4f8c-a198-a7e42ee9aa38 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 02:33:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:075fce6b-3efd-452d-886d-22cd05c7e146 -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:075fce6b-3efd-452d-886d-22cd05c7e146 -p 172.16.0.220:3260" returned: 21 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ac0dd8d6-c97f-41ad-98e2-dedca09a17cf]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-2f09b9f9-8c23-4f8c-a198-a7e42ee9aa38 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Aug 09 02:33:03 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:075fce6b-3efd-452d-886d-22cd05c7e146 -p 172.16.0.220:3260 --interface default --op new {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:075fce6b-3efd-452d-886d-22cd05c7e146 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[cd14f8ed-71a6-418b-a326-072140633f05]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:075fce6b-3efd-452d-886d-22cd05c7e146] added\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-2f09b9f9-8c23-4f8c-a198-a7e42ee9aa38 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:075fce6b-3efd-452d-886d-22cd05c7e146] added
Aug 09 02:33:03 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:075fce6b-3efd-452d-886d-22cd05c7e146 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG nova.network.neutron [None req-bfb0caa3-0059-49d2-b0de-3737426601f8 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Updating instance_info_cache with network_info: [{"id": "a83afb1f-511f-406c-b925-eb70b32bed90", "address": "fa:16:3e:16:83:e9", "network": {"id": "0b3a34b6-6d32-4414-9135-22bd2a1dae20", "bridge": "br-int", "label": "tempest-TestShelveInstance-2024538322-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.128", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4d0cc057c220422a9541bcb9fa7f598b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa83afb1f-51", "ovs_interfaceid": "a83afb1f-511f-406c-b925-eb70b32bed90", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:075fce6b-3efd-452d-886d-22cd05c7e146 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[4f5cc158-5890-4847-993a-0cab09c0293e]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-2f09b9f9-8c23-4f8c-a198-a7e42ee9aa38 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:075fce6b-3efd-452d-886d-22cd05c7e146 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-bfb0caa3-0059-49d2-b0de-3737426601f8 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Releasing lock "refresh_cache-8e970bdd-54a1-4058-b09d-784cb6b5b1a4" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c4d35d8e-df42-42fa-a0fe-4e6e3332fe28 req-fe875017-ec87-4472-91f5-2913b139cc8b service nova] Acquired lock "refresh_cache-8e970bdd-54a1-4058-b09d-784cb6b5b1a4" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG nova.network.neutron [req-c4d35d8e-df42-42fa-a0fe-4e6e3332fe28 req-fe875017-ec87-4472-91f5-2913b139cc8b service nova] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Refreshing network info cache for port a83afb1f-511f-406c-b925-eb70b32bed90 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:075fce6b-3efd-452d-886d-22cd05c7e146 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[1f686e23-cc66-40cf-ab07-56ec6f06b139]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-2f09b9f9-8c23-4f8c-a198-a7e42ee9aa38 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:075fce6b-3efd-452d-886d-22cd05c7e146 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v BJOXsaHo {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:075fce6b-3efd-452d-886d-22cd05c7e146 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v BJOXsaHo" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e2c0a2cf-5dbc-407c-b77b-8c0082cde5f1]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-2f09b9f9-8c23-4f8c-a198-a7e42ee9aa38 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'BJOXsaHo'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:075fce6b-3efd-452d-886d-22cd05c7e146 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:075fce6b-3efd-452d-886d-22cd05c7e146 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.025s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[bf9e1f68-9f1b-425c-8075-22b17b534e6f]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-2f09b9f9-8c23-4f8c-a198-a7e42ee9aa38 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.032s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[1f29a691-8cf6-4263-8a6a-2e5113208d14]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)\ntcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)\ntcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)\ntcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-2f09b9f9-8c23-4f8c-a198-a7e42ee9aa38 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:33:03 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:33:03 user nova-compute[72351]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)
Aug 09 02:33:03 user nova-compute[72351]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:33:03 user nova-compute[72351]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)
Aug 09 02:33:03 user nova-compute[72351]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc (non-flash)
Aug 09 02:33:03 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-2f09b9f9-8c23-4f8c-a198-a7e42ee9aa38 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:33:03 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:33:03 user nova-compute[72351]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)
Aug 09 02:33:03 user nova-compute[72351]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:33:03 user nova-compute[72351]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)
Aug 09 02:33:03 user nova-compute[72351]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc (non-flash)
Aug 09 02:33:03 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:075fce6b-3efd-452d-886d-22cd05c7e146 -p 172.16.0.220:3260 --login {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:075fce6b-3efd-452d-886d-22cd05c7e146 -p 172.16.0.220:3260 --login" returned: 0 in 0.088s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[40621795-61ce-49c4-85cc-e58bad44613c]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:075fce6b-3efd-452d-886d-22cd05c7e146, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:075fce6b-3efd-452d-886d-22cd05c7e146, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-2f09b9f9-8c23-4f8c-a198-a7e42ee9aa38 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:075fce6b-3efd-452d-886d-22cd05c7e146, portal: 172.16.0.220,3260]
Aug 09 02:33:03 user nova-compute[72351]: Login to [iface: default, target: iqn.2010-10.org.openstack:075fce6b-3efd-452d-886d-22cd05c7e146, portal: 172.16.0.220,3260] successful.
Aug 09 02:33:03 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:075fce6b-3efd-452d-886d-22cd05c7e146 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:075fce6b-3efd-452d-886d-22cd05c7e146 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[7af48428-284b-4a4b-b377-6dbb3102803c]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-2f09b9f9-8c23-4f8c-a198-a7e42ee9aa38 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.034s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ecbee1aa-2e22-4ada-a449-5b5988ba55f5]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)\ntcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)\ntcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)\ntcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc (non-flash)\ntcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:075fce6b-3efd-452d-886d-22cd05c7e146 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-2f09b9f9-8c23-4f8c-a198-a7e42ee9aa38 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:33:03 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:33:03 user nova-compute[72351]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)
Aug 09 02:33:03 user nova-compute[72351]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:33:03 user nova-compute[72351]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)
Aug 09 02:33:03 user nova-compute[72351]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc (non-flash)
Aug 09 02:33:03 user nova-compute[72351]: tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:075fce6b-3efd-452d-886d-22cd05c7e146 (non-flash)
Aug 09 02:33:03 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-2f09b9f9-8c23-4f8c-a198-a7e42ee9aa38 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:33:03 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:33:03 user nova-compute[72351]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)
Aug 09 02:33:03 user nova-compute[72351]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:33:03 user nova-compute[72351]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)
Aug 09 02:33:03 user nova-compute[72351]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc (non-flash)
Aug 09 02:33:03 user nova-compute[72351]: tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:075fce6b-3efd-452d-886d-22cd05c7e146 (non-flash)
Aug 09 02:33:03 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-2f09b9f9-8c23-4f8c-a198-a7e42ee9aa38 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:075fce6b-3efd-452d-886d-22cd05c7e146" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.349s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-2f09b9f9-8c23-4f8c-a198-a7e42ee9aa38 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-2f09b9f9-8c23-4f8c-a198-a7e42ee9aa38 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] HCTL ('35', '-', '-', 0) found on session 42 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-2f09b9f9-8c23-4f8c-a198-a7e42ee9aa38 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Scanning host 35 c: -, t: -, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host35/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host35/scan" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[5ff5969a-87b7-47ad-bdb2-34d79945ef66]: (4, ('- - 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-2f09b9f9-8c23-4f8c-a198-a7e42ee9aa38 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Searching for a device in session 42 and hctl ['35', '*', '*', 0] yield: None {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG nova.network.neutron [req-c4d35d8e-df42-42fa-a0fe-4e6e3332fe28 req-fe875017-ec87-4472-91f5-2913b139cc8b service nova] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Updated VIF entry in instance network info cache for port a83afb1f-511f-406c-b925-eb70b32bed90. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG nova.network.neutron [req-c4d35d8e-df42-42fa-a0fe-4e6e3332fe28 req-fe875017-ec87-4472-91f5-2913b139cc8b service nova] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Updating instance_info_cache with network_info: [{"id": "a83afb1f-511f-406c-b925-eb70b32bed90", "address": "fa:16:3e:16:83:e9", "network": {"id": "0b3a34b6-6d32-4414-9135-22bd2a1dae20", "bridge": "br-int", "label": "tempest-TestShelveInstance-2024538322-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.128", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4d0cc057c220422a9541bcb9fa7f598b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa83afb1f-51", "ovs_interfaceid": "a83afb1f-511f-406c-b925-eb70b32bed90", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c4d35d8e-df42-42fa-a0fe-4e6e3332fe28 req-fe875017-ec87-4472-91f5-2913b139cc8b service nova] Releasing lock "refresh_cache-8e970bdd-54a1-4058-b09d-784cb6b5b1a4" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG nova.compute.manager [req-c4d35d8e-df42-42fa-a0fe-4e6e3332fe28 req-fe875017-ec87-4472-91f5-2913b139cc8b service nova] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Received event network-vif-unplugged-a83afb1f-511f-406c-b925-eb70b32bed90 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c4d35d8e-df42-42fa-a0fe-4e6e3332fe28 req-fe875017-ec87-4472-91f5-2913b139cc8b service nova] Acquiring lock "8e970bdd-54a1-4058-b09d-784cb6b5b1a4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c4d35d8e-df42-42fa-a0fe-4e6e3332fe28 req-fe875017-ec87-4472-91f5-2913b139cc8b service nova] Lock "8e970bdd-54a1-4058-b09d-784cb6b5b1a4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c4d35d8e-df42-42fa-a0fe-4e6e3332fe28 req-fe875017-ec87-4472-91f5-2913b139cc8b service nova] Lock "8e970bdd-54a1-4058-b09d-784cb6b5b1a4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG nova.compute.manager [req-c4d35d8e-df42-42fa-a0fe-4e6e3332fe28 req-fe875017-ec87-4472-91f5-2913b139cc8b service nova] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] No waiting events found dispatching network-vif-unplugged-a83afb1f-511f-406c-b925-eb70b32bed90 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:33:03 user nova-compute[72351]: WARNING nova.compute.manager [req-c4d35d8e-df42-42fa-a0fe-4e6e3332fe28 req-fe875017-ec87-4472-91f5-2913b139cc8b service nova] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Received unexpected event network-vif-unplugged-a83afb1f-511f-406c-b925-eb70b32bed90 for instance with vm_state active and task_state shelving.
Aug 09 02:33:03 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Instance destroyed successfully.
Aug 09 02:33:03 user nova-compute[72351]: DEBUG nova.objects.instance [None req-bfb0caa3-0059-49d2-b0de-3737426601f8 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Lazy-loading 'resources' on Instance uuid 8e970bdd-54a1-4058-b09d-784cb6b5b1a4 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72351) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10461}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-bfb0caa3-0059-49d2-b0de-3737426601f8 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:31:07Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestShelveInstance-server-805905709',display_name='tempest-TestShelveInstance-server-805905709',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testshelveinstance-server-805905709',id=37,image_ref='',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBD9usWQkTfQS1QEHlx5U391/Ea1m4d+DjLQmZxWBZf5X7UiiUjcuxsL0cAFVTRKCtfIt43KygEG5qAmqMe/mW/kt3gzCiL7+L73rWvmAvMWlhU7Lvh6YyPMPT8wlQXad2w==',key_name='tempest-TestShelveInstance-826835257',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:31:26Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=<?>,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='4d0cc057c220422a9541bcb9fa7f598b',ramdisk_id='',reservation_id='r-ndibkwoj',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TestShelveInstance-1998695689',owner_user_name='tempest-TestShelveInstance-1998695689-project-member'},tags=<?>,task_state='shelving',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:31:26Z,user_data=None,user_id='0aa403f4ead84aecb78a538c2ced64e9',uuid=8e970bdd-54a1-4058-b09d-784cb6b5b1a4,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "a83afb1f-511f-406c-b925-eb70b32bed90", "address": "fa:16:3e:16:83:e9", "network": {"id": "0b3a34b6-6d32-4414-9135-22bd2a1dae20", "bridge": "br-int", "label": "tempest-TestShelveInstance-2024538322-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.128", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4d0cc057c220422a9541bcb9fa7f598b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa83afb1f-51", "ovs_interfaceid": "a83afb1f-511f-406c-b925-eb70b32bed90", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-bfb0caa3-0059-49d2-b0de-3737426601f8 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Converting VIF {"id": "a83afb1f-511f-406c-b925-eb70b32bed90", "address": "fa:16:3e:16:83:e9", "network": {"id": "0b3a34b6-6d32-4414-9135-22bd2a1dae20", "bridge": "br-int", "label": "tempest-TestShelveInstance-2024538322-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.128", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4d0cc057c220422a9541bcb9fa7f598b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa83afb1f-51", "ovs_interfaceid": "a83afb1f-511f-406c-b925-eb70b32bed90", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-bfb0caa3-0059-49d2-b0de-3737426601f8 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:16:83:e9,bridge_name='br-int',has_traffic_filtering=True,id=a83afb1f-511f-406c-b925-eb70b32bed90,network=Network(0b3a34b6-6d32-4414-9135-22bd2a1dae20),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa83afb1f-51') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG os_vif [None req-bfb0caa3-0059-49d2-b0de-3737426601f8 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:16:83:e9,bridge_name='br-int',has_traffic_filtering=True,id=a83afb1f-511f-406c-b925-eb70b32bed90,network=Network(0b3a34b6-6d32-4414-9135-22bd2a1dae20),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa83afb1f-51') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapa83afb1f-51, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:33:03 user nova-compute[72351]: INFO os_vif [None req-bfb0caa3-0059-49d2-b0de-3737426601f8 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:16:83:e9,bridge_name='br-int',has_traffic_filtering=True,id=a83afb1f-511f-406c-b925-eb70b32bed90,network=Network(0b3a34b6-6d32-4414-9135-22bd2a1dae20),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa83afb1f-51')
Aug 09 02:33:03 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-bfb0caa3-0059-49d2-b0de-3737426601f8 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 02:33:03 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-bfb0caa3-0059-49d2-b0de-3737426601f8 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'EalQALYR', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sde'}, None), 'kwargs': True}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:33:03 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-bfb0caa3-0059-49d2-b0de-3737426601f8 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:33:03 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-bfb0caa3-0059-49d2-b0de-3737426601f8 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-2f09b9f9-8c23-4f8c-a198-a7e42ee9aa38 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Searching for a device in session 42 and hctl ['35', '*', '*', 0] yield: sdd {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-2f09b9f9-8c23-4f8c-a198-a7e42ee9aa38 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Connected to sdd using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'BJOXsaHo', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-2f09b9f9-8c23-4f8c-a198-a7e42ee9aa38 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.382s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-2f09b9f9-8c23-4f8c-a198-a7e42ee9aa38 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] <== connect_volume: return (1384ms) {'type': 'block', 'scsi_wwn': '26566613765626561', 'path': '/dev/sdd'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-2f09b9f9-8c23-4f8c-a198-a7e42ee9aa38 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26566613765626561', 'path': '/dev/sdd'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-bfb0caa3-0059-49d2-b0de-3737426601f8 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.543s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-bfb0caa3-0059-49d2-b0de-3737426601f8 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG nova.objects.instance [None req-2f09b9f9-8c23-4f8c-a198-a7e42ee9aa38 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lazy-loading 'flavor' on Instance uuid f37b7fb7-1b94-421b-be79-7066010efabf {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.026s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[af2890aa-effa-4ecb-afdc-b203635e990f]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:075fce6b-3efd-452d-886d-22cd05c7e146\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-2f09b9f9-8c23-4f8c-a198-a7e42ee9aa38 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] attach device xml: <disk type="block" device="disk">
Aug 09 02:33:04 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:33:04 user nova-compute[72351]:   <source dev="/dev/sdd"/>
Aug 09 02:33:04 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:33:04 user nova-compute[72351]:   <serial>075fce6b-3efd-452d-886d-22cd05c7e146</serial>
Aug 09 02:33:04 user nova-compute[72351]: </disk>
Aug 09 02:33:04 user nova-compute[72351]:  {{(pid=72351) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.047s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[72e2d450-61e9-44f6-9130-905a9cbbae47]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)\ntcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)\ntcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)\ntcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc (non-flash)\ntcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:075fce6b-3efd-452d-886d-22cd05c7e146 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-bfb0caa3-0059-49d2-b0de-3737426601f8 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:33:04 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:33:04 user nova-compute[72351]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)
Aug 09 02:33:04 user nova-compute[72351]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:33:04 user nova-compute[72351]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)
Aug 09 02:33:04 user nova-compute[72351]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc (non-flash)
Aug 09 02:33:04 user nova-compute[72351]: tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:075fce6b-3efd-452d-886d-22cd05c7e146 (non-flash)
Aug 09 02:33:04 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-bfb0caa3-0059-49d2-b0de-3737426601f8 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:33:04 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:33:04 user nova-compute[72351]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)
Aug 09 02:33:04 user nova-compute[72351]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:33:04 user nova-compute[72351]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)
Aug 09 02:33:04 user nova-compute[72351]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc (non-flash)
Aug 09 02:33:04 user nova-compute[72351]: tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:075fce6b-3efd-452d-886d-22cd05c7e146 (non-flash)
Aug 09 02:33:04 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-bfb0caa3-0059-49d2-b0de-3737426601f8 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fcfdc1be170>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3'): ({'sde'}, set())}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-bfb0caa3-0059-49d2-b0de-3737426601f8 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Removing single pathed devices sde {{(pid=72351) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[6487b502-7a70-4226-a4a1-0e42a41b7072]: (4, ('path checker states:\nup                  7\n\npaths: 0\nbusy: False\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sde {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sde" returned: 0 in 0.026s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[c3d49b89-b5a4-47e0-8d10-e6ed87e4c5d6]: (4, ('ok\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-bfb0caa3-0059-49d2-b0de-3737426601f8 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Flushing IO for device /dev/sde {{(pid=72351) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sde {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sde" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[c8a4897d-9348-4893-b468-fc6e7f582fb6]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-bfb0caa3-0059-49d2-b0de-3737426601f8 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Remove SCSI device /dev/sde with /sys/block/sde/device/delete {{(pid=72351) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sde/device/delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sde/device/delete" returned: 0 in 0.062s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[63948935-5440-4cc0-aeb3-6b5550d27c8a]: (4, ('1', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-bfb0caa3-0059-49d2-b0de-3737426601f8 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Checking to see if SCSI volumes sde have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-bfb0caa3-0059-49d2-b0de-3737426601f8 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] SCSI volumes sde have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-bfb0caa3-0059-49d2-b0de-3737426601f8 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3')] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.026s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[810ef6f7-518e-49de-bb83-f4f11823541f]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-bfb0caa3-0059-49d2-b0de-3737426601f8 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 -p 172.16.0.220:3260 --logout {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG nova.compute.manager [req-6ecb4c35-33f4-4fed-8011-bde96da95a88 req-d863b990-c44f-45c5-b0c1-17d3f484e8b9 service nova] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Received event network-vif-plugged-a83afb1f-511f-406c-b925-eb70b32bed90 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-6ecb4c35-33f4-4fed-8011-bde96da95a88 req-d863b990-c44f-45c5-b0c1-17d3f484e8b9 service nova] Acquiring lock "8e970bdd-54a1-4058-b09d-784cb6b5b1a4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-6ecb4c35-33f4-4fed-8011-bde96da95a88 req-d863b990-c44f-45c5-b0c1-17d3f484e8b9 service nova] Lock "8e970bdd-54a1-4058-b09d-784cb6b5b1a4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-6ecb4c35-33f4-4fed-8011-bde96da95a88 req-d863b990-c44f-45c5-b0c1-17d3f484e8b9 service nova] Lock "8e970bdd-54a1-4058-b09d-784cb6b5b1a4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG nova.compute.manager [req-6ecb4c35-33f4-4fed-8011-bde96da95a88 req-d863b990-c44f-45c5-b0c1-17d3f484e8b9 service nova] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] No waiting events found dispatching network-vif-plugged-a83afb1f-511f-406c-b925-eb70b32bed90 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:33:04 user nova-compute[72351]: WARNING nova.compute.manager [req-6ecb4c35-33f4-4fed-8011-bde96da95a88 req-d863b990-c44f-45c5-b0c1-17d3f484e8b9 service nova] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Received unexpected event network-vif-plugged-a83afb1f-511f-406c-b925-eb70b32bed90 for instance with vm_state active and task_state shelving.
Aug 09 02:33:04 user nova-compute[72351]: DEBUG nova.compute.manager [req-6ecb4c35-33f4-4fed-8011-bde96da95a88 req-d863b990-c44f-45c5-b0c1-17d3f484e8b9 service nova] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Received event network-changed-a83afb1f-511f-406c-b925-eb70b32bed90 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG nova.compute.manager [req-6ecb4c35-33f4-4fed-8011-bde96da95a88 req-d863b990-c44f-45c5-b0c1-17d3f484e8b9 service nova] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Refreshing instance network info cache due to event network-changed-a83afb1f-511f-406c-b925-eb70b32bed90. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-6ecb4c35-33f4-4fed-8011-bde96da95a88 req-d863b990-c44f-45c5-b0c1-17d3f484e8b9 service nova] Acquiring lock "refresh_cache-8e970bdd-54a1-4058-b09d-784cb6b5b1a4" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-6ecb4c35-33f4-4fed-8011-bde96da95a88 req-d863b990-c44f-45c5-b0c1-17d3f484e8b9 service nova] Acquired lock "refresh_cache-8e970bdd-54a1-4058-b09d-784cb6b5b1a4" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG nova.network.neutron [req-6ecb4c35-33f4-4fed-8011-bde96da95a88 req-d863b990-c44f-45c5-b0c1-17d3f484e8b9 service nova] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Refreshing network info cache for port a83afb1f-511f-406c-b925-eb70b32bed90 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 -p 172.16.0.220:3260 --logout" returned: 0 in 0.094s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[fac1e8c5-1f03-4623-8e5e-1de551a3839e]: (4, ('Logging out of session [sid: 38, target: iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3, portal: 172.16.0.220,3260]\nLogout of [sid: 38, target: iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-bfb0caa3-0059-49d2-b0de-3737426601f8 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 38, target: iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3, portal: 172.16.0.220,3260]
Aug 09 02:33:04 user nova-compute[72351]: Logout of [sid: 38, target: iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3, portal: 172.16.0.220,3260] successful.
Aug 09 02:33:04 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 -p 172.16.0.220:3260 --op delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.035s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a33bad04-4256-454b-9f1a-1f2c1742f38a]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-bfb0caa3-0059-49d2-b0de-3737426601f8 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-bfb0caa3-0059-49d2-b0de-3737426601f8 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.428s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-bfb0caa3-0059-49d2-b0de-3737426601f8 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] <== disconnect_volume: return (971ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-bfb0caa3-0059-49d2-b0de-3737426601f8 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 02:33:04 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-bfb0caa3-0059-49d2-b0de-3737426601f8 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Deleting instance files /opt/stack/data/nova/instances/8e970bdd-54a1-4058-b09d-784cb6b5b1a4_del
Aug 09 02:33:04 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-bfb0caa3-0059-49d2-b0de-3737426601f8 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Deletion of /opt/stack/data/nova/instances/8e970bdd-54a1-4058-b09d-784cb6b5b1a4_del complete
Aug 09 02:33:04 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-2f09b9f9-8c23-4f8c-a198-a7e42ee9aa38 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-2f09b9f9-8c23-4f8c-a198-a7e42ee9aa38 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:33:04 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-2f09b9f9-8c23-4f8c-a198-a7e42ee9aa38 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] No VIF found with MAC fa:16:3e:0a:ba:2e, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:33:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-2f09b9f9-8c23-4f8c-a198-a7e42ee9aa38 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lock "f37b7fb7-1b94-421b-be79-7066010efabf" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 21.014s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:33:05 user nova-compute[72351]: DEBUG nova.compute.manager [req-d2d403fd-81a6-436a-85c7-f75870b9c6c1 req-5e2023a2-ca79-47a8-8a80-690b7d22dd44 service nova] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Received event network-changed-e521fae4-093d-4c92-a32d-684c1a5137f5 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:33:05 user nova-compute[72351]: DEBUG nova.compute.manager [req-d2d403fd-81a6-436a-85c7-f75870b9c6c1 req-5e2023a2-ca79-47a8-8a80-690b7d22dd44 service nova] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Refreshing instance network info cache due to event network-changed-e521fae4-093d-4c92-a32d-684c1a5137f5. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:33:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d2d403fd-81a6-436a-85c7-f75870b9c6c1 req-5e2023a2-ca79-47a8-8a80-690b7d22dd44 service nova] Acquiring lock "refresh_cache-585c81a5-ac24-4d12-a8a6-1e7b12a2d53a" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:33:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d2d403fd-81a6-436a-85c7-f75870b9c6c1 req-5e2023a2-ca79-47a8-8a80-690b7d22dd44 service nova] Acquired lock "refresh_cache-585c81a5-ac24-4d12-a8a6-1e7b12a2d53a" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:33:05 user nova-compute[72351]: DEBUG nova.network.neutron [req-d2d403fd-81a6-436a-85c7-f75870b9c6c1 req-5e2023a2-ca79-47a8-8a80-690b7d22dd44 service nova] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Refreshing network info cache for port e521fae4-093d-4c92-a32d-684c1a5137f5 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:33:05 user nova-compute[72351]: DEBUG nova.network.neutron [req-6ecb4c35-33f4-4fed-8011-bde96da95a88 req-d863b990-c44f-45c5-b0c1-17d3f484e8b9 service nova] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Updated VIF entry in instance network info cache for port a83afb1f-511f-406c-b925-eb70b32bed90. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:33:05 user nova-compute[72351]: DEBUG nova.network.neutron [req-6ecb4c35-33f4-4fed-8011-bde96da95a88 req-d863b990-c44f-45c5-b0c1-17d3f484e8b9 service nova] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Updating instance_info_cache with network_info: [{"id": "a83afb1f-511f-406c-b925-eb70b32bed90", "address": "fa:16:3e:16:83:e9", "network": {"id": "0b3a34b6-6d32-4414-9135-22bd2a1dae20", "bridge": null, "label": "tempest-TestShelveInstance-2024538322-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.128", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4d0cc057c220422a9541bcb9fa7f598b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "unbound", "details": {}, "devname": "tapa83afb1f-51", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:33:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-6ecb4c35-33f4-4fed-8011-bde96da95a88 req-d863b990-c44f-45c5-b0c1-17d3f484e8b9 service nova] Releasing lock "refresh_cache-8e970bdd-54a1-4058-b09d-784cb6b5b1a4" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:33:05 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._run_pending_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:33:05 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Cleaning up deleted instances {{(pid=72351) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11129}}
Aug 09 02:33:05 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] There are 0 instances to clean {{(pid=72351) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11138}}
Aug 09 02:33:06 user nova-compute[72351]: DEBUG nova.network.neutron [req-d2d403fd-81a6-436a-85c7-f75870b9c6c1 req-5e2023a2-ca79-47a8-8a80-690b7d22dd44 service nova] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Updated VIF entry in instance network info cache for port e521fae4-093d-4c92-a32d-684c1a5137f5. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:33:06 user nova-compute[72351]: DEBUG nova.network.neutron [req-d2d403fd-81a6-436a-85c7-f75870b9c6c1 req-5e2023a2-ca79-47a8-8a80-690b7d22dd44 service nova] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Updating instance_info_cache with network_info: [{"id": "e521fae4-093d-4c92-a32d-684c1a5137f5", "address": "fa:16:3e:e5:b0:10", "network": {"id": "b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-210689817-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "69d584c230b94ff8aae722eb2f364c5c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape521fae4-09", "ovs_interfaceid": "e521fae4-093d-4c92-a32d-684c1a5137f5", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:33:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d2d403fd-81a6-436a-85c7-f75870b9c6c1 req-5e2023a2-ca79-47a8-8a80-690b7d22dd44 service nova] Releasing lock "refresh_cache-585c81a5-ac24-4d12-a8a6-1e7b12a2d53a" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:33:06 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:06 user nova-compute[72351]: INFO nova.scheduler.client.report [None req-bfb0caa3-0059-49d2-b0de-3737426601f8 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Deleted allocations for instance 8e970bdd-54a1-4058-b09d-784cb6b5b1a4
Aug 09 02:33:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-bfb0caa3-0059-49d2-b0de-3737426601f8 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:33:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-bfb0caa3-0059-49d2-b0de-3737426601f8 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:33:06 user nova-compute[72351]: DEBUG nova.objects.instance [None req-bfb0caa3-0059-49d2-b0de-3737426601f8 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Lazy-loading 'numa_topology' on Instance uuid 8e970bdd-54a1-4058-b09d-784cb6b5b1a4 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:33:06 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-bfb0caa3-0059-49d2-b0de-3737426601f8 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:33:06 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-bfb0caa3-0059-49d2-b0de-3737426601f8 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:33:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-bfb0caa3-0059-49d2-b0de-3737426601f8 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.524s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:33:06 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:33:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:33:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:33:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:33:06 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Auditing locally available compute resources for user (node: user) {{(pid=72351) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Aug 09 02:33:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-bfb0caa3-0059-49d2-b0de-3737426601f8 tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Lock "8e970bdd-54a1-4058-b09d-784cb6b5b1a4" "released" by "nova.compute.manager.ComputeManager.shelve_offload_instance.<locals>.do_shelve_offload_instance" :: held 5.562s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:33:07 user nova-compute[72351]: DEBUG nova.compute.manager [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Stashing vm_state: active {{(pid=72351) _prep_resize /opt/stack/nova/nova/compute/manager.py:5535}}
Aug 09 02:33:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:07 user nova-compute[72351]: DEBUG nova.compute.manager [req-377908a7-26fe-4919-aaea-a9573a83e342 req-215740b8-f00e-423d-a8f0-e4d7afac381f service nova] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Received event network-changed-fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:33:07 user nova-compute[72351]: DEBUG nova.compute.manager [req-377908a7-26fe-4919-aaea-a9573a83e342 req-215740b8-f00e-423d-a8f0-e4d7afac381f service nova] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Refreshing instance network info cache due to event network-changed-fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:33:07 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-377908a7-26fe-4919-aaea-a9573a83e342 req-215740b8-f00e-423d-a8f0-e4d7afac381f service nova] Acquiring lock "refresh_cache-80e87454-90b9-4168-bf00-b7d93b245526" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:33:07 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-377908a7-26fe-4919-aaea-a9573a83e342 req-215740b8-f00e-423d-a8f0-e4d7afac381f service nova] Acquired lock "refresh_cache-80e87454-90b9-4168-bf00-b7d93b245526" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:33:07 user nova-compute[72351]: DEBUG nova.network.neutron [req-377908a7-26fe-4919-aaea-a9573a83e342 req-215740b8-f00e-423d-a8f0-e4d7afac381f service nova] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Refreshing network info cache for port fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:33:07 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.resize_claim" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:33:07 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.resize_claim" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:33:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json" returned: 0 in 0.160s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:07 user nova-compute[72351]: DEBUG nova.objects.instance [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lazy-loading 'pci_requests' on Instance uuid 80e87454-90b9-4168-bf00-b7d93b245526 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:33:07 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72351) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Aug 09 02:33:07 user nova-compute[72351]: INFO nova.compute.claims [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Claim successful on node user
Aug 09 02:33:07 user nova-compute[72351]: DEBUG nova.objects.instance [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lazy-loading 'resources' on Instance uuid 80e87454-90b9-4168-bf00-b7d93b245526 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:33:07 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5fa8fa48-a3c2-44c3-b0a0-40dafb632eff tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Acquiring lock "f37b7fb7-1b94-421b-be79-7066010efabf" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:33:07 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5fa8fa48-a3c2-44c3-b0a0-40dafb632eff tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lock "f37b7fb7-1b94-421b-be79-7066010efabf" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:33:07 user nova-compute[72351]: DEBUG nova.objects.instance [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lazy-loading 'pci_devices' on Instance uuid 80e87454-90b9-4168-bf00-b7d93b245526 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:33:07 user nova-compute[72351]: INFO nova.compute.manager [None req-5fa8fa48-a3c2-44c3-b0a0-40dafb632eff tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Detaching volume 075fce6b-3efd-452d-886d-22cd05c7e146
Aug 09 02:33:07 user nova-compute[72351]: INFO nova.compute.resource_tracker [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Updating resource usage from migration 5464193d-df16-4ccd-b137-4df0859f7495
Aug 09 02:33:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json" returned: 0 in 0.196s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:07 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdg (vda) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:33:07 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdh (vda) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:33:07 user nova-compute[72351]: INFO nova.virt.block_device [None req-5fa8fa48-a3c2-44c3-b0a0-40dafb632eff tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Attempting to driver detach volume 075fce6b-3efd-452d-886d-22cd05c7e146 from mountpoint /dev/vdb
Aug 09 02:33:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/47afce95-e5f7-4703-a99f-7ef8b28c4bd2/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:07 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-5fa8fa48-a3c2-44c3-b0a0-40dafb632eff tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Attempting to detach device vdb from instance f37b7fb7-1b94-421b-be79-7066010efabf from the persistent domain config. {{(pid=72351) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2477}}
Aug 09 02:33:07 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-5fa8fa48-a3c2-44c3-b0a0-40dafb632eff tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 02:33:07 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:33:07 user nova-compute[72351]:   <source dev="/dev/sdd"/>
Aug 09 02:33:07 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:33:07 user nova-compute[72351]:   <serial>075fce6b-3efd-452d-886d-22cd05c7e146</serial>
Aug 09 02:33:07 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Aug 09 02:33:07 user nova-compute[72351]: </disk>
Aug 09 02:33:07 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 02:33:07 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-5fa8fa48-a3c2-44c3-b0a0-40dafb632eff tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Successfully detached device vdb from instance f37b7fb7-1b94-421b-be79-7066010efabf from the persistent domain config.
Aug 09 02:33:07 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-5fa8fa48-a3c2-44c3-b0a0-40dafb632eff tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance f37b7fb7-1b94-421b-be79-7066010efabf from the live domain config. {{(pid=72351) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2513}}
Aug 09 02:33:07 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-5fa8fa48-a3c2-44c3-b0a0-40dafb632eff tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 02:33:07 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:33:07 user nova-compute[72351]:   <source dev="/dev/sdd"/>
Aug 09 02:33:07 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:33:07 user nova-compute[72351]:   <serial>075fce6b-3efd-452d-886d-22cd05c7e146</serial>
Aug 09 02:33:07 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Aug 09 02:33:07 user nova-compute[72351]: </disk>
Aug 09 02:33:07 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 02:33:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/47afce95-e5f7-4703-a99f-7ef8b28c4bd2/disk --force-share --output=json" returned: 0 in 0.166s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/47afce95-e5f7-4703-a99f-7ef8b28c4bd2/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/47afce95-e5f7-4703-a99f-7ef8b28c4bd2/disk --force-share --output=json" returned: 0 in 0.148s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.rescue --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG nova.network.neutron [req-377908a7-26fe-4919-aaea-a9573a83e342 req-215740b8-f00e-423d-a8f0-e4d7afac381f service nova] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Updated VIF entry in instance network info cache for port fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG nova.network.neutron [req-377908a7-26fe-4919-aaea-a9573a83e342 req-215740b8-f00e-423d-a8f0-e4d7afac381f service nova] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Updating instance_info_cache with network_info: [{"id": "fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4", "address": "fa:16:3e:7a:77:69", "network": {"id": "b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-210689817-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.41", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "69d584c230b94ff8aae722eb2f364c5c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfcb0ceaf-ff", "ovs_interfaceid": "fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-377908a7-26fe-4919-aaea-a9573a83e342 req-215740b8-f00e-423d-a8f0-e4d7afac381f service nova] Releasing lock "refresh_cache-80e87454-90b9-4168-bf00-b7d93b245526" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.rescue --force-share --output=json" returned: 0 in 0.138s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.rescue --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Received event <DeviceRemovedEvent: 1691548388.1955774, f37b7fb7-1b94-421b-be79-7066010efabf => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=72351) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2360}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-5fa8fa48-a3c2-44c3-b0a0-40dafb632eff tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance f37b7fb7-1b94-421b-be79-7066010efabf {{(pid=72351) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2589}}
Aug 09 02:33:08 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-5fa8fa48-a3c2-44c3-b0a0-40dafb632eff tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Successfully detached device vdb from instance f37b7fb7-1b94-421b-be79-7066010efabf from the live domain config.
Aug 09 02:33:08 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-5fa8fa48-a3c2-44c3-b0a0-40dafb632eff tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5fa8fa48-a3c2-44c3-b0a0-40dafb632eff tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:075fce6b-3efd-452d-886d-22cd05c7e146'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'BJOXsaHo', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdd'}, None), 'kwargs': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:33:08 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-5fa8fa48-a3c2-44c3-b0a0-40dafb632eff tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:33:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-5fa8fa48-a3c2-44c3-b0a0-40dafb632eff tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-5fa8fa48-a3c2-44c3-b0a0-40dafb632eff tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5fa8fa48-a3c2-44c3-b0a0-40dafb632eff tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:075fce6b-3efd-452d-886d-22cd05c7e146', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.025s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[178addfa-29cc-46db-bb49-4fbd20b0e319]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:075fce6b-3efd-452d-886d-22cd05c7e146\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.033s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[c98755c8-b12e-4730-b528-84200f7e9e69]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)\ntcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)\ntcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc (non-flash)\ntcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:075fce6b-3efd-452d-886d-22cd05c7e146 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5fa8fa48-a3c2-44c3-b0a0-40dafb632eff tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:33:08 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:33:08 user nova-compute[72351]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:33:08 user nova-compute[72351]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)
Aug 09 02:33:08 user nova-compute[72351]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc (non-flash)
Aug 09 02:33:08 user nova-compute[72351]: tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:075fce6b-3efd-452d-886d-22cd05c7e146 (non-flash)
Aug 09 02:33:08 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5fa8fa48-a3c2-44c3-b0a0-40dafb632eff tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:33:08 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:33:08 user nova-compute[72351]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:33:08 user nova-compute[72351]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)
Aug 09 02:33:08 user nova-compute[72351]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc (non-flash)
Aug 09 02:33:08 user nova-compute[72351]: tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:075fce6b-3efd-452d-886d-22cd05c7e146 (non-flash)
Aug 09 02:33:08 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5fa8fa48-a3c2-44c3-b0a0-40dafb632eff tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fcfbc706d40>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:075fce6b-3efd-452d-886d-22cd05c7e146'): ({'sdd'}, set())}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-5fa8fa48-a3c2-44c3-b0a0-40dafb632eff tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Removing single pathed devices sdd {{(pid=72351) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.rescue --force-share --output=json" returned: 0 in 0.166s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[cd567f8c-3998-4519-b2bd-e768223a9ed9]: (4, ('path checker states:\nup                  6\n\npaths: 0\nbusy: False\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdd {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdd" returned: 0 in 0.024s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ff22e833-f7bf-4572-8e75-78b6207d8e6e]: (4, ('ok\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-5fa8fa48-a3c2-44c3-b0a0-40dafb632eff tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Flushing IO for device /dev/sdd {{(pid=72351) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdd {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdd" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[0e81aa25-942c-4e38-afe3-5bf8d3a36ff8]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-5fa8fa48-a3c2-44c3-b0a0-40dafb632eff tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Remove SCSI device /dev/sdd with /sys/block/sdd/device/delete {{(pid=72351) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdd/device/delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json" returned: 0 in 0.166s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdd/device/delete" returned: 0 in 0.039s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[5fb0245d-ff3e-4a69-abdc-68f9c2c5915f]: (4, ('1', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.resize_claim" :: held 1.089s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:33:08 user nova-compute[72351]: INFO nova.compute.manager [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Migrating
Aug 09 02:33:08 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-5fa8fa48-a3c2-44c3-b0a0-40dafb632eff tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Checking to see if SCSI volumes sdd have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-5fa8fa48-a3c2-44c3-b0a0-40dafb632eff tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] SCSI volumes sdd have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5fa8fa48-a3c2-44c3-b0a0-40dafb632eff tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:075fce6b-3efd-452d-886d-22cd05c7e146')] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:075fce6b-3efd-452d-886d-22cd05c7e146 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:075fce6b-3efd-452d-886d-22cd05c7e146 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.031s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[4361b984-55f2-407d-9937-f81cfb2da939]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Acquiring lock "refresh_cache-80e87454-90b9-4168-bf00-b7d93b245526" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Acquired lock "refresh_cache-80e87454-90b9-4168-bf00-b7d93b245526" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG nova.network.neutron [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5fa8fa48-a3c2-44c3-b0a0-40dafb632eff tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:075fce6b-3efd-452d-886d-22cd05c7e146 -p 172.16.0.220:3260 --logout {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json" returned: 0 in 0.195s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:075fce6b-3efd-452d-886d-22cd05c7e146 -p 172.16.0.220:3260 --logout" returned: 0 in 0.072s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[7161dd3c-c430-4300-9e26-6fe39337a371]: (4, ('Logging out of session [sid: 42, target: iqn.2010-10.org.openstack:075fce6b-3efd-452d-886d-22cd05c7e146, portal: 172.16.0.220,3260]\nLogout of [sid: 42, target: iqn.2010-10.org.openstack:075fce6b-3efd-452d-886d-22cd05c7e146, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5fa8fa48-a3c2-44c3-b0a0-40dafb632eff tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 42, target: iqn.2010-10.org.openstack:075fce6b-3efd-452d-886d-22cd05c7e146, portal: 172.16.0.220,3260]
Aug 09 02:33:08 user nova-compute[72351]: Logout of [sid: 42, target: iqn.2010-10.org.openstack:075fce6b-3efd-452d-886d-22cd05c7e146, portal: 172.16.0.220,3260] successful.
Aug 09 02:33:08 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:075fce6b-3efd-452d-886d-22cd05c7e146 -p 172.16.0.220:3260 --op delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:075fce6b-3efd-452d-886d-22cd05c7e146 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.033s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[91b5242f-2a8b-466d-8f1e-af8962374bca]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5fa8fa48-a3c2-44c3-b0a0-40dafb632eff tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-5fa8fa48-a3c2-44c3-b0a0-40dafb632eff tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.558s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-5fa8fa48-a3c2-44c3-b0a0-40dafb632eff tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] <== disconnect_volume: return (560ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-5fa8fa48-a3c2-44c3-b0a0-40dafb632eff tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json" returned: 0 in 0.155s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json" returned: 0 in 0.154s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:09 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdf (vda) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:33:09 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/41b5e305-3973-4937-b262-d32db1091e20/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:09 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/41b5e305-3973-4937-b262-d32db1091e20/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:09 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/41b5e305-3973-4937-b262-d32db1091e20/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:09 user nova-compute[72351]: DEBUG nova.network.neutron [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Updating instance_info_cache with network_info: [{"id": "fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4", "address": "fa:16:3e:7a:77:69", "network": {"id": "b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-210689817-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.41", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "69d584c230b94ff8aae722eb2f364c5c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfcb0ceaf-ff", "ovs_interfaceid": "fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:33:09 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Releasing lock "refresh_cache-80e87454-90b9-4168-bf00-b7d93b245526" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:33:09 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/41b5e305-3973-4937-b262-d32db1091e20/disk --force-share --output=json" returned: 0 in 0.176s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:09 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:09 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Starting migrate_disk_and_power_off {{(pid=72351) migrate_disk_and_power_off /opt/stack/nova/nova/virt/libvirt/driver.py:11487}}
Aug 09 02:33:09 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json" returned: 0 in 0.185s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:09 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:09 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json" returned: 0 in 0.193s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:09 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7b6f5d60-5f23-4ef1-b972-9ba7086f56cb/disk.rescue --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:09 user nova-compute[72351]: DEBUG nova.objects.instance [None req-5fa8fa48-a3c2-44c3-b0a0-40dafb632eff tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lazy-loading 'flavor' on Instance uuid f37b7fb7-1b94-421b-be79-7066010efabf {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:33:09 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5fa8fa48-a3c2-44c3-b0a0-40dafb632eff tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lock "f37b7fb7-1b94-421b-be79-7066010efabf" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 2.334s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:33:09 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7b6f5d60-5f23-4ef1-b972-9ba7086f56cb/disk.rescue --force-share --output=json" returned: 0 in 0.157s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:09 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7b6f5d60-5f23-4ef1-b972-9ba7086f56cb/disk.rescue --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:09 user nova-compute[72351]: DEBUG nova.compute.manager [req-e15b6876-d68a-479e-a186-424e2724c858 req-ce430720-f599-4056-8da9-5fbcc086a6cd service nova] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Received event network-vif-unplugged-fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:33:09 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-e15b6876-d68a-479e-a186-424e2724c858 req-ce430720-f599-4056-8da9-5fbcc086a6cd service nova] Acquiring lock "80e87454-90b9-4168-bf00-b7d93b245526-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:33:09 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-e15b6876-d68a-479e-a186-424e2724c858 req-ce430720-f599-4056-8da9-5fbcc086a6cd service nova] Lock "80e87454-90b9-4168-bf00-b7d93b245526-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:33:09 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-e15b6876-d68a-479e-a186-424e2724c858 req-ce430720-f599-4056-8da9-5fbcc086a6cd service nova] Lock "80e87454-90b9-4168-bf00-b7d93b245526-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:33:09 user nova-compute[72351]: DEBUG nova.compute.manager [req-e15b6876-d68a-479e-a186-424e2724c858 req-ce430720-f599-4056-8da9-5fbcc086a6cd service nova] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] No waiting events found dispatching network-vif-unplugged-fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:33:09 user nova-compute[72351]: WARNING nova.compute.manager [req-e15b6876-d68a-479e-a186-424e2724c858 req-ce430720-f599-4056-8da9-5fbcc086a6cd service nova] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Received unexpected event network-vif-unplugged-fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4 for instance with vm_state active and task_state resize_migrating.
Aug 09 02:33:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7b6f5d60-5f23-4ef1-b972-9ba7086f56cb/disk.rescue --force-share --output=json" returned: 0 in 0.166s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /opt/stack/data/nova/instances/7b6f5d60-5f23-4ef1-b972-9ba7086f56cb/disk (vdb) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7bef74f4-3bb3-4ad0-bff7-34156be31249/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7bef74f4-3bb3-4ad0-bff7-34156be31249/disk --force-share --output=json" returned: 0 in 0.147s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7bef74f4-3bb3-4ad0-bff7-34156be31249/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:10 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Instance destroyed successfully.
Aug 09 02:33:10 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:31:13Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestOtherA-server-2126139398',display_name='tempest-ServerActionsTestOtherA-server-2126139398',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serveractionstestothera-server-2126139398',id=38,image_ref='',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBB4HNXglactRKXsG6uozX3XR7A39DBllYpgpCz+5+1RhfLOeXdmVkNfDkIG7vQrgoVNKWK/MwqtjPDLEVURcnWQfOuv4ySYdfEPj4hs6FpDqfJbAa5qX0FAbZQuEg69FVQ==',key_name='tempest-keypair-976839412',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:31:39Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=MigrationContext,new_flavor=Flavor(7),node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='69d584c230b94ff8aae722eb2f364c5c',ramdisk_id='',reservation_id='r-97d5c06t',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=ServiceList,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',old_vm_state='active',owner_project_name='tempest-ServerActionsTestOtherA-1303549305',owner_user_name='tempest-ServerActionsTestOtherA-1303549305-project-member'},tags=<?>,task_state='resize_migrating',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:33:07Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='5dca34fdadbb4386a1ff1cb255265d47',uuid=80e87454-90b9-4168-bf00-b7d93b245526,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4", "address": "fa:16:3e:7a:77:69", "network": {"id": "b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-210689817-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.41", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestOtherA-210689817-network", "vif_mac": "fa:16:3e:7a:77:69"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "69d584c230b94ff8aae722eb2f364c5c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfcb0ceaf-ff", "ovs_interfaceid": "fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Converting VIF {"id": "fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4", "address": "fa:16:3e:7a:77:69", "network": {"id": "b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-210689817-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.41", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestOtherA-210689817-network", "vif_mac": "fa:16:3e:7a:77:69"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "69d584c230b94ff8aae722eb2f364c5c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfcb0ceaf-ff", "ovs_interfaceid": "fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:7a:77:69,bridge_name='br-int',has_traffic_filtering=True,id=fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4,network=Network(b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfcb0ceaf-ff') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG os_vif [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:7a:77:69,bridge_name='br-int',has_traffic_filtering=True,id=fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4,network=Network(b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfcb0ceaf-ff') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapfcb0ceaf-ff, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:33:10 user nova-compute[72351]: INFO os_vif [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:7a:77:69,bridge_name='br-int',has_traffic_filtering=True,id=fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4,network=Network(b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfcb0ceaf-ff')
Aug 09 02:33:10 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'oxcrdOpE', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdh'}, None), 'kwargs': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:33:10 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:33:10 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7bef74f4-3bb3-4ad0-bff7-34156be31249/disk --force-share --output=json" returned: 0 in 0.152s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[61dd88c4-d135-4c98-aee5-a72f37ee6cd1]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a03fbded-23ad-440b-97e7-bea07425e873/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.033s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[17b3d4a8-8a90-4d16-8483-b9f82c495b7a]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)\ntcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)\ntcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:33:10 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:33:10 user nova-compute[72351]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:33:10 user nova-compute[72351]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)
Aug 09 02:33:10 user nova-compute[72351]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc (non-flash)
Aug 09 02:33:10 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:33:10 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:33:10 user nova-compute[72351]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:33:10 user nova-compute[72351]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)
Aug 09 02:33:10 user nova-compute[72351]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc (non-flash)
Aug 09 02:33:10 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fcffc1b6290>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238'): ({'sdh'}, set())}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Removing single pathed devices sdh {{(pid=72351) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.031s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[632ee40f-b4a3-455b-8a0e-81dbb5af2c29]: (4, ('path checker states:\nup                  5\n\npaths: 0\nbusy: False\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdh {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdh" returned: 0 in 0.024s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[b69ff7c6-16ba-4731-a716-32569bf2ba22]: (4, ('ok\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Flushing IO for device /dev/sdh {{(pid=72351) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdh {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdh" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[8a35b279-a43a-4445-ae20-61a69a4182fc]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Remove SCSI device /dev/sdh with /sys/block/sdh/device/delete {{(pid=72351) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdh/device/delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a03fbded-23ad-440b-97e7-bea07425e873/disk --force-share --output=json" returned: 0 in 0.190s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a03fbded-23ad-440b-97e7-bea07425e873/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdh/device/delete" returned: 0 in 0.078s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[1069df41-18cf-46b0-880c-f1f968a8d8ff]: (4, ('1', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Checking to see if SCSI volumes sdh have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] SCSI volumes sdh have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG os_brick.privileged.rootwrap [-] Unlinking ('/dev/disk/by-id/scsi-23832643035363835', '/dev/disk/by-id/scsi-1SCST_BIOd262e66a-82d056853fcbd2b5', '/dev/disk/by-id/scsi-SSCST_BIO_Storage_d262e66a') {{(pid=83366) unlink_root /usr/local/lib/python3.10/dist-packages/os_brick/privileged/rootwrap.py:219}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[c3d050c2-899d-47c1-8516-3ee92e35fbe3]: (4, None) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238')] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[831e6774-9ad5-4eca-a41e-219df3be2dcb]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 -p 172.16.0.220:3260 --logout {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 -p 172.16.0.220:3260 --logout" returned: 0 in 0.081s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[b0241fb1-ad21-41b9-b8f8-7234cf781bf9]: (4, ('Logging out of session [sid: 39, target: iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238, portal: 172.16.0.220,3260]\nLogout of [sid: 39, target: iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 39, target: iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238, portal: 172.16.0.220,3260]
Aug 09 02:33:10 user nova-compute[72351]: Logout of [sid: 39, target: iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238, portal: 172.16.0.220,3260] successful.
Aug 09 02:33:10 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 -p 172.16.0.220:3260 --op delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.028s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a5282d88-e8dc-4f5b-bcdf-728c981c5899]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.460s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] <== disconnect_volume: return (463ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a03fbded-23ad-440b-97e7-bea07425e873/disk --force-share --output=json" returned: 0 in 0.224s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] skipping disk /dev/sdh (vda) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:33:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f37b7fb7-1b94-421b-be79-7066010efabf/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f37b7fb7-1b94-421b-be79-7066010efabf/disk --force-share --output=json" returned: 0 in 0.179s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f37b7fb7-1b94-421b-be79-7066010efabf/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f37b7fb7-1b94-421b-be79-7066010efabf/disk --force-share --output=json" returned: 0 in 0.153s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:11 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdc (vda) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:33:11 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:33:11 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:33:11 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Hypervisor/Node resource view: name=user free_ram=6863MB free_disk=25.058570861816406GB free_vcpus=-2 pci_devices=[{"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72351) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Aug 09 02:33:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:33:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:33:12 user nova-compute[72351]: DEBUG nova.compute.manager [req-c37b1222-982b-48e1-967a-36b2d00c7f0b req-36c785b6-0062-44b5-8614-cc161a7583e8 service nova] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Received event network-vif-plugged-fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:33:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c37b1222-982b-48e1-967a-36b2d00c7f0b req-36c785b6-0062-44b5-8614-cc161a7583e8 service nova] Acquiring lock "80e87454-90b9-4168-bf00-b7d93b245526-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:33:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c37b1222-982b-48e1-967a-36b2d00c7f0b req-36c785b6-0062-44b5-8614-cc161a7583e8 service nova] Lock "80e87454-90b9-4168-bf00-b7d93b245526-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:33:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c37b1222-982b-48e1-967a-36b2d00c7f0b req-36c785b6-0062-44b5-8614-cc161a7583e8 service nova] Lock "80e87454-90b9-4168-bf00-b7d93b245526-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:33:12 user nova-compute[72351]: DEBUG nova.compute.manager [req-c37b1222-982b-48e1-967a-36b2d00c7f0b req-36c785b6-0062-44b5-8614-cc161a7583e8 service nova] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] No waiting events found dispatching network-vif-plugged-fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:33:12 user nova-compute[72351]: WARNING nova.compute.manager [req-c37b1222-982b-48e1-967a-36b2d00c7f0b req-36c785b6-0062-44b5-8614-cc161a7583e8 service nova] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Received unexpected event network-vif-plugged-fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4 for instance with vm_state active and task_state resize_migrating.
Aug 09 02:33:12 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Applying migration context for instance 80e87454-90b9-4168-bf00-b7d93b245526 as it has an incoming, in-progress migration 5464193d-df16-4ccd-b137-4df0859f7495. Migration status is migrating {{(pid=72351) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1016}}
Aug 09 02:33:12 user nova-compute[72351]: INFO nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Updating resource usage from migration 5464193d-df16-4ccd-b137-4df0859f7495
Aug 09 02:33:12 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:33:12 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance eb86cb49-2018-42c2-bd97-c662b829775d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:33:12 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:33:12 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance a7f3d076-864b-4b68-9207-214ce2c21444 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:33:12 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 47afce95-e5f7-4703-a99f-7ef8b28c4bd2 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:33:12 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 957c1510-1320-4492-9052-a58ff540319d actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:33:12 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance a03fbded-23ad-440b-97e7-bea07425e873 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:33:12 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:33:12 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 49a10075-0d3c-4cd6-bb43-10238cad3f37 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:33:12 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance f37b7fb7-1b94-421b-be79-7066010efabf actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:33:12 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 7bef74f4-3bb3-4ad0-bff7-34156be31249 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:33:12 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 41b5e305-3973-4937-b262-d32db1091e20 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:33:12 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:33:12 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Migration 5464193d-df16-4ccd-b137-4df0859f7495 is active on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1709}}
Aug 09 02:33:12 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 80e87454-90b9-4168-bf00-b7d93b245526 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 2048, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:33:12 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Total usable vcpus: 12, total allocated vcpus: 15 {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Aug 09 02:33:12 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Final resource view: name=user phys_ram=16011MB used_ram=9728MB phys_disk=40GB used_disk=10GB total_vcpus=12 used_vcpus=15 pci_stats=[] {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Aug 09 02:33:12 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:33:12 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:33:13 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Compute_service record updated for user:user {{(pid=72351) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Aug 09 02:33:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.810s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:33:13 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._cleanup_incomplete_migrations {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:33:13 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Cleaning up deleted instances with incomplete migration  {{(pid=72351) _cleanup_incomplete_migrations /opt/stack/nova/nova/compute/manager.py:11167}}
Aug 09 02:33:13 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._cleanup_expired_console_auth_tokens {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:33:13 user nova-compute[72351]: DEBUG nova.network.neutron [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Port fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4 binding to destination host user is already ACTIVE {{(pid=72351) migrate_instance_start /opt/stack/nova/nova/network/neutron.py:3163}}
Aug 09 02:33:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Acquiring lock "80e87454-90b9-4168-bf00-b7d93b245526-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:33:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "80e87454-90b9-4168-bf00-b7d93b245526-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:33:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "80e87454-90b9-4168-bf00-b7d93b245526-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:33:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Acquiring lock "refresh_cache-80e87454-90b9-4168-bf00-b7d93b245526" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:33:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Acquired lock "refresh_cache-80e87454-90b9-4168-bf00-b7d93b245526" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:33:13 user nova-compute[72351]: DEBUG nova.network.neutron [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:33:13 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:33:13 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:33:13 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:33:13 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Starting heal instance info cache {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9842}}
Aug 09 02:33:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "refresh_cache-a7f3d076-864b-4b68-9207-214ce2c21444" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:33:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquired lock "refresh_cache-a7f3d076-864b-4b68-9207-214ce2c21444" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:33:13 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Forcefully refreshing network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1996}}
Aug 09 02:33:14 user nova-compute[72351]: DEBUG nova.network.neutron [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Updating instance_info_cache with network_info: [{"id": "fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4", "address": "fa:16:3e:7a:77:69", "network": {"id": "b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-210689817-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.41", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "69d584c230b94ff8aae722eb2f364c5c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfcb0ceaf-ff", "ovs_interfaceid": "fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:33:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Releasing lock "refresh_cache-80e87454-90b9-4168-bf00-b7d93b245526" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:33:14 user nova-compute[72351]: DEBUG os_brick.utils [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:33:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.016s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:14 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[cd844fe2-9a44-4a6c-8dfd-555dc7ea93f0]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:a52269602ff4\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.016s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:14 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d7ebdac9-f8d3-4e6f-b0c7-61a96f934971]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:14 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[5cb4a9c8-0b21-4f1f-82fa-f03dea3530ce]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:14 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[01dc6939-0108-4c43-8681-303b0dfd448f]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Running cmd (subprocess): nvme version {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] 'nvme version' failed. Not Retrying. {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:33:14 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.nvmeof [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] nvme not present on system {{(pid=72351) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Aug 09 02:33:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:33:14 user nova-compute[72351]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Aug 09 02:33:14 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[f9fe276b-c50c-492a-89ab-31e20f98a16d]: (4, '') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:14 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72351) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Aug 09 02:33:14 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Aug 09 02:33:14 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] LIGHTOS: no hostnqn found. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Aug 09 02:33:14 user nova-compute[72351]: DEBUG os_brick.utils [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] <== get_connector_properties: return (128ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:a52269602ff4', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:33:14 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Updating instance_info_cache with network_info: [{"id": "64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f", "address": "fa:16:3e:ed:9c:88", "network": {"id": "9a452414-d61f-4045-a6cc-9be93c892c4c", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "321111f935dd4e8aab4a083206ea9a7e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap64bd80e2-6c", "ovs_interfaceid": "64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:33:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Releasing lock "refresh_cache-a7f3d076-864b-4b68-9207-214ce2c21444" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:33:14 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Updated the network info_cache for instance {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9913}}
Aug 09 02:33:14 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:33:14 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:33:14 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:33:14 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:33:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:15 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:17 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691548382.2113454, 8e970bdd-54a1-4058-b09d-784cb6b5b1a4 => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:33:17 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] VM Stopped (Lifecycle Event)
Aug 09 02:33:17 user nova-compute[72351]: DEBUG nova.compute.manager [None req-f1531886-1c0d-4639-a1b9-a364c6d8dbac None None] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:33:17 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Starting finish_migration {{(pid=72351) finish_migration /opt/stack/nova/nova/virt/libvirt/driver.py:11674}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Instance directory exists: not creating {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4713}}
Aug 09 02:33:23 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Creating image(s)
Aug 09 02:33:23 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Did not create local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4853}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Ensure instance console log exists: /opt/stack/data/nova/instances/80e87454-90b9-4168-bf00-b7d93b245526/console.log {{(pid=72351) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4603}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Start _get_guest_xml network_info=[{"id": "fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4", "address": "fa:16:3e:7a:77:69", "network": {"id": "b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-210689817-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.41", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestOtherA-210689817-network", "vif_mac": "fa:16:3e:7a:77:69"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "69d584c230b94ff8aae722eb2f364c5c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfcb0ceaf-ff", "ovs_interfaceid": "fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'device_type': 'disk', 'guest_format': None, 'disk_bus': 'virtio', 'mount_device': '/dev/vda', 'boot_index': 0, 'attachment_id': 'bb155c17-8df0-4385-b599-f2edf16afbb0', 'delete_on_termination': True, 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqns': ['iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'oxcrdOpE', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'attaching', 'instance': '80e87454-90b9-4168-bf00-b7d93b245526', 'attached_at': '2023-08-09T02:33:23.000000', 'detached_at': '', 'volume_id': '7e32fe78-f2c4-409c-813a-36924e611238', 'serial': '7e32fe78-f2c4-409c-813a-36924e611238'}, 'volume_type': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:33:23 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:33:23 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:33:23 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:06Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='2',id=7,is_public=True,memory_mb=2048,name='m1.small',projects=<?>,root_gb=20,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG nova.objects.instance [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lazy-loading 'vcpu_model' on Instance uuid 80e87454-90b9-4168-bf00-b7d93b245526 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'oxcrdOpE', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:33:23 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:33:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:33:23 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:33:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:33:23 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 02:33:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 -p 172.16.0.220:3260" returned: 21 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[54335ca2-63ba-4d7e-89de-ab4015cff1c0]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Aug 09 02:33:23 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 -p 172.16.0.220:3260 --interface default --op new {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[5215e3ba-263e-463a-b2b4-e9c76af410a3]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238] added\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238] added
Aug 09 02:33:23 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[c2fce6c9-8aca-401f-a39d-c3f170266112]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[202268d5-05a9-4dcd-bd3b-4e90261bb88b]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v oxcrdOpE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v oxcrdOpE" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[6c57f45b-97ea-453e-8944-63206a10bdec]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'oxcrdOpE'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[f4bca899-45a3-4114-9692-46e6e8044fd2]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[f4d0af53-9239-47a0-91ea-b75b9f54b0e5]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)\ntcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:33:23 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:33:23 user nova-compute[72351]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)
Aug 09 02:33:23 user nova-compute[72351]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc (non-flash)
Aug 09 02:33:23 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:33:23 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:33:23 user nova-compute[72351]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)
Aug 09 02:33:23 user nova-compute[72351]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc (non-flash)
Aug 09 02:33:23 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 -p 172.16.0.220:3260 --login {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 -p 172.16.0.220:3260 --login" returned: 0 in 0.068s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[9f2b469d-88b6-4ee3-9200-031b8afab6c4]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238, portal: 172.16.0.220,3260]
Aug 09 02:33:23 user nova-compute[72351]: Login to [iface: default, target: iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238, portal: 172.16.0.220,3260] successful.
Aug 09 02:33:23 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[39f774a0-cb76-4da0-825f-197ca1458ac9]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.026s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[b04a5be9-b6f9-4d10-aeba-11fcc3f20c58]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)\ntcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc (non-flash)\ntcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:33:23 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:33:23 user nova-compute[72351]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)
Aug 09 02:33:23 user nova-compute[72351]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc (non-flash)
Aug 09 02:33:23 user nova-compute[72351]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:33:23 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:33:23 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:33:23 user nova-compute[72351]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)
Aug 09 02:33:23 user nova-compute[72351]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc (non-flash)
Aug 09 02:33:23 user nova-compute[72351]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:33:23 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.300s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] HCTL ('35', '-', '-', 0) found on session 43 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Scanning host 35 c: -, t: -, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host35/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host35/scan" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[2b6909e4-1fbe-404d-a8a0-283126643fad]: (4, ('- - 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:23 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Searching for a device in session 43 and hctl ['35', '*', '*', 0] yield: None {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:33:24 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Searching for a device in session 43 and hctl ['35', '*', '*', 0] yield: sdd {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:33:24 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Connected to sdd using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'oxcrdOpE', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 02:33:24 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.331s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:33:24 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] <== connect_volume: return (1332ms) {'type': 'block', 'scsi_wwn': '23832643035363835', 'path': '/dev/sdd'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:33:24 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23832643035363835', 'path': '/dev/sdd'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 02:33:24 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:31:13Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestOtherA-server-2126139398',display_name='tempest-ServerActionsTestOtherA-server-2126139398',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(7),hidden=False,host='user',hostname='tempest-serveractionstestothera-server-2126139398',id=38,image_ref='',info_cache=InstanceInfoCache,instance_type_id=7,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBB4HNXglactRKXsG6uozX3XR7A39DBllYpgpCz+5+1RhfLOeXdmVkNfDkIG7vQrgoVNKWK/MwqtjPDLEVURcnWQfOuv4ySYdfEPj4hs6FpDqfJbAa5qX0FAbZQuEg69FVQ==',key_name='tempest-keypair-976839412',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:31:39Z,launched_on='user',locked=False,locked_by=None,memory_mb=2048,metadata={},migration_context=MigrationContext,new_flavor=Flavor(7),node='user',numa_topology=None,old_flavor=Flavor(6),os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='69d584c230b94ff8aae722eb2f364c5c',ramdisk_id='',reservation_id='r-97d5c06t',resources=None,root_device_name='/dev/vda',root_gb=20,security_groups=SecurityGroupList,services=ServiceList,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',old_vm_state='active',owner_project_name='tempest-ServerActionsTestOtherA-1303549305',owner_user_name='tempest-ServerActionsTestOtherA-1303549305-project-member'},tags=<?>,task_state='resize_finish',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:33:13Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='5dca34fdadbb4386a1ff1cb255265d47',uuid=80e87454-90b9-4168-bf00-b7d93b245526,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4", "address": "fa:16:3e:7a:77:69", "network": {"id": "b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-210689817-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.41", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestOtherA-210689817-network", "vif_mac": "fa:16:3e:7a:77:69"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "69d584c230b94ff8aae722eb2f364c5c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfcb0ceaf-ff", "ovs_interfaceid": "fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:33:24 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Converting VIF {"id": "fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4", "address": "fa:16:3e:7a:77:69", "network": {"id": "b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-210689817-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.41", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestOtherA-210689817-network", "vif_mac": "fa:16:3e:7a:77:69"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "69d584c230b94ff8aae722eb2f364c5c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfcb0ceaf-ff", "ovs_interfaceid": "fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:33:24 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:7a:77:69,bridge_name='br-int',has_traffic_filtering=True,id=fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4,network=Network(b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfcb0ceaf-ff') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:33:24 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:33:24 user nova-compute[72351]:   <uuid>80e87454-90b9-4168-bf00-b7d93b245526</uuid>
Aug 09 02:33:24 user nova-compute[72351]:   <name>instance-00000026</name>
Aug 09 02:33:24 user nova-compute[72351]:   <memory>2097152</memory>
Aug 09 02:33:24 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:33:24 user nova-compute[72351]:   <metadata>
Aug 09 02:33:24 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:33:24 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:33:24 user nova-compute[72351]:       <nova:name>tempest-ServerActionsTestOtherA-server-2126139398</nova:name>
Aug 09 02:33:24 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:33:23</nova:creationTime>
Aug 09 02:33:24 user nova-compute[72351]:       <nova:flavor name="m1.small">
Aug 09 02:33:24 user nova-compute[72351]:         <nova:memory>2048</nova:memory>
Aug 09 02:33:24 user nova-compute[72351]:         <nova:disk>20</nova:disk>
Aug 09 02:33:24 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:33:24 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:33:24 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:33:24 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:33:24 user nova-compute[72351]:       <nova:owner>
Aug 09 02:33:24 user nova-compute[72351]:         <nova:user uuid="5dca34fdadbb4386a1ff1cb255265d47">tempest-ServerActionsTestOtherA-1303549305-project-member</nova:user>
Aug 09 02:33:24 user nova-compute[72351]:         <nova:project uuid="69d584c230b94ff8aae722eb2f364c5c">tempest-ServerActionsTestOtherA-1303549305</nova:project>
Aug 09 02:33:24 user nova-compute[72351]:       </nova:owner>
Aug 09 02:33:24 user nova-compute[72351]:       <nova:ports>
Aug 09 02:33:24 user nova-compute[72351]:         <nova:port uuid="fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4">
Aug 09 02:33:24 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.9" ipVersion="4"/>
Aug 09 02:33:24 user nova-compute[72351]:         </nova:port>
Aug 09 02:33:24 user nova-compute[72351]:       </nova:ports>
Aug 09 02:33:24 user nova-compute[72351]:     </nova:instance>
Aug 09 02:33:24 user nova-compute[72351]:   </metadata>
Aug 09 02:33:24 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:33:24 user nova-compute[72351]:     <system>
Aug 09 02:33:24 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:33:24 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:33:24 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:33:24 user nova-compute[72351]:       <entry name="serial">80e87454-90b9-4168-bf00-b7d93b245526</entry>
Aug 09 02:33:24 user nova-compute[72351]:       <entry name="uuid">80e87454-90b9-4168-bf00-b7d93b245526</entry>
Aug 09 02:33:24 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:33:24 user nova-compute[72351]:     </system>
Aug 09 02:33:24 user nova-compute[72351]:   </sysinfo>
Aug 09 02:33:24 user nova-compute[72351]:   <os>
Aug 09 02:33:24 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:33:24 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:33:24 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:33:24 user nova-compute[72351]:   </os>
Aug 09 02:33:24 user nova-compute[72351]:   <features>
Aug 09 02:33:24 user nova-compute[72351]:     <acpi/>
Aug 09 02:33:24 user nova-compute[72351]:     <apic/>
Aug 09 02:33:24 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:33:24 user nova-compute[72351]:   </features>
Aug 09 02:33:24 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:33:24 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:33:24 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:33:24 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:33:24 user nova-compute[72351]:   </clock>
Aug 09 02:33:24 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:33:24 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:33:24 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:33:24 user nova-compute[72351]:   </cpu>
Aug 09 02:33:24 user nova-compute[72351]:   <devices>
Aug 09 02:33:24 user nova-compute[72351]:     <disk type="block" device="disk">
Aug 09 02:33:24 user nova-compute[72351]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:33:24 user nova-compute[72351]:       <source dev="/dev/sdd"/>
Aug 09 02:33:24 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:33:24 user nova-compute[72351]:       <serial>7e32fe78-f2c4-409c-813a-36924e611238</serial>
Aug 09 02:33:24 user nova-compute[72351]:     </disk>
Aug 09 02:33:24 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:33:24 user nova-compute[72351]:       <mac address="fa:16:3e:7a:77:69"/>
Aug 09 02:33:24 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:33:24 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:33:24 user nova-compute[72351]:       <target dev="tapfcb0ceaf-ff"/>
Aug 09 02:33:24 user nova-compute[72351]:     </interface>
Aug 09 02:33:24 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:33:24 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/80e87454-90b9-4168-bf00-b7d93b245526/console.log" append="off"/>
Aug 09 02:33:24 user nova-compute[72351]:     </serial>
Aug 09 02:33:24 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:33:24 user nova-compute[72351]:     <video>
Aug 09 02:33:24 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:33:24 user nova-compute[72351]:     </video>
Aug 09 02:33:24 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:33:24 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:33:24 user nova-compute[72351]:     </rng>
Aug 09 02:33:24 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:33:24 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:33:24 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:33:24 user nova-compute[72351]:     </memballoon>
Aug 09 02:33:24 user nova-compute[72351]:   </devices>
Aug 09 02:33:24 user nova-compute[72351]: </domain>
Aug 09 02:33:24 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:33:24 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:31:13Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestOtherA-server-2126139398',display_name='tempest-ServerActionsTestOtherA-server-2126139398',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(7),hidden=False,host='user',hostname='tempest-serveractionstestothera-server-2126139398',id=38,image_ref='',info_cache=InstanceInfoCache,instance_type_id=7,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBB4HNXglactRKXsG6uozX3XR7A39DBllYpgpCz+5+1RhfLOeXdmVkNfDkIG7vQrgoVNKWK/MwqtjPDLEVURcnWQfOuv4ySYdfEPj4hs6FpDqfJbAa5qX0FAbZQuEg69FVQ==',key_name='tempest-keypair-976839412',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:31:39Z,launched_on='user',locked=False,locked_by=None,memory_mb=2048,metadata={},migration_context=MigrationContext,new_flavor=Flavor(7),node='user',numa_topology=None,old_flavor=Flavor(6),os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='69d584c230b94ff8aae722eb2f364c5c',ramdisk_id='',reservation_id='r-97d5c06t',resources=None,root_device_name='/dev/vda',root_gb=20,security_groups=SecurityGroupList,services=ServiceList,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',old_vm_state='active',owner_project_name='tempest-ServerActionsTestOtherA-1303549305',owner_user_name='tempest-ServerActionsTestOtherA-1303549305-project-member'},tags=<?>,task_state='resize_finish',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:33:13Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='5dca34fdadbb4386a1ff1cb255265d47',uuid=80e87454-90b9-4168-bf00-b7d93b245526,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4", "address": "fa:16:3e:7a:77:69", "network": {"id": "b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-210689817-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.41", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestOtherA-210689817-network", "vif_mac": "fa:16:3e:7a:77:69"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "69d584c230b94ff8aae722eb2f364c5c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfcb0ceaf-ff", "ovs_interfaceid": "fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:33:24 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Converting VIF {"id": "fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4", "address": "fa:16:3e:7a:77:69", "network": {"id": "b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-210689817-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.41", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestOtherA-210689817-network", "vif_mac": "fa:16:3e:7a:77:69"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "69d584c230b94ff8aae722eb2f364c5c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfcb0ceaf-ff", "ovs_interfaceid": "fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:33:24 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:7a:77:69,bridge_name='br-int',has_traffic_filtering=True,id=fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4,network=Network(b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfcb0ceaf-ff') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:33:24 user nova-compute[72351]: DEBUG os_vif [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:7a:77:69,bridge_name='br-int',has_traffic_filtering=True,id=fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4,network=Network(b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfcb0ceaf-ff') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:33:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:33:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:33:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapfcb0ceaf-ff, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:33:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapfcb0ceaf-ff, col_values=(('external_ids', {'iface-id': 'fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:7a:77:69', 'vm-uuid': '80e87454-90b9-4168-bf00-b7d93b245526'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:33:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:33:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:24 user nova-compute[72351]: INFO os_vif [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:7a:77:69,bridge_name='br-int',has_traffic_filtering=True,id=fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4,network=Network(b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfcb0ceaf-ff')
Aug 09 02:33:24 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:33:24 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] No VIF found with MAC fa:16:3e:7a:77:69, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:33:25 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691548390.2822874, 80e87454-90b9-4168-bf00-b7d93b245526 => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:33:25 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] VM Stopped (Lifecycle Event)
Aug 09 02:33:25 user nova-compute[72351]: DEBUG nova.compute.manager [None req-2cc9c5b1-42d4-4c8d-bd4a-ef592d724168 None None] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:33:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:26 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:26 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:26 user nova-compute[72351]: DEBUG nova.compute.manager [req-dc6dc948-2fa2-4a37-aec7-ba1de76918ba req-36dbee22-77be-4b02-bdf7-f119e0e70e8f service nova] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Received event network-vif-plugged-fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:33:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-dc6dc948-2fa2-4a37-aec7-ba1de76918ba req-36dbee22-77be-4b02-bdf7-f119e0e70e8f service nova] Acquiring lock "80e87454-90b9-4168-bf00-b7d93b245526-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:33:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-dc6dc948-2fa2-4a37-aec7-ba1de76918ba req-36dbee22-77be-4b02-bdf7-f119e0e70e8f service nova] Lock "80e87454-90b9-4168-bf00-b7d93b245526-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:33:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-dc6dc948-2fa2-4a37-aec7-ba1de76918ba req-36dbee22-77be-4b02-bdf7-f119e0e70e8f service nova] Lock "80e87454-90b9-4168-bf00-b7d93b245526-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:33:26 user nova-compute[72351]: DEBUG nova.compute.manager [req-dc6dc948-2fa2-4a37-aec7-ba1de76918ba req-36dbee22-77be-4b02-bdf7-f119e0e70e8f service nova] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] No waiting events found dispatching network-vif-plugged-fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:33:26 user nova-compute[72351]: WARNING nova.compute.manager [req-dc6dc948-2fa2-4a37-aec7-ba1de76918ba req-36dbee22-77be-4b02-bdf7-f119e0e70e8f service nova] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Received unexpected event network-vif-plugged-fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4 for instance with vm_state active and task_state resize_finish.
Aug 09 02:33:28 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548408.1668415, 80e87454-90b9-4168-bf00-b7d93b245526 => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:33:28 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] VM Resumed (Lifecycle Event)
Aug 09 02:33:28 user nova-compute[72351]: DEBUG nova.compute.manager [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:33:28 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Instance running successfully.
Aug 09 02:33:28 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Failed to set time: agent not configured {{(pid=72351) sync_guest_time /opt/stack/nova/nova/virt/libvirt/guest.py:200}}
Aug 09 02:33:28 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6c958570-ca19-4aba-bc0a-58767d7ffeac tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] finish_migration finished successfully. {{(pid=72351) finish_migration /opt/stack/nova/nova/virt/libvirt/driver.py:11769}}
Aug 09 02:33:28 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:33:28 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: active, current task_state: resize_finish, current DB power_state: 1, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:33:28 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] During sync_power_state the instance has a pending task (resize_finish). Skip.
Aug 09 02:33:28 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548408.1713123, 80e87454-90b9-4168-bf00-b7d93b245526 => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:33:28 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] VM Started (Lifecycle Event)
Aug 09 02:33:28 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:33:28 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Synchronizing instance power state after lifecycle event "Started"; current vm_state: active, current task_state: resize_finish, current DB power_state: 1, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:33:28 user nova-compute[72351]: DEBUG nova.compute.manager [req-15a460a0-d305-4ed2-897c-986c8f6ea172 req-ee65f9f0-971e-4bcc-ac01-9bcb67fa41ee service nova] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Received event network-vif-plugged-fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:33:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-15a460a0-d305-4ed2-897c-986c8f6ea172 req-ee65f9f0-971e-4bcc-ac01-9bcb67fa41ee service nova] Acquiring lock "80e87454-90b9-4168-bf00-b7d93b245526-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:33:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-15a460a0-d305-4ed2-897c-986c8f6ea172 req-ee65f9f0-971e-4bcc-ac01-9bcb67fa41ee service nova] Lock "80e87454-90b9-4168-bf00-b7d93b245526-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:33:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-15a460a0-d305-4ed2-897c-986c8f6ea172 req-ee65f9f0-971e-4bcc-ac01-9bcb67fa41ee service nova] Lock "80e87454-90b9-4168-bf00-b7d93b245526-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:33:28 user nova-compute[72351]: DEBUG nova.compute.manager [req-15a460a0-d305-4ed2-897c-986c8f6ea172 req-ee65f9f0-971e-4bcc-ac01-9bcb67fa41ee service nova] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] No waiting events found dispatching network-vif-plugged-fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:33:28 user nova-compute[72351]: WARNING nova.compute.manager [req-15a460a0-d305-4ed2-897c-986c8f6ea172 req-ee65f9f0-971e-4bcc-ac01-9bcb67fa41ee service nova] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Received unexpected event network-vif-plugged-fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4 for instance with vm_state active and task_state resize_finish.
Aug 09 02:33:28 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] During sync_power_state the instance has a pending task (resize_finish). Skip.
Aug 09 02:33:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Acquiring lock "ad9b389e-ea11-4fd7-9233-a1be43052e60" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:33:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Lock "ad9b389e-ea11-4fd7-9233-a1be43052e60" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:33:28 user nova-compute[72351]: DEBUG nova.compute.manager [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Starting instance... {{(pid=72351) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2406}}
Aug 09 02:33:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:33:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:33:28 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72351) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Aug 09 02:33:28 user nova-compute[72351]: INFO nova.compute.claims [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Claim successful on node user
Aug 09 02:33:28 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:28 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:33:28 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:33:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.542s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:33:28 user nova-compute[72351]: DEBUG nova.compute.manager [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Start building networks asynchronously for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2803}}
Aug 09 02:33:29 user nova-compute[72351]: DEBUG nova.compute.manager [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Allocating IP information in the background. {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1956}}
Aug 09 02:33:29 user nova-compute[72351]: DEBUG nova.network.neutron [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] allocate_for_instance() {{(pid=72351) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Aug 09 02:33:29 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Aug 09 02:33:29 user nova-compute[72351]: DEBUG nova.compute.manager [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Start building block device mappings for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2838}}
Aug 09 02:33:29 user nova-compute[72351]: DEBUG nova.policy [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '801c60b1fab94f38b4d40222e9946f89', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '624e34ea80f944509fde2cb5472ed203', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72351) authorize /opt/stack/nova/nova/policy.py:203}}
Aug 09 02:33:29 user nova-compute[72351]: INFO nova.virt.block_device [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Booting with volume 75a3424d-ab51-409b-b1e1-f702f4a84d3d at /dev/vda
Aug 09 02:33:29 user nova-compute[72351]: DEBUG os_brick.utils [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:33:29 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:29 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:29 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[252c4fae-db96-4475-b967-cace5aa91590]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:a52269602ff4\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:29 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:29 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:29 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[f7f16608-4c29-4167-a6ee-61a4aa068dee]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:29 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:29 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:29 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[fe224735-d8ef-4f39-9157-2be9bc780e9c]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:29 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e0ba2556-45a8-441b-8038-d319de3bfc57]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:29 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Running cmd (subprocess): nvme version {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:29 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] 'nvme version' failed. Not Retrying. {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:33:29 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.nvmeof [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] nvme not present on system {{(pid=72351) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Aug 09 02:33:29 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:29 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:33:29 user nova-compute[72351]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Aug 09 02:33:29 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[cdf6c394-ed2d-4699-9853-b39b5aac7d17]: (4, '') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:29 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72351) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Aug 09 02:33:29 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Aug 09 02:33:29 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] LIGHTOS: no hostnqn found. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Aug 09 02:33:29 user nova-compute[72351]: DEBUG os_brick.utils [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] <== get_connector_properties: return (130ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:a52269602ff4', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:33:29 user nova-compute[72351]: DEBUG nova.virt.block_device [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Updating existing volume attachment record: 2cacd23d-4850-41aa-baf0-677030361616 {{(pid=72351) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Aug 09 02:33:29 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:30 user nova-compute[72351]: DEBUG nova.network.neutron [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Successfully created port: 64d8b550-334e-42c4-b57f-145ccc73517c {{(pid=72351) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Aug 09 02:33:30 user nova-compute[72351]: DEBUG nova.network.neutron [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Successfully updated port: 64d8b550-334e-42c4-b57f-145ccc73517c {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:33:30 user nova-compute[72351]: DEBUG nova.compute.manager [req-e595a9a1-304d-4daf-a940-aed10c2e9625 req-efac21a2-c42f-4df4-9c9d-b674258d85f3 service nova] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Received event network-changed-d8727f3f-d838-442c-a9a8-278f38aa87d4 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:33:30 user nova-compute[72351]: DEBUG nova.compute.manager [req-e595a9a1-304d-4daf-a940-aed10c2e9625 req-efac21a2-c42f-4df4-9c9d-b674258d85f3 service nova] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Refreshing instance network info cache due to event network-changed-d8727f3f-d838-442c-a9a8-278f38aa87d4. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:33:30 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-e595a9a1-304d-4daf-a940-aed10c2e9625 req-efac21a2-c42f-4df4-9c9d-b674258d85f3 service nova] Acquiring lock "refresh_cache-7bef74f4-3bb3-4ad0-bff7-34156be31249" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:33:30 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-e595a9a1-304d-4daf-a940-aed10c2e9625 req-efac21a2-c42f-4df4-9c9d-b674258d85f3 service nova] Acquired lock "refresh_cache-7bef74f4-3bb3-4ad0-bff7-34156be31249" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:33:30 user nova-compute[72351]: DEBUG nova.network.neutron [req-e595a9a1-304d-4daf-a940-aed10c2e9625 req-efac21a2-c42f-4df4-9c9d-b674258d85f3 service nova] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Refreshing network info cache for port d8727f3f-d838-442c-a9a8-278f38aa87d4 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:33:30 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Acquiring lock "refresh_cache-ad9b389e-ea11-4fd7-9233-a1be43052e60" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:33:30 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Acquired lock "refresh_cache-ad9b389e-ea11-4fd7-9233-a1be43052e60" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:33:30 user nova-compute[72351]: DEBUG nova.network.neutron [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:33:30 user nova-compute[72351]: DEBUG nova.compute.manager [req-fcdd1480-7729-4f27-88fc-a590c3b5f417 req-4b55f13c-43d0-4820-9b47-fa6f8cfe6b5c service nova] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Received event network-changed-64d8b550-334e-42c4-b57f-145ccc73517c {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:33:30 user nova-compute[72351]: DEBUG nova.compute.manager [req-fcdd1480-7729-4f27-88fc-a590c3b5f417 req-4b55f13c-43d0-4820-9b47-fa6f8cfe6b5c service nova] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Refreshing instance network info cache due to event network-changed-64d8b550-334e-42c4-b57f-145ccc73517c. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:33:30 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-fcdd1480-7729-4f27-88fc-a590c3b5f417 req-4b55f13c-43d0-4820-9b47-fa6f8cfe6b5c service nova] Acquiring lock "refresh_cache-ad9b389e-ea11-4fd7-9233-a1be43052e60" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:33:30 user nova-compute[72351]: DEBUG nova.network.neutron [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:33:31 user nova-compute[72351]: DEBUG nova.network.neutron [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Updating instance_info_cache with network_info: [{"id": "64d8b550-334e-42c4-b57f-145ccc73517c", "address": "fa:16:3e:52:34:4d", "network": {"id": "f7042dd1-90f3-4ee1-99cd-e52f9264dc87", "bridge": "br-int", "label": "tempest-ServerActionsV293TestJSON-1595239201-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "624e34ea80f944509fde2cb5472ed203", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap64d8b550-33", "ovs_interfaceid": "64d8b550-334e-42c4-b57f-145ccc73517c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:33:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Releasing lock "refresh_cache-ad9b389e-ea11-4fd7-9233-a1be43052e60" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:33:31 user nova-compute[72351]: DEBUG nova.compute.manager [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Instance network_info: |[{"id": "64d8b550-334e-42c4-b57f-145ccc73517c", "address": "fa:16:3e:52:34:4d", "network": {"id": "f7042dd1-90f3-4ee1-99cd-e52f9264dc87", "bridge": "br-int", "label": "tempest-ServerActionsV293TestJSON-1595239201-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "624e34ea80f944509fde2cb5472ed203", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap64d8b550-33", "ovs_interfaceid": "64d8b550-334e-42c4-b57f-145ccc73517c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1971}}
Aug 09 02:33:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-fcdd1480-7729-4f27-88fc-a590c3b5f417 req-4b55f13c-43d0-4820-9b47-fa6f8cfe6b5c service nova] Acquired lock "refresh_cache-ad9b389e-ea11-4fd7-9233-a1be43052e60" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:33:31 user nova-compute[72351]: DEBUG nova.network.neutron [req-fcdd1480-7729-4f27-88fc-a590c3b5f417 req-4b55f13c-43d0-4820-9b47-fa6f8cfe6b5c service nova] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Refreshing network info cache for port 64d8b550-334e-42c4-b57f-145ccc73517c {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:33:31 user nova-compute[72351]: DEBUG nova.network.neutron [req-e595a9a1-304d-4daf-a940-aed10c2e9625 req-efac21a2-c42f-4df4-9c9d-b674258d85f3 service nova] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Updated VIF entry in instance network info cache for port d8727f3f-d838-442c-a9a8-278f38aa87d4. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:33:31 user nova-compute[72351]: DEBUG nova.network.neutron [req-e595a9a1-304d-4daf-a940-aed10c2e9625 req-efac21a2-c42f-4df4-9c9d-b674258d85f3 service nova] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Updating instance_info_cache with network_info: [{"id": "d8727f3f-d838-442c-a9a8-278f38aa87d4", "address": "fa:16:3e:b4:8f:2e", "network": {"id": "626455de-9781-4ee8-b0be-c51ac124b189", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-1560904912-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.150", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0dd686dc07ca42f39aa898351ca281c2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd8727f3f-d8", "ovs_interfaceid": "d8727f3f-d838-442c-a9a8-278f38aa87d4", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:33:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-e595a9a1-304d-4daf-a940-aed10c2e9625 req-efac21a2-c42f-4df4-9c9d-b674258d85f3 service nova] Releasing lock "refresh_cache-7bef74f4-3bb3-4ad0-bff7-34156be31249" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:33:31 user nova-compute[72351]: DEBUG nova.network.neutron [req-fcdd1480-7729-4f27-88fc-a590c3b5f417 req-4b55f13c-43d0-4820-9b47-fa6f8cfe6b5c service nova] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Updated VIF entry in instance network info cache for port 64d8b550-334e-42c4-b57f-145ccc73517c. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:33:31 user nova-compute[72351]: DEBUG nova.network.neutron [req-fcdd1480-7729-4f27-88fc-a590c3b5f417 req-4b55f13c-43d0-4820-9b47-fa6f8cfe6b5c service nova] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Updating instance_info_cache with network_info: [{"id": "64d8b550-334e-42c4-b57f-145ccc73517c", "address": "fa:16:3e:52:34:4d", "network": {"id": "f7042dd1-90f3-4ee1-99cd-e52f9264dc87", "bridge": "br-int", "label": "tempest-ServerActionsV293TestJSON-1595239201-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "624e34ea80f944509fde2cb5472ed203", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap64d8b550-33", "ovs_interfaceid": "64d8b550-334e-42c4-b57f-145ccc73517c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:33:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-fcdd1480-7729-4f27-88fc-a590c3b5f417 req-4b55f13c-43d0-4820-9b47-fa6f8cfe6b5c service nova] Releasing lock "refresh_cache-ad9b389e-ea11-4fd7-9233-a1be43052e60" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:33:33 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:34 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-47a7dc0b-c620-4c6c-9433-3206e44e7fc1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Acquiring lock "7bef74f4-3bb3-4ad0-bff7-34156be31249" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:33:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-47a7dc0b-c620-4c6c-9433-3206e44e7fc1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "7bef74f4-3bb3-4ad0-bff7-34156be31249" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:33:35 user nova-compute[72351]: DEBUG nova.objects.instance [None req-47a7dc0b-c620-4c6c-9433-3206e44e7fc1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lazy-loading 'flavor' on Instance uuid 7bef74f4-3bb3-4ad0-bff7-34156be31249 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:33:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-47a7dc0b-c620-4c6c-9433-3206e44e7fc1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "7bef74f4-3bb3-4ad0-bff7-34156be31249" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.061s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:33:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-47a7dc0b-c620-4c6c-9433-3206e44e7fc1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Acquiring lock "7bef74f4-3bb3-4ad0-bff7-34156be31249" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:33:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-47a7dc0b-c620-4c6c-9433-3206e44e7fc1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "7bef74f4-3bb3-4ad0-bff7-34156be31249" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:33:35 user nova-compute[72351]: INFO nova.compute.manager [None req-47a7dc0b-c620-4c6c-9433-3206e44e7fc1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Attaching volume 8ecec058-a02b-413c-acca-556f68fa0d19 to /dev/vdb
Aug 09 02:33:35 user nova-compute[72351]: DEBUG os_brick.utils [None req-47a7dc0b-c620-4c6c-9433-3206e44e7fc1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:33:35 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:35 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:35 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[877e130a-ff9d-409b-8321-22abda831ea5]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:a52269602ff4\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:35 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:35 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.015s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:35 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[16e272bb-288a-46f0-8bb5-a69f9f196fbb]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:35 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:35 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:35 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[44d7f004-393e-496e-b2ed-3bb20a48b1c8]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:35 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[dcdbe1c0-4a9a-4fa8-8aa0-42eb838c7af1]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:35 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-47a7dc0b-c620-4c6c-9433-3206e44e7fc1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Running cmd (subprocess): nvme version {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:35 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-47a7dc0b-c620-4c6c-9433-3206e44e7fc1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] 'nvme version' failed. Not Retrying. {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:33:35 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.nvmeof [None req-47a7dc0b-c620-4c6c-9433-3206e44e7fc1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] nvme not present on system {{(pid=72351) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Aug 09 02:33:35 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:35 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:33:35 user nova-compute[72351]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Aug 09 02:33:35 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[99f52bbf-4ec5-4b9c-b326-db3cc4585f88]: (4, '') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:35 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-47a7dc0b-c620-4c6c-9433-3206e44e7fc1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72351) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Aug 09 02:33:35 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-47a7dc0b-c620-4c6c-9433-3206e44e7fc1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Aug 09 02:33:35 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-47a7dc0b-c620-4c6c-9433-3206e44e7fc1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] LIGHTOS: no hostnqn found. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Aug 09 02:33:35 user nova-compute[72351]: DEBUG os_brick.utils [None req-47a7dc0b-c620-4c6c-9433-3206e44e7fc1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] <== get_connector_properties: return (127ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:a52269602ff4', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:33:35 user nova-compute[72351]: DEBUG nova.virt.block_device [None req-47a7dc0b-c620-4c6c-9433-3206e44e7fc1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Updating existing volume attachment record: ccec6792-1506-4e5d-b98e-643bda0c75c9 {{(pid=72351) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Aug 09 02:33:35 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._sync_power_states {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:33:36 user nova-compute[72351]: WARNING nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] While synchronizing instance power states, found 15 instances in the database and 14 instances on the hypervisor.
Aug 09 02:33:36 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Triggering sync for uuid e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335 {{(pid=72351) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10252}}
Aug 09 02:33:36 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Triggering sync for uuid eb86cb49-2018-42c2-bd97-c662b829775d {{(pid=72351) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10252}}
Aug 09 02:33:36 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Triggering sync for uuid 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a {{(pid=72351) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10252}}
Aug 09 02:33:36 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Triggering sync for uuid a7f3d076-864b-4b68-9207-214ce2c21444 {{(pid=72351) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10252}}
Aug 09 02:33:36 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Triggering sync for uuid 47afce95-e5f7-4703-a99f-7ef8b28c4bd2 {{(pid=72351) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10252}}
Aug 09 02:33:36 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Triggering sync for uuid 957c1510-1320-4492-9052-a58ff540319d {{(pid=72351) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10252}}
Aug 09 02:33:36 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Triggering sync for uuid a03fbded-23ad-440b-97e7-bea07425e873 {{(pid=72351) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10252}}
Aug 09 02:33:36 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Triggering sync for uuid 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb {{(pid=72351) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10252}}
Aug 09 02:33:36 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Triggering sync for uuid 49a10075-0d3c-4cd6-bb43-10238cad3f37 {{(pid=72351) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10252}}
Aug 09 02:33:36 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Triggering sync for uuid f37b7fb7-1b94-421b-be79-7066010efabf {{(pid=72351) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10252}}
Aug 09 02:33:36 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Triggering sync for uuid 80e87454-90b9-4168-bf00-b7d93b245526 {{(pid=72351) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10252}}
Aug 09 02:33:36 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Triggering sync for uuid 7bef74f4-3bb3-4ad0-bff7-34156be31249 {{(pid=72351) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10252}}
Aug 09 02:33:36 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Triggering sync for uuid 41b5e305-3973-4937-b262-d32db1091e20 {{(pid=72351) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10252}}
Aug 09 02:33:36 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Triggering sync for uuid 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb {{(pid=72351) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10252}}
Aug 09 02:33:36 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Triggering sync for uuid ad9b389e-ea11-4fd7-9233-a1be43052e60 {{(pid=72351) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10252}}
Aug 09 02:33:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:33:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:33:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "eb86cb49-2018-42c2-bd97-c662b829775d" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:33:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "eb86cb49-2018-42c2-bd97-c662b829775d" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:33:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "585c81a5-ac24-4d12-a8a6-1e7b12a2d53a" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:33:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "585c81a5-ac24-4d12-a8a6-1e7b12a2d53a" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:33:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "a7f3d076-864b-4b68-9207-214ce2c21444" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:33:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "a7f3d076-864b-4b68-9207-214ce2c21444" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:33:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "47afce95-e5f7-4703-a99f-7ef8b28c4bd2" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:33:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "47afce95-e5f7-4703-a99f-7ef8b28c4bd2" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:33:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "957c1510-1320-4492-9052-a58ff540319d" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:33:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "957c1510-1320-4492-9052-a58ff540319d" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:33:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "a03fbded-23ad-440b-97e7-bea07425e873" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:33:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "a03fbded-23ad-440b-97e7-bea07425e873" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:33:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "7b6f5d60-5f23-4ef1-b972-9ba7086f56cb" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:33:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "7b6f5d60-5f23-4ef1-b972-9ba7086f56cb" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:33:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "49a10075-0d3c-4cd6-bb43-10238cad3f37" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:33:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "49a10075-0d3c-4cd6-bb43-10238cad3f37" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:33:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "f37b7fb7-1b94-421b-be79-7066010efabf" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:33:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "f37b7fb7-1b94-421b-be79-7066010efabf" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:33:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "80e87454-90b9-4168-bf00-b7d93b245526" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:33:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "80e87454-90b9-4168-bf00-b7d93b245526" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:33:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "7bef74f4-3bb3-4ad0-bff7-34156be31249" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:33:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "41b5e305-3973-4937-b262-d32db1091e20" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:33:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "41b5e305-3973-4937-b262-d32db1091e20" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:33:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:33:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:33:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "ad9b389e-ea11-4fd7-9233-a1be43052e60" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:33:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.043s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:33:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "eb86cb49-2018-42c2-bd97-c662b829775d" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.048s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:33:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "585c81a5-ac24-4d12-a8a6-1e7b12a2d53a" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.056s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:33:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "a7f3d076-864b-4b68-9207-214ce2c21444" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.064s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:33:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "957c1510-1320-4492-9052-a58ff540319d" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.086s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:33:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "47afce95-e5f7-4703-a99f-7ef8b28c4bd2" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.091s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:33:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "a03fbded-23ad-440b-97e7-bea07425e873" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.092s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:33:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "7b6f5d60-5f23-4ef1-b972-9ba7086f56cb" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.102s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:33:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "f37b7fb7-1b94-421b-be79-7066010efabf" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.120s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:33:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "80e87454-90b9-4168-bf00-b7d93b245526" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.128s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:33:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.130s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:33:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "49a10075-0d3c-4cd6-bb43-10238cad3f37" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.149s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:33:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "41b5e305-3973-4937-b262-d32db1091e20" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.155s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:33:38 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:38 user nova-compute[72351]: DEBUG nova.compute.manager [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Start spawning the instance on the hypervisor. {{(pid=72351) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2612}}
Aug 09 02:33:38 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Creating instance directory {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4717}}
Aug 09 02:33:38 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Creating image(s)
Aug 09 02:33:38 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Did not create local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4853}}
Aug 09 02:33:38 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Ensure instance console log exists: /opt/stack/data/nova/instances/ad9b389e-ea11-4fd7-9233-a1be43052e60/console.log {{(pid=72351) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4603}}
Aug 09 02:33:38 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:33:38 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:33:38 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:33:38 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Start _get_guest_xml network_info=[{"id": "64d8b550-334e-42c4-b57f-145ccc73517c", "address": "fa:16:3e:52:34:4d", "network": {"id": "f7042dd1-90f3-4ee1-99cd-e52f9264dc87", "bridge": "br-int", "label": "tempest-ServerActionsV293TestJSON-1595239201-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "624e34ea80f944509fde2cb5472ed203", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap64d8b550-33", "ovs_interfaceid": "64d8b550-334e-42c4-b57f-145ccc73517c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'device_type': 'disk', 'guest_format': None, 'disk_bus': 'virtio', 'mount_device': '/dev/vda', 'boot_index': 0, 'attachment_id': '2cacd23d-4850-41aa-baf0-677030361616', 'delete_on_termination': True, 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqns': ['iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'arRsVtvH', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': 'ad9b389e-ea11-4fd7-9233-a1be43052e60', 'attached_at': '', 'detached_at': '', 'volume_id': '75a3424d-ab51-409b-b1e1-f702f4a84d3d', 'serial': '75a3424d-ab51-409b-b1e1-f702f4a84d3d'}, 'volume_type': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:33:38 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:33:38 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:33:38 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:33:38 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:33:38 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:33:38 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:33:38 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:33:38 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:33:38 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:33:38 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:33:38 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:33:38 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:33:38 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:33:38 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:33:38 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:33:38 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:33:38 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:33:38 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:33:38 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 02:33:38 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'arRsVtvH', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:33:38 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:33:38 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:33:38 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:33:38 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:33:38 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:33:38 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:33:38 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 02:33:38 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:38 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d -p 172.16.0.220:3260" returned: 21 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:38 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[91b79699-291f-48cc-a080-799cd0f52c4d]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:38 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Aug 09 02:33:38 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:33:38 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d -p 172.16.0.220:3260 --interface default --op new {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:39 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[98d260f1-657b-43da-b5d0-caf2a4d3669e]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d] added\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d] added
Aug 09 02:33:39 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:33:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:39 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[70918209-4403-4050-a674-bb6874ecd2fa]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:33:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:39 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a99b6bcc-45a0-4b72-bfeb-4f6e5cccde99]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:33:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d -p 172.16.0.220:3260 --op update -n node.session.auth.username -v arRsVtvH {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d -p 172.16.0.220:3260 --op update -n node.session.auth.username -v arRsVtvH" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:39 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[2fcfc6e0-35f8-480b-9d42-f29cc994b995]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'arRsVtvH'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:33:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:39 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[b171ad5d-f04a-4092-a770-701749682519]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:33:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.027s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:39 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[047b9ebb-0a7f-44cf-b714-e582178e2ce6]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)\ntcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc (non-flash)\ntcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:33:39 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:33:39 user nova-compute[72351]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)
Aug 09 02:33:39 user nova-compute[72351]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc (non-flash)
Aug 09 02:33:39 user nova-compute[72351]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:33:39 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:33:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:33:39 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:33:39 user nova-compute[72351]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)
Aug 09 02:33:39 user nova-compute[72351]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc (non-flash)
Aug 09 02:33:39 user nova-compute[72351]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:33:39 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:33:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d -p 172.16.0.220:3260 --login {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d -p 172.16.0.220:3260 --login" returned: 0 in 0.072s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:39 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[380a0ee9-35de-447f-8da8-1a6e159be73f]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d, portal: 172.16.0.220,3260]
Aug 09 02:33:39 user nova-compute[72351]: Login to [iface: default, target: iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d, portal: 172.16.0.220,3260] successful.
Aug 09 02:33:39 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:33:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:39 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[10725d50-dcae-4688-b381-7cec40a0fd18]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:33:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.031s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:39 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[0f264925-bc26-4005-b4be-da8145bc7eb1]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)\ntcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc (non-flash)\ntcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)\ntcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:33:39 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:33:39 user nova-compute[72351]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)
Aug 09 02:33:39 user nova-compute[72351]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc (non-flash)
Aug 09 02:33:39 user nova-compute[72351]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:33:39 user nova-compute[72351]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)
Aug 09 02:33:39 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:33:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:33:39 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:33:39 user nova-compute[72351]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)
Aug 09 02:33:39 user nova-compute[72351]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc (non-flash)
Aug 09 02:33:39 user nova-compute[72351]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:33:39 user nova-compute[72351]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)
Aug 09 02:33:39 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:33:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.305s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:33:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 02:33:39 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] HCTL ('36', '-', '-', 0) found on session 44 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 02:33:39 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Scanning host 36 c: -, t: -, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 02:33:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host36/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host36/scan" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:39 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a70dba27-6b64-447f-b05e-3d51b3112126]: (4, ('- - 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:39 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Searching for a device in session 44 and hctl ['36', '*', '*', 0] yield: sde {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:33:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Connected to sde using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'arRsVtvH', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 02:33:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:40 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.341s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:33:40 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] <== connect_volume: return (1342ms) {'type': 'block', 'scsi_wwn': '23435343235323239', 'path': '/dev/sde'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:33:40 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23435343235323239', 'path': '/dev/sde'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 02:33:40 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:33:27Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-ServerActionsV293TestJSON-server-657819557',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serveractionsv293testjson-server-657819557',id=42,image_ref='',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBEunnOgCeXfv3GBp09pSPyhwCL3D6Q1n187gHEnIiV9PA3lK8lSVc8G3tp5ZU9m6m0OxdZGYGPGwrp313iNid8wPNsg46CMpUwvg2Jv3lbvp6Ufeyxq5cu1dosjL2Qlv6A==',key_name='tempest-keypair-461212954',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='624e34ea80f944509fde2cb5472ed203',ramdisk_id='',reservation_id='r-wzsh0h7v',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerActionsV293TestJSON-475025801',owner_user_name='tempest-ServerActionsV293TestJSON-475025801-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:33:29Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='801c60b1fab94f38b4d40222e9946f89',uuid=ad9b389e-ea11-4fd7-9233-a1be43052e60,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "64d8b550-334e-42c4-b57f-145ccc73517c", "address": "fa:16:3e:52:34:4d", "network": {"id": "f7042dd1-90f3-4ee1-99cd-e52f9264dc87", "bridge": "br-int", "label": "tempest-ServerActionsV293TestJSON-1595239201-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "624e34ea80f944509fde2cb5472ed203", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap64d8b550-33", "ovs_interfaceid": "64d8b550-334e-42c4-b57f-145ccc73517c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:33:40 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Converting VIF {"id": "64d8b550-334e-42c4-b57f-145ccc73517c", "address": "fa:16:3e:52:34:4d", "network": {"id": "f7042dd1-90f3-4ee1-99cd-e52f9264dc87", "bridge": "br-int", "label": "tempest-ServerActionsV293TestJSON-1595239201-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "624e34ea80f944509fde2cb5472ed203", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap64d8b550-33", "ovs_interfaceid": "64d8b550-334e-42c4-b57f-145ccc73517c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:33:40 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:52:34:4d,bridge_name='br-int',has_traffic_filtering=True,id=64d8b550-334e-42c4-b57f-145ccc73517c,network=Network(f7042dd1-90f3-4ee1-99cd-e52f9264dc87),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap64d8b550-33') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:33:40 user nova-compute[72351]: DEBUG nova.objects.instance [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Lazy-loading 'pci_devices' on Instance uuid ad9b389e-ea11-4fd7-9233-a1be43052e60 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:33:40 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:33:40 user nova-compute[72351]:   <uuid>ad9b389e-ea11-4fd7-9233-a1be43052e60</uuid>
Aug 09 02:33:40 user nova-compute[72351]:   <name>instance-0000002a</name>
Aug 09 02:33:40 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:33:40 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:33:40 user nova-compute[72351]:   <metadata>
Aug 09 02:33:40 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:33:40 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:33:40 user nova-compute[72351]:       <nova:name>tempest-ServerActionsV293TestJSON-server-657819557</nova:name>
Aug 09 02:33:40 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:33:38</nova:creationTime>
Aug 09 02:33:40 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:33:40 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:33:40 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:33:40 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:33:40 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:33:40 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:33:40 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:33:40 user nova-compute[72351]:       <nova:owner>
Aug 09 02:33:40 user nova-compute[72351]:         <nova:user uuid="801c60b1fab94f38b4d40222e9946f89">tempest-ServerActionsV293TestJSON-475025801-project-member</nova:user>
Aug 09 02:33:40 user nova-compute[72351]:         <nova:project uuid="624e34ea80f944509fde2cb5472ed203">tempest-ServerActionsV293TestJSON-475025801</nova:project>
Aug 09 02:33:40 user nova-compute[72351]:       </nova:owner>
Aug 09 02:33:40 user nova-compute[72351]:       <nova:ports>
Aug 09 02:33:40 user nova-compute[72351]:         <nova:port uuid="64d8b550-334e-42c4-b57f-145ccc73517c">
Aug 09 02:33:40 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.6" ipVersion="4"/>
Aug 09 02:33:40 user nova-compute[72351]:         </nova:port>
Aug 09 02:33:40 user nova-compute[72351]:       </nova:ports>
Aug 09 02:33:40 user nova-compute[72351]:     </nova:instance>
Aug 09 02:33:40 user nova-compute[72351]:   </metadata>
Aug 09 02:33:40 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:33:40 user nova-compute[72351]:     <system>
Aug 09 02:33:40 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:33:40 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:33:40 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:33:40 user nova-compute[72351]:       <entry name="serial">ad9b389e-ea11-4fd7-9233-a1be43052e60</entry>
Aug 09 02:33:40 user nova-compute[72351]:       <entry name="uuid">ad9b389e-ea11-4fd7-9233-a1be43052e60</entry>
Aug 09 02:33:40 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:33:40 user nova-compute[72351]:     </system>
Aug 09 02:33:40 user nova-compute[72351]:   </sysinfo>
Aug 09 02:33:40 user nova-compute[72351]:   <os>
Aug 09 02:33:40 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:33:40 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:33:40 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:33:40 user nova-compute[72351]:   </os>
Aug 09 02:33:40 user nova-compute[72351]:   <features>
Aug 09 02:33:40 user nova-compute[72351]:     <acpi/>
Aug 09 02:33:40 user nova-compute[72351]:     <apic/>
Aug 09 02:33:40 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:33:40 user nova-compute[72351]:   </features>
Aug 09 02:33:40 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:33:40 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:33:40 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:33:40 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:33:40 user nova-compute[72351]:   </clock>
Aug 09 02:33:40 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:33:40 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:33:40 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:33:40 user nova-compute[72351]:   </cpu>
Aug 09 02:33:40 user nova-compute[72351]:   <devices>
Aug 09 02:33:40 user nova-compute[72351]:     <disk type="block" device="disk">
Aug 09 02:33:40 user nova-compute[72351]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:33:40 user nova-compute[72351]:       <source dev="/dev/sde"/>
Aug 09 02:33:40 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:33:40 user nova-compute[72351]:       <serial>75a3424d-ab51-409b-b1e1-f702f4a84d3d</serial>
Aug 09 02:33:40 user nova-compute[72351]:     </disk>
Aug 09 02:33:40 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:33:40 user nova-compute[72351]:       <mac address="fa:16:3e:52:34:4d"/>
Aug 09 02:33:40 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:33:40 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:33:40 user nova-compute[72351]:       <target dev="tap64d8b550-33"/>
Aug 09 02:33:40 user nova-compute[72351]:     </interface>
Aug 09 02:33:40 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:33:40 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/ad9b389e-ea11-4fd7-9233-a1be43052e60/console.log" append="off"/>
Aug 09 02:33:40 user nova-compute[72351]:     </serial>
Aug 09 02:33:40 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:33:40 user nova-compute[72351]:     <video>
Aug 09 02:33:40 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:33:40 user nova-compute[72351]:     </video>
Aug 09 02:33:40 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:33:40 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:33:40 user nova-compute[72351]:     </rng>
Aug 09 02:33:40 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:33:40 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:33:40 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:33:40 user nova-compute[72351]:     </memballoon>
Aug 09 02:33:40 user nova-compute[72351]:   </devices>
Aug 09 02:33:40 user nova-compute[72351]: </domain>
Aug 09 02:33:40 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:33:40 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:33:27Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-ServerActionsV293TestJSON-server-657819557',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serveractionsv293testjson-server-657819557',id=42,image_ref='',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBEunnOgCeXfv3GBp09pSPyhwCL3D6Q1n187gHEnIiV9PA3lK8lSVc8G3tp5ZU9m6m0OxdZGYGPGwrp313iNid8wPNsg46CMpUwvg2Jv3lbvp6Ufeyxq5cu1dosjL2Qlv6A==',key_name='tempest-keypair-461212954',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='624e34ea80f944509fde2cb5472ed203',ramdisk_id='',reservation_id='r-wzsh0h7v',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerActionsV293TestJSON-475025801',owner_user_name='tempest-ServerActionsV293TestJSON-475025801-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:33:29Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='801c60b1fab94f38b4d40222e9946f89',uuid=ad9b389e-ea11-4fd7-9233-a1be43052e60,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "64d8b550-334e-42c4-b57f-145ccc73517c", "address": "fa:16:3e:52:34:4d", "network": {"id": "f7042dd1-90f3-4ee1-99cd-e52f9264dc87", "bridge": "br-int", "label": "tempest-ServerActionsV293TestJSON-1595239201-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "624e34ea80f944509fde2cb5472ed203", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap64d8b550-33", "ovs_interfaceid": "64d8b550-334e-42c4-b57f-145ccc73517c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:33:40 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Converting VIF {"id": "64d8b550-334e-42c4-b57f-145ccc73517c", "address": "fa:16:3e:52:34:4d", "network": {"id": "f7042dd1-90f3-4ee1-99cd-e52f9264dc87", "bridge": "br-int", "label": "tempest-ServerActionsV293TestJSON-1595239201-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "624e34ea80f944509fde2cb5472ed203", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap64d8b550-33", "ovs_interfaceid": "64d8b550-334e-42c4-b57f-145ccc73517c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:33:40 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:52:34:4d,bridge_name='br-int',has_traffic_filtering=True,id=64d8b550-334e-42c4-b57f-145ccc73517c,network=Network(f7042dd1-90f3-4ee1-99cd-e52f9264dc87),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap64d8b550-33') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:33:40 user nova-compute[72351]: DEBUG os_vif [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:52:34:4d,bridge_name='br-int',has_traffic_filtering=True,id=64d8b550-334e-42c4-b57f-145ccc73517c,network=Network(f7042dd1-90f3-4ee1-99cd-e52f9264dc87),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap64d8b550-33') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:33:40 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:40 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:33:40 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:33:40 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:40 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap64d8b550-33, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:33:40 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap64d8b550-33, col_values=(('external_ids', {'iface-id': '64d8b550-334e-42c4-b57f-145ccc73517c', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:52:34:4d', 'vm-uuid': 'ad9b389e-ea11-4fd7-9233-a1be43052e60'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:33:40 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:40 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:33:40 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:40 user nova-compute[72351]: INFO os_vif [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:52:34:4d,bridge_name='br-int',has_traffic_filtering=True,id=64d8b550-334e-42c4-b57f-145ccc73517c,network=Network(f7042dd1-90f3-4ee1-99cd-e52f9264dc87),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap64d8b550-33')
Aug 09 02:33:40 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:33:40 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] No VIF found with MAC fa:16:3e:52:34:4d, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:33:41 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:41 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:41 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:41 user nova-compute[72351]: DEBUG nova.compute.manager [req-36447628-98ba-4c77-9e3b-f88b2307a8b6 req-414d32f9-1431-404d-8d14-852e5dd2b08d service nova] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Received event network-vif-plugged-64d8b550-334e-42c4-b57f-145ccc73517c {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:33:41 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-36447628-98ba-4c77-9e3b-f88b2307a8b6 req-414d32f9-1431-404d-8d14-852e5dd2b08d service nova] Acquiring lock "ad9b389e-ea11-4fd7-9233-a1be43052e60-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:33:41 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-36447628-98ba-4c77-9e3b-f88b2307a8b6 req-414d32f9-1431-404d-8d14-852e5dd2b08d service nova] Lock "ad9b389e-ea11-4fd7-9233-a1be43052e60-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:33:41 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-36447628-98ba-4c77-9e3b-f88b2307a8b6 req-414d32f9-1431-404d-8d14-852e5dd2b08d service nova] Lock "ad9b389e-ea11-4fd7-9233-a1be43052e60-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:33:41 user nova-compute[72351]: DEBUG nova.compute.manager [req-36447628-98ba-4c77-9e3b-f88b2307a8b6 req-414d32f9-1431-404d-8d14-852e5dd2b08d service nova] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] No waiting events found dispatching network-vif-plugged-64d8b550-334e-42c4-b57f-145ccc73517c {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:33:41 user nova-compute[72351]: WARNING nova.compute.manager [req-36447628-98ba-4c77-9e3b-f88b2307a8b6 req-414d32f9-1431-404d-8d14-852e5dd2b08d service nova] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Received unexpected event network-vif-plugged-64d8b550-334e-42c4-b57f-145ccc73517c for instance with vm_state building and task_state spawning.
Aug 09 02:33:42 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:42 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:42 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:43 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:43 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548423.8574533, ad9b389e-ea11-4fd7-9233-a1be43052e60 => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:33:43 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] VM Resumed (Lifecycle Event)
Aug 09 02:33:43 user nova-compute[72351]: DEBUG nova.compute.manager [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:33:43 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Guest created on hypervisor {{(pid=72351) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4411}}
Aug 09 02:33:43 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Instance spawned successfully.
Aug 09 02:33:43 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:894}}
Aug 09 02:33:43 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:33:43 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:33:43 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Found default for hw_cdrom_bus of ide {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:33:43 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Found default for hw_disk_bus of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:33:43 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Found default for hw_input_bus of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:33:43 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Found default for hw_pointer_model of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:33:43 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Found default for hw_video_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:33:43 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Found default for hw_vif_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:33:43 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:33:43 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548423.8590677, ad9b389e-ea11-4fd7-9233-a1be43052e60 => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:33:43 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] VM Started (Lifecycle Event)
Aug 09 02:33:43 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:33:43 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:33:43 user nova-compute[72351]: INFO nova.compute.manager [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Took 5.08 seconds to spawn the instance on the hypervisor.
Aug 09 02:33:43 user nova-compute[72351]: DEBUG nova.compute.manager [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:33:43 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:33:44 user nova-compute[72351]: INFO nova.compute.manager [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Took 15.72 seconds to build instance.
Aug 09 02:33:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6e818457-fe9d-4e61-8771-33c4f3bb8559 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Lock "ad9b389e-ea11-4fd7-9233-a1be43052e60" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 15.811s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:33:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "ad9b389e-ea11-4fd7-9233-a1be43052e60" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 8.061s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:33:44 user nova-compute[72351]: INFO nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] During sync_power_state the instance has a pending task (block_device_mapping). Skip.
Aug 09 02:33:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "ad9b389e-ea11-4fd7-9233-a1be43052e60" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:33:44 user nova-compute[72351]: DEBUG nova.compute.manager [req-34367d35-e089-4d17-8ed2-366ac0c05a5e req-236cfc72-d27e-4245-abac-6c2b66d7a662 service nova] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Received event network-vif-plugged-64d8b550-334e-42c4-b57f-145ccc73517c {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:33:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-34367d35-e089-4d17-8ed2-366ac0c05a5e req-236cfc72-d27e-4245-abac-6c2b66d7a662 service nova] Acquiring lock "ad9b389e-ea11-4fd7-9233-a1be43052e60-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:33:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-34367d35-e089-4d17-8ed2-366ac0c05a5e req-236cfc72-d27e-4245-abac-6c2b66d7a662 service nova] Lock "ad9b389e-ea11-4fd7-9233-a1be43052e60-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:33:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-34367d35-e089-4d17-8ed2-366ac0c05a5e req-236cfc72-d27e-4245-abac-6c2b66d7a662 service nova] Lock "ad9b389e-ea11-4fd7-9233-a1be43052e60-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:33:44 user nova-compute[72351]: DEBUG nova.compute.manager [req-34367d35-e089-4d17-8ed2-366ac0c05a5e req-236cfc72-d27e-4245-abac-6c2b66d7a662 service nova] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] No waiting events found dispatching network-vif-plugged-64d8b550-334e-42c4-b57f-145ccc73517c {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:33:44 user nova-compute[72351]: WARNING nova.compute.manager [req-34367d35-e089-4d17-8ed2-366ac0c05a5e req-236cfc72-d27e-4245-abac-6c2b66d7a662 service nova] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Received unexpected event network-vif-plugged-64d8b550-334e-42c4-b57f-145ccc73517c for instance with vm_state building and task_state spawning.
Aug 09 02:33:45 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-47a7dc0b-c620-4c6c-9433-3206e44e7fc1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 02:33:45 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-47a7dc0b-c620-4c6c-9433-3206e44e7fc1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:8ecec058-a02b-413c-acca-556f68fa0d19'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'vFKkMVmH', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:33:45 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-47a7dc0b-c620-4c6c-9433-3206e44e7fc1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:33:45 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-47a7dc0b-c620-4c6c-9433-3206e44e7fc1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:33:45 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-47a7dc0b-c620-4c6c-9433-3206e44e7fc1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:33:45 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-47a7dc0b-c620-4c6c-9433-3206e44e7fc1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:33:45 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-47a7dc0b-c620-4c6c-9433-3206e44e7fc1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:8ecec058-a02b-413c-acca-556f68fa0d19" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:33:45 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-47a7dc0b-c620-4c6c-9433-3206e44e7fc1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:8ecec058-a02b-413c-acca-556f68fa0d19" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:33:45 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-47a7dc0b-c620-4c6c-9433-3206e44e7fc1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 02:33:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8ecec058-a02b-413c-acca-556f68fa0d19 -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8ecec058-a02b-413c-acca-556f68fa0d19 -p 172.16.0.220:3260" returned: 21 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:45 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[bac367ee-8edd-4c5e-9c72-0173dae05db1]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:45 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-47a7dc0b-c620-4c6c-9433-3206e44e7fc1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Aug 09 02:33:45 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:33:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8ecec058-a02b-413c-acca-556f68fa0d19 -p 172.16.0.220:3260 --interface default --op new {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8ecec058-a02b-413c-acca-556f68fa0d19 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:45 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[f3da769f-d237-4287-9f78-355d961e257f]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:8ecec058-a02b-413c-acca-556f68fa0d19] added\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:45 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-47a7dc0b-c620-4c6c-9433-3206e44e7fc1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:8ecec058-a02b-413c-acca-556f68fa0d19] added
Aug 09 02:33:45 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:33:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8ecec058-a02b-413c-acca-556f68fa0d19 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8ecec058-a02b-413c-acca-556f68fa0d19 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:45 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[c961c993-e704-442f-9f19-d42e3432e1d6]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:45 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-47a7dc0b-c620-4c6c-9433-3206e44e7fc1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:33:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8ecec058-a02b-413c-acca-556f68fa0d19 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8ecec058-a02b-413c-acca-556f68fa0d19 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:45 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a422fccb-6fe9-49f8-8832-c4b1a4513b59]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:45 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-47a7dc0b-c620-4c6c-9433-3206e44e7fc1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:33:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8ecec058-a02b-413c-acca-556f68fa0d19 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v vFKkMVmH {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8ecec058-a02b-413c-acca-556f68fa0d19 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v vFKkMVmH" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:45 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[64004cdc-91fe-45dc-a9dc-8953aa0c62fe]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:45 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-47a7dc0b-c620-4c6c-9433-3206e44e7fc1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'vFKkMVmH'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:33:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8ecec058-a02b-413c-acca-556f68fa0d19 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8ecec058-a02b-413c-acca-556f68fa0d19 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:45 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[37d07293-1c1d-4d92-9246-c2865766ba1d]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:45 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-47a7dc0b-c620-4c6c-9433-3206e44e7fc1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:33:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.031s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:45 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[f391032b-d79f-480d-8070-cd8dee2224a0]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)\ntcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc (non-flash)\ntcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)\ntcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:45 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-47a7dc0b-c620-4c6c-9433-3206e44e7fc1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:33:45 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:33:45 user nova-compute[72351]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)
Aug 09 02:33:45 user nova-compute[72351]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc (non-flash)
Aug 09 02:33:45 user nova-compute[72351]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:33:45 user nova-compute[72351]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)
Aug 09 02:33:45 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:33:45 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-47a7dc0b-c620-4c6c-9433-3206e44e7fc1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:33:45 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:33:45 user nova-compute[72351]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)
Aug 09 02:33:45 user nova-compute[72351]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc (non-flash)
Aug 09 02:33:45 user nova-compute[72351]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:33:45 user nova-compute[72351]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)
Aug 09 02:33:45 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:33:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8ecec058-a02b-413c-acca-556f68fa0d19 -p 172.16.0.220:3260 --login {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8ecec058-a02b-413c-acca-556f68fa0d19 -p 172.16.0.220:3260 --login" returned: 0 in 0.085s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:45 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[cd0b9142-2b33-41d3-a56c-5863228ab27d]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:8ecec058-a02b-413c-acca-556f68fa0d19, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:8ecec058-a02b-413c-acca-556f68fa0d19, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:45 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-47a7dc0b-c620-4c6c-9433-3206e44e7fc1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:8ecec058-a02b-413c-acca-556f68fa0d19, portal: 172.16.0.220,3260]
Aug 09 02:33:45 user nova-compute[72351]: Login to [iface: default, target: iqn.2010-10.org.openstack:8ecec058-a02b-413c-acca-556f68fa0d19, portal: 172.16.0.220,3260] successful.
Aug 09 02:33:45 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:33:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8ecec058-a02b-413c-acca-556f68fa0d19 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8ecec058-a02b-413c-acca-556f68fa0d19 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:45 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[db8517ce-06dd-43c0-b2ba-26d203f4f34a]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:45 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-47a7dc0b-c620-4c6c-9433-3206e44e7fc1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:33:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.032s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:45 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[45a11a84-0a31-423f-9c32-7b867255631f]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)\ntcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc (non-flash)\ntcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)\ntcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)\ntcp: [45] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ecec058-a02b-413c-acca-556f68fa0d19 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:45 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-47a7dc0b-c620-4c6c-9433-3206e44e7fc1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:33:45 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:33:45 user nova-compute[72351]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)
Aug 09 02:33:45 user nova-compute[72351]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc (non-flash)
Aug 09 02:33:45 user nova-compute[72351]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:33:45 user nova-compute[72351]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)
Aug 09 02:33:45 user nova-compute[72351]: tcp: [45] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ecec058-a02b-413c-acca-556f68fa0d19 (non-flash)
Aug 09 02:33:45 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:33:45 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-47a7dc0b-c620-4c6c-9433-3206e44e7fc1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:33:45 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:33:45 user nova-compute[72351]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)
Aug 09 02:33:45 user nova-compute[72351]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc (non-flash)
Aug 09 02:33:45 user nova-compute[72351]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:33:45 user nova-compute[72351]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)
Aug 09 02:33:45 user nova-compute[72351]: tcp: [45] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ecec058-a02b-413c-acca-556f68fa0d19 (non-flash)
Aug 09 02:33:45 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:33:45 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-47a7dc0b-c620-4c6c-9433-3206e44e7fc1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:8ecec058-a02b-413c-acca-556f68fa0d19" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.323s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:33:45 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-47a7dc0b-c620-4c6c-9433-3206e44e7fc1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 02:33:45 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-47a7dc0b-c620-4c6c-9433-3206e44e7fc1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] HCTL ('39', '-', '-', 0) found on session 45 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 02:33:45 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-47a7dc0b-c620-4c6c-9433-3206e44e7fc1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Scanning host 39 c: -, t: -, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 02:33:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host39/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:45 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host39/scan" returned: 0 in 0.016s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:45 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[72658fe7-1147-41d2-a9ce-6288cc71c1bc]: (4, ('- - 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:33:45 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-47a7dc0b-c620-4c6c-9433-3206e44e7fc1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Searching for a device in session 45 and hctl ['39', '*', '*', 0] yield: sdh {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:33:45 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-47a7dc0b-c620-4c6c-9433-3206e44e7fc1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Connected to sdh using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'vFKkMVmH', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 02:33:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-47a7dc0b-c620-4c6c-9433-3206e44e7fc1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.360s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:33:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-47a7dc0b-c620-4c6c-9433-3206e44e7fc1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] <== connect_volume: return (1362ms) {'type': 'block', 'scsi_wwn': '23363636130316136', 'path': '/dev/sdh'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:33:46 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-47a7dc0b-c620-4c6c-9433-3206e44e7fc1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23363636130316136', 'path': '/dev/sdh'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 02:33:46 user nova-compute[72351]: DEBUG nova.objects.instance [None req-47a7dc0b-c620-4c6c-9433-3206e44e7fc1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lazy-loading 'flavor' on Instance uuid 7bef74f4-3bb3-4ad0-bff7-34156be31249 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:33:46 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-47a7dc0b-c620-4c6c-9433-3206e44e7fc1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] attach device xml: <disk type="block" device="disk">
Aug 09 02:33:46 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:33:46 user nova-compute[72351]:   <source dev="/dev/sdh"/>
Aug 09 02:33:46 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:33:46 user nova-compute[72351]:   <serial>8ecec058-a02b-413c-acca-556f68fa0d19</serial>
Aug 09 02:33:46 user nova-compute[72351]: </disk>
Aug 09 02:33:46 user nova-compute[72351]:  {{(pid=72351) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Aug 09 02:33:46 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-47a7dc0b-c620-4c6c-9433-3206e44e7fc1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:33:46 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-47a7dc0b-c620-4c6c-9433-3206e44e7fc1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:33:46 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-47a7dc0b-c620-4c6c-9433-3206e44e7fc1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] No VIF found with MAC fa:16:3e:b4:8f:2e, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:33:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-47a7dc0b-c620-4c6c-9433-3206e44e7fc1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "7bef74f4-3bb3-4ad0-bff7-34156be31249" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 11.247s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:33:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "7bef74f4-3bb3-4ad0-bff7-34156be31249" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 10.904s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:33:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "7bef74f4-3bb3-4ad0-bff7-34156be31249" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.029s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:33:48 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:50 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:50 user nova-compute[72351]: DEBUG nova.compute.manager [None req-5c4c183c-f502-4358-97a8-cdb183d863eb tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:33:50 user nova-compute[72351]: INFO nova.compute.manager [None req-5c4c183c-f502-4358-97a8-cdb183d863eb tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 41b5e305-3973-4937-b262-d32db1091e20] instance snapshotting
Aug 09 02:33:50 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-5c4c183c-f502-4358-97a8-cdb183d863eb tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Beginning live snapshot process
Aug 09 02:33:50 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-5c4c183c-f502-4358-97a8-cdb183d863eb tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/41b5e305-3973-4937-b262-d32db1091e20/disk --force-share --output=json -f qcow2 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:50 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-5c4c183c-f502-4358-97a8-cdb183d863eb tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/41b5e305-3973-4937-b262-d32db1091e20/disk --force-share --output=json -f qcow2" returned: 0 in 0.132s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:50 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-5c4c183c-f502-4358-97a8-cdb183d863eb tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/41b5e305-3973-4937-b262-d32db1091e20/disk --force-share --output=json -f qcow2 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:51 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-5c4c183c-f502-4358-97a8-cdb183d863eb tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/41b5e305-3973-4937-b262-d32db1091e20/disk --force-share --output=json -f qcow2" returned: 0 in 0.135s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:51 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-5c4c183c-f502-4358-97a8-cdb183d863eb tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:51 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-5c4c183c-f502-4358-97a8-cdb183d863eb tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.133s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:51 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-5c4c183c-f502-4358-97a8-cdb183d863eb tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/snapshots/tmply06tafp/e3a79f07d7f04540b1cc7fdf476424ed.delta 1073741824 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:51 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-5c4c183c-f502-4358-97a8-cdb183d863eb tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/snapshots/tmply06tafp/e3a79f07d7f04540b1cc7fdf476424ed.delta 1073741824" returned: 0 in 0.053s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:51 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-5c4c183c-f502-4358-97a8-cdb183d863eb tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Quiescing instance not available: QEMU guest agent is not enabled.
Aug 09 02:33:52 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-5c4c183c-f502-4358-97a8-cdb183d863eb tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] COPY block job progress, current cursor: 0 final cursor: 75497472 {{(pid=72351) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Aug 09 02:33:52 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-5c4c183c-f502-4358-97a8-cdb183d863eb tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] COPY block job progress, current cursor: 75497472 final cursor: 75497472 {{(pid=72351) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Aug 09 02:33:52 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-5c4c183c-f502-4358-97a8-cdb183d863eb tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Skipping quiescing instance: QEMU guest agent is not enabled.
Aug 09 02:33:52 user nova-compute[72351]: DEBUG nova.privsep.utils [None req-5c4c183c-f502-4358-97a8-cdb183d863eb tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=72351) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Aug 09 02:33:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-5c4c183c-f502-4358-97a8-cdb183d863eb tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Running cmd (subprocess): qemu-img convert -t none -O qcow2 -f qcow2 /opt/stack/data/nova/instances/snapshots/tmply06tafp/e3a79f07d7f04540b1cc7fdf476424ed.delta /opt/stack/data/nova/instances/snapshots/tmply06tafp/e3a79f07d7f04540b1cc7fdf476424ed {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:33:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-5c4c183c-f502-4358-97a8-cdb183d863eb tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] CMD "qemu-img convert -t none -O qcow2 -f qcow2 /opt/stack/data/nova/instances/snapshots/tmply06tafp/e3a79f07d7f04540b1cc7fdf476424ed.delta /opt/stack/data/nova/instances/snapshots/tmply06tafp/e3a79f07d7f04540b1cc7fdf476424ed" returned: 0 in 0.514s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:33:53 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-5c4c183c-f502-4358-97a8-cdb183d863eb tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Snapshot extracted, beginning image upload
Aug 09 02:33:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:55 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:55 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-5c4c183c-f502-4358-97a8-cdb183d863eb tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Snapshot image upload complete
Aug 09 02:33:55 user nova-compute[72351]: INFO nova.compute.manager [None req-5c4c183c-f502-4358-97a8-cdb183d863eb tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Took 4.82 seconds to snapshot the instance on the hypervisor.
Aug 09 02:33:58 user nova-compute[72351]: DEBUG nova.compute.manager [req-6d03a010-db8b-4938-8c7d-a7fa0a716f32 req-86f32553-6eba-4965-8b59-ef67ffce1a09 service nova] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Received event network-changed-1f2da7a0-a0f0-4715-8c5d-1df2a85fd295 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:33:58 user nova-compute[72351]: DEBUG nova.compute.manager [req-6d03a010-db8b-4938-8c7d-a7fa0a716f32 req-86f32553-6eba-4965-8b59-ef67ffce1a09 service nova] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Refreshing instance network info cache due to event network-changed-1f2da7a0-a0f0-4715-8c5d-1df2a85fd295. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:33:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-6d03a010-db8b-4938-8c7d-a7fa0a716f32 req-86f32553-6eba-4965-8b59-ef67ffce1a09 service nova] Acquiring lock "refresh_cache-f37b7fb7-1b94-421b-be79-7066010efabf" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:33:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-6d03a010-db8b-4938-8c7d-a7fa0a716f32 req-86f32553-6eba-4965-8b59-ef67ffce1a09 service nova] Acquired lock "refresh_cache-f37b7fb7-1b94-421b-be79-7066010efabf" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:33:58 user nova-compute[72351]: DEBUG nova.network.neutron [req-6d03a010-db8b-4938-8c7d-a7fa0a716f32 req-86f32553-6eba-4965-8b59-ef67ffce1a09 service nova] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Refreshing network info cache for port 1f2da7a0-a0f0-4715-8c5d-1df2a85fd295 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:33:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5d163df9-f461-4285-a89e-a855b9bab28a tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Acquiring lock "f37b7fb7-1b94-421b-be79-7066010efabf" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:33:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5d163df9-f461-4285-a89e-a855b9bab28a tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lock "f37b7fb7-1b94-421b-be79-7066010efabf" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:33:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5d163df9-f461-4285-a89e-a855b9bab28a tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Acquiring lock "f37b7fb7-1b94-421b-be79-7066010efabf-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:33:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5d163df9-f461-4285-a89e-a855b9bab28a tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lock "f37b7fb7-1b94-421b-be79-7066010efabf-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:33:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5d163df9-f461-4285-a89e-a855b9bab28a tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lock "f37b7fb7-1b94-421b-be79-7066010efabf-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:33:58 user nova-compute[72351]: INFO nova.compute.manager [None req-5d163df9-f461-4285-a89e-a855b9bab28a tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Terminating instance
Aug 09 02:33:58 user nova-compute[72351]: DEBUG nova.compute.manager [None req-5d163df9-f461-4285-a89e-a855b9bab28a tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Start destroying the instance on the hypervisor. {{(pid=72351) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3124}}
Aug 09 02:33:58 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:58 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:58 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:58 user nova-compute[72351]: DEBUG nova.compute.manager [req-b94ef807-e87d-440b-8e75-b076314fa0db req-90256b75-aa82-47ec-ac42-d242c18ae8e1 service nova] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Received event network-vif-unplugged-1f2da7a0-a0f0-4715-8c5d-1df2a85fd295 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:33:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-b94ef807-e87d-440b-8e75-b076314fa0db req-90256b75-aa82-47ec-ac42-d242c18ae8e1 service nova] Acquiring lock "f37b7fb7-1b94-421b-be79-7066010efabf-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:33:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-b94ef807-e87d-440b-8e75-b076314fa0db req-90256b75-aa82-47ec-ac42-d242c18ae8e1 service nova] Lock "f37b7fb7-1b94-421b-be79-7066010efabf-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:33:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-b94ef807-e87d-440b-8e75-b076314fa0db req-90256b75-aa82-47ec-ac42-d242c18ae8e1 service nova] Lock "f37b7fb7-1b94-421b-be79-7066010efabf-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:33:58 user nova-compute[72351]: DEBUG nova.compute.manager [req-b94ef807-e87d-440b-8e75-b076314fa0db req-90256b75-aa82-47ec-ac42-d242c18ae8e1 service nova] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] No waiting events found dispatching network-vif-unplugged-1f2da7a0-a0f0-4715-8c5d-1df2a85fd295 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:33:58 user nova-compute[72351]: DEBUG nova.compute.manager [req-b94ef807-e87d-440b-8e75-b076314fa0db req-90256b75-aa82-47ec-ac42-d242c18ae8e1 service nova] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Received event network-vif-unplugged-1f2da7a0-a0f0-4715-8c5d-1df2a85fd295 for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:33:58 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:58 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:58 user nova-compute[72351]: DEBUG nova.network.neutron [req-6d03a010-db8b-4938-8c7d-a7fa0a716f32 req-86f32553-6eba-4965-8b59-ef67ffce1a09 service nova] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Updated VIF entry in instance network info cache for port 1f2da7a0-a0f0-4715-8c5d-1df2a85fd295. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:33:58 user nova-compute[72351]: DEBUG nova.network.neutron [req-6d03a010-db8b-4938-8c7d-a7fa0a716f32 req-86f32553-6eba-4965-8b59-ef67ffce1a09 service nova] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Updating instance_info_cache with network_info: [{"id": "1f2da7a0-a0f0-4715-8c5d-1df2a85fd295", "address": "fa:16:3e:0a:ba:2e", "network": {"id": "49bfc70e-93c7-4607-9507-4711bda1a686", "bridge": "br-int", "label": "tempest-TestStampPattern-1656229455-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fc12f9037cd54bb6a2fdff3071132dd4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1f2da7a0-a0", "ovs_interfaceid": "1f2da7a0-a0f0-4715-8c5d-1df2a85fd295", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:33:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-6d03a010-db8b-4938-8c7d-a7fa0a716f32 req-86f32553-6eba-4965-8b59-ef67ffce1a09 service nova] Releasing lock "refresh_cache-f37b7fb7-1b94-421b-be79-7066010efabf" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:33:59 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Instance destroyed successfully.
Aug 09 02:33:59 user nova-compute[72351]: DEBUG nova.objects.instance [None req-5d163df9-f461-4285-a89e-a855b9bab28a tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lazy-loading 'resources' on Instance uuid f37b7fb7-1b94-421b-be79-7066010efabf {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:33:59 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-5d163df9-f461-4285-a89e-a855b9bab28a tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:30:50Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestStampPattern-server-1757563702',display_name='tempest-TestStampPattern-server-1757563702',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-teststamppattern-server-1757563702',id=36,image_ref='abaa1849-3122-41bc-9c40-d1d80b6911e7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBPzyrlB54Gi/8YMv0Cfs/esdmr1xNOdiloZGKR/XYFcg8Nos/w3Zm8ZOseVT8bKKFqeq3DP2KQSbuWg07AAvODzdTY0X+5ZdNLlP+cvLLuP4ef0UvshCJkUtWTANxgqmUw==',key_name='tempest-TestStampPattern-2061411239',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:30:59Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='fc12f9037cd54bb6a2fdff3071132dd4',ramdisk_id='',reservation_id='r-qrv9pcf2',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_boot_roles='reader,member',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_image_location='snapshot',image_image_state='available',image_image_type='snapshot',image_instance_uuid='a03fbded-23ad-440b-97e7-bea07425e873',image_min_disk='1',image_min_ram='0',image_owner_id='fc12f9037cd54bb6a2fdff3071132dd4',image_owner_project_name='tempest-TestStampPattern-1244733964',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',image_owner_user_name='tempest-TestStampPattern-1244733964-project-member',image_user_id='a103ebb9b63f4a259163ec25a33a0dcc',owner_project_name='tempest-TestStampPattern-1244733964',owner_user_name='tempest-TestStampPattern-1244733964-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:30:59Z,user_data=None,user_id='a103ebb9b63f4a259163ec25a33a0dcc',uuid=f37b7fb7-1b94-421b-be79-7066010efabf,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "1f2da7a0-a0f0-4715-8c5d-1df2a85fd295", "address": "fa:16:3e:0a:ba:2e", "network": {"id": "49bfc70e-93c7-4607-9507-4711bda1a686", "bridge": "br-int", "label": "tempest-TestStampPattern-1656229455-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.112", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fc12f9037cd54bb6a2fdff3071132dd4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1f2da7a0-a0", "ovs_interfaceid": "1f2da7a0-a0f0-4715-8c5d-1df2a85fd295", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:33:59 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-5d163df9-f461-4285-a89e-a855b9bab28a tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Converting VIF {"id": "1f2da7a0-a0f0-4715-8c5d-1df2a85fd295", "address": "fa:16:3e:0a:ba:2e", "network": {"id": "49bfc70e-93c7-4607-9507-4711bda1a686", "bridge": "br-int", "label": "tempest-TestStampPattern-1656229455-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.112", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fc12f9037cd54bb6a2fdff3071132dd4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1f2da7a0-a0", "ovs_interfaceid": "1f2da7a0-a0f0-4715-8c5d-1df2a85fd295", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:33:59 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-5d163df9-f461-4285-a89e-a855b9bab28a tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:0a:ba:2e,bridge_name='br-int',has_traffic_filtering=True,id=1f2da7a0-a0f0-4715-8c5d-1df2a85fd295,network=Network(49bfc70e-93c7-4607-9507-4711bda1a686),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1f2da7a0-a0') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:33:59 user nova-compute[72351]: DEBUG os_vif [None req-5d163df9-f461-4285-a89e-a855b9bab28a tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:0a:ba:2e,bridge_name='br-int',has_traffic_filtering=True,id=1f2da7a0-a0f0-4715-8c5d-1df2a85fd295,network=Network(49bfc70e-93c7-4607-9507-4711bda1a686),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1f2da7a0-a0') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:33:59 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:59 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap1f2da7a0-a0, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:33:59 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:59 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:33:59 user nova-compute[72351]: INFO os_vif [None req-5d163df9-f461-4285-a89e-a855b9bab28a tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:0a:ba:2e,bridge_name='br-int',has_traffic_filtering=True,id=1f2da7a0-a0f0-4715-8c5d-1df2a85fd295,network=Network(49bfc70e-93c7-4607-9507-4711bda1a686),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1f2da7a0-a0')
Aug 09 02:33:59 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-5d163df9-f461-4285-a89e-a855b9bab28a tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Deleting instance files /opt/stack/data/nova/instances/f37b7fb7-1b94-421b-be79-7066010efabf_del
Aug 09 02:33:59 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-5d163df9-f461-4285-a89e-a855b9bab28a tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Deletion of /opt/stack/data/nova/instances/f37b7fb7-1b94-421b-be79-7066010efabf_del complete
Aug 09 02:33:59 user nova-compute[72351]: INFO nova.compute.manager [None req-5d163df9-f461-4285-a89e-a855b9bab28a tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Took 0.89 seconds to destroy the instance on the hypervisor.
Aug 09 02:33:59 user nova-compute[72351]: DEBUG oslo.service.loopingcall [None req-5d163df9-f461-4285-a89e-a855b9bab28a tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72351) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Aug 09 02:33:59 user nova-compute[72351]: DEBUG nova.compute.manager [-] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Deallocating network for instance {{(pid=72351) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2263}}
Aug 09 02:33:59 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] deallocate_for_instance() {{(pid=72351) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1795}}
Aug 09 02:33:59 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:33:59 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Took 0.46 seconds to deallocate network for instance.
Aug 09 02:34:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5d163df9-f461-4285-a89e-a855b9bab28a tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:34:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5d163df9-f461-4285-a89e-a855b9bab28a tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:34:00 user nova-compute[72351]: DEBUG nova.compute.manager [req-5d12aba3-5343-4f67-98fb-120c13989742 req-dbc596c3-4500-45e8-9269-acf10e5e8afd service nova] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Received event network-vif-deleted-1f2da7a0-a0f0-4715-8c5d-1df2a85fd295 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:34:00 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-5d163df9-f461-4285-a89e-a855b9bab28a tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:34:00 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-5d163df9-f461-4285-a89e-a855b9bab28a tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:34:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5d163df9-f461-4285-a89e-a855b9bab28a tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.440s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:34:00 user nova-compute[72351]: INFO nova.scheduler.client.report [None req-5d163df9-f461-4285-a89e-a855b9bab28a tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Deleted allocations for instance f37b7fb7-1b94-421b-be79-7066010efabf
Aug 09 02:34:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5d163df9-f461-4285-a89e-a855b9bab28a tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lock "f37b7fb7-1b94-421b-be79-7066010efabf" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.285s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:34:00 user nova-compute[72351]: DEBUG nova.compute.manager [req-160d4d1a-788c-44f1-b082-1b29c8d19cfe req-a3d836c6-034c-4e9c-935f-64bcd36f90df service nova] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Received event network-vif-plugged-1f2da7a0-a0f0-4715-8c5d-1df2a85fd295 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:34:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-160d4d1a-788c-44f1-b082-1b29c8d19cfe req-a3d836c6-034c-4e9c-935f-64bcd36f90df service nova] Acquiring lock "f37b7fb7-1b94-421b-be79-7066010efabf-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:34:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-160d4d1a-788c-44f1-b082-1b29c8d19cfe req-a3d836c6-034c-4e9c-935f-64bcd36f90df service nova] Lock "f37b7fb7-1b94-421b-be79-7066010efabf-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:34:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-160d4d1a-788c-44f1-b082-1b29c8d19cfe req-a3d836c6-034c-4e9c-935f-64bcd36f90df service nova] Lock "f37b7fb7-1b94-421b-be79-7066010efabf-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:34:00 user nova-compute[72351]: DEBUG nova.compute.manager [req-160d4d1a-788c-44f1-b082-1b29c8d19cfe req-a3d836c6-034c-4e9c-935f-64bcd36f90df service nova] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] No waiting events found dispatching network-vif-plugged-1f2da7a0-a0f0-4715-8c5d-1df2a85fd295 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:34:00 user nova-compute[72351]: WARNING nova.compute.manager [req-160d4d1a-788c-44f1-b082-1b29c8d19cfe req-a3d836c6-034c-4e9c-935f-64bcd36f90df service nova] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Received unexpected event network-vif-plugged-1f2da7a0-a0f0-4715-8c5d-1df2a85fd295 for instance with vm_state deleted and task_state None.
Aug 09 02:34:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:34:03 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:34:03 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72351) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10461}}
Aug 09 02:34:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:34:04 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:34:06 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:34:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:34:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:34:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:34:06 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Auditing locally available compute resources for user (node: user) {{(pid=72351) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Aug 09 02:34:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7bef74f4-3bb3-4ad0-bff7-34156be31249/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:34:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7bef74f4-3bb3-4ad0-bff7-34156be31249/disk --force-share --output=json" returned: 0 in 0.143s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:34:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7bef74f4-3bb3-4ad0-bff7-34156be31249/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:34:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7bef74f4-3bb3-4ad0-bff7-34156be31249/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:34:07 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdh (vdb) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:34:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/41b5e305-3973-4937-b262-d32db1091e20/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:34:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/41b5e305-3973-4937-b262-d32db1091e20/disk --force-share --output=json" returned: 0 in 0.146s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:34:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/41b5e305-3973-4937-b262-d32db1091e20/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:34:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/41b5e305-3973-4937-b262-d32db1091e20/disk --force-share --output=json" returned: 0 in 0.146s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:34:07 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sde (vda) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:34:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:34:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:34:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:34:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json" returned: 0 in 0.139s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:34:08 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdc (vda) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:34:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/47afce95-e5f7-4703-a99f-7ef8b28c4bd2/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:34:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/47afce95-e5f7-4703-a99f-7ef8b28c4bd2/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:34:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/47afce95-e5f7-4703-a99f-7ef8b28c4bd2/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:34:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/47afce95-e5f7-4703-a99f-7ef8b28c4bd2/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:34:08 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdf (vda) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:34:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.rescue --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:34:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.rescue --force-share --output=json" returned: 0 in 0.139s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:34:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.rescue --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:34:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:34:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.rescue --force-share --output=json" returned: 0 in 0.144s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:34:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:34:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json" returned: 0 in 0.155s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:34:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:34:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:34:08 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdg (vda) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:34:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7b6f5d60-5f23-4ef1-b972-9ba7086f56cb/disk.rescue --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:34:09 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7b6f5d60-5f23-4ef1-b972-9ba7086f56cb/disk.rescue --force-share --output=json" returned: 0 in 0.137s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:34:09 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7b6f5d60-5f23-4ef1-b972-9ba7086f56cb/disk.rescue --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:34:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:34:09 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7b6f5d60-5f23-4ef1-b972-9ba7086f56cb/disk.rescue --force-share --output=json" returned: 0 in 0.135s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:34:09 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /opt/stack/data/nova/instances/7b6f5d60-5f23-4ef1-b972-9ba7086f56cb/disk (vdb) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:34:09 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:34:09 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json" returned: 0 in 0.151s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:34:09 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:34:09 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json" returned: 0 in 0.133s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:34:09 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a03fbded-23ad-440b-97e7-bea07425e873/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:34:09 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a03fbded-23ad-440b-97e7-bea07425e873/disk --force-share --output=json" returned: 0 in 0.144s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:34:09 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a03fbded-23ad-440b-97e7-bea07425e873/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:34:09 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a03fbded-23ad-440b-97e7-bea07425e873/disk --force-share --output=json" returned: 0 in 0.151s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:34:09 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdd (vda) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:34:09 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:34:09 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1cccb138-f2f1-4798-a502-bafe90cc10f1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Acquiring lock "7bef74f4-3bb3-4ad0-bff7-34156be31249" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:34:09 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1cccb138-f2f1-4798-a502-bafe90cc10f1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "7bef74f4-3bb3-4ad0-bff7-34156be31249" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:34:09 user nova-compute[72351]: INFO nova.compute.manager [None req-1cccb138-f2f1-4798-a502-bafe90cc10f1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Detaching volume 8ecec058-a02b-413c-acca-556f68fa0d19
Aug 09 02:34:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json" returned: 0 in 0.146s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:34:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:34:10 user nova-compute[72351]: INFO nova.virt.block_device [None req-1cccb138-f2f1-4798-a502-bafe90cc10f1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Attempting to driver detach volume 8ecec058-a02b-413c-acca-556f68fa0d19 from mountpoint /dev/vdb
Aug 09 02:34:10 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-1cccb138-f2f1-4798-a502-bafe90cc10f1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Attempting to detach device vdb from instance 7bef74f4-3bb3-4ad0-bff7-34156be31249 from the persistent domain config. {{(pid=72351) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2477}}
Aug 09 02:34:10 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-1cccb138-f2f1-4798-a502-bafe90cc10f1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 02:34:10 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:34:10 user nova-compute[72351]:   <source dev="/dev/sdh"/>
Aug 09 02:34:10 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:34:10 user nova-compute[72351]:   <serial>8ecec058-a02b-413c-acca-556f68fa0d19</serial>
Aug 09 02:34:10 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Aug 09 02:34:10 user nova-compute[72351]: </disk>
Aug 09 02:34:10 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 02:34:10 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-1cccb138-f2f1-4798-a502-bafe90cc10f1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Successfully detached device vdb from instance 7bef74f4-3bb3-4ad0-bff7-34156be31249 from the persistent domain config.
Aug 09 02:34:10 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-1cccb138-f2f1-4798-a502-bafe90cc10f1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 7bef74f4-3bb3-4ad0-bff7-34156be31249 from the live domain config. {{(pid=72351) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2513}}
Aug 09 02:34:10 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-1cccb138-f2f1-4798-a502-bafe90cc10f1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 02:34:10 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:34:10 user nova-compute[72351]:   <source dev="/dev/sdh"/>
Aug 09 02:34:10 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:34:10 user nova-compute[72351]:   <serial>8ecec058-a02b-413c-acca-556f68fa0d19</serial>
Aug 09 02:34:10 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Aug 09 02:34:10 user nova-compute[72351]: </disk>
Aug 09 02:34:10 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 02:34:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json" returned: 0 in 0.140s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:34:10 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Received event <DeviceRemovedEvent: 1691548450.5713272, 7bef74f4-3bb3-4ad0-bff7-34156be31249 => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=72351) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2360}}
Aug 09 02:34:10 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-1cccb138-f2f1-4798-a502-bafe90cc10f1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 7bef74f4-3bb3-4ad0-bff7-34156be31249 {{(pid=72351) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2589}}
Aug 09 02:34:10 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-1cccb138-f2f1-4798-a502-bafe90cc10f1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Successfully detached device vdb from instance 7bef74f4-3bb3-4ad0-bff7-34156be31249 from the live domain config.
Aug 09 02:34:10 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:34:10 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:34:10 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Hypervisor/Node resource view: name=user free_ram=6756MB free_disk=25.005752563476562GB free_vcpus=-2 pci_devices=[{"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72351) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Aug 09 02:34:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:34:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:34:10 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-1cccb138-f2f1-4798-a502-bafe90cc10f1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 02:34:10 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1cccb138-f2f1-4798-a502-bafe90cc10f1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:8ecec058-a02b-413c-acca-556f68fa0d19'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'vFKkMVmH', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdh'}, None), 'kwargs': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:34:10 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-1cccb138-f2f1-4798-a502-bafe90cc10f1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:34:10 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-1cccb138-f2f1-4798-a502-bafe90cc10f1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:34:10 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-1cccb138-f2f1-4798-a502-bafe90cc10f1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:34:10 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1cccb138-f2f1-4798-a502-bafe90cc10f1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:8ecec058-a02b-413c-acca-556f68fa0d19', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 02:34:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:34:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:34:10 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[64c21e95-f1a2-4495-a3ec-14acf2e32a1e]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:8ecec058-a02b-413c-acca-556f68fa0d19\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:34:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:34:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.028s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:34:10 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[76a4403d-ccb5-4af6-8a46-c2ba1ef276d3]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)\ntcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc (non-flash)\ntcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)\ntcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)\ntcp: [45] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ecec058-a02b-413c-acca-556f68fa0d19 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:34:10 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1cccb138-f2f1-4798-a502-bafe90cc10f1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:34:10 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:34:10 user nova-compute[72351]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)
Aug 09 02:34:10 user nova-compute[72351]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc (non-flash)
Aug 09 02:34:10 user nova-compute[72351]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:34:10 user nova-compute[72351]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)
Aug 09 02:34:10 user nova-compute[72351]: tcp: [45] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ecec058-a02b-413c-acca-556f68fa0d19 (non-flash)
Aug 09 02:34:10 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:34:10 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1cccb138-f2f1-4798-a502-bafe90cc10f1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:34:10 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:34:10 user nova-compute[72351]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)
Aug 09 02:34:10 user nova-compute[72351]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc (non-flash)
Aug 09 02:34:10 user nova-compute[72351]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:34:10 user nova-compute[72351]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)
Aug 09 02:34:10 user nova-compute[72351]: tcp: [45] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ecec058-a02b-413c-acca-556f68fa0d19 (non-flash)
Aug 09 02:34:10 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:34:10 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1cccb138-f2f1-4798-a502-bafe90cc10f1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fcffc1b63b0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:8ecec058-a02b-413c-acca-556f68fa0d19'): ({'sdh'}, set())}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 02:34:10 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-1cccb138-f2f1-4798-a502-bafe90cc10f1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Removing single pathed devices sdh {{(pid=72351) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Aug 09 02:34:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:34:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:34:10 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[4324192e-b2a2-477e-aea0-3f8e5cbaf938]: (4, ('path checker states:\nup                  7\n\npaths: 0\nbusy: False\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:34:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdh {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:34:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdh" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:34:10 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[38015b02-b356-4ebb-9db8-a8fade1ac61d]: (4, ('ok\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:34:10 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-1cccb138-f2f1-4798-a502-bafe90cc10f1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Flushing IO for device /dev/sdh {{(pid=72351) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Aug 09 02:34:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdh {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:34:10 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Applying migration context for instance 80e87454-90b9-4168-bf00-b7d93b245526 as it has an incoming, in-progress migration 5464193d-df16-4ccd-b137-4df0859f7495. Migration status is finished {{(pid=72351) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1016}}
Aug 09 02:34:10 user nova-compute[72351]: INFO nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Updating resource usage from migration 5464193d-df16-4ccd-b137-4df0859f7495
Aug 09 02:34:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdh" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:34:10 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[b10b4c69-f5c5-4caf-9cba-0db8d9471082]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:34:10 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-1cccb138-f2f1-4798-a502-bafe90cc10f1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Remove SCSI device /dev/sdh with /sys/block/sdh/device/delete {{(pid=72351) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Aug 09 02:34:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdh/device/delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:34:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdh/device/delete" returned: 0 in 0.041s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:34:10 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[5e2f2e06-0e57-4915-a275-2662f9559879]: (4, ('1', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:34:10 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-1cccb138-f2f1-4798-a502-bafe90cc10f1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Checking to see if SCSI volumes sdh have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Aug 09 02:34:10 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-1cccb138-f2f1-4798-a502-bafe90cc10f1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] SCSI volumes sdh have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Aug 09 02:34:10 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1cccb138-f2f1-4798-a502-bafe90cc10f1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:8ecec058-a02b-413c-acca-556f68fa0d19')] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 02:34:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8ecec058-a02b-413c-acca-556f68fa0d19 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:34:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8ecec058-a02b-413c-acca-556f68fa0d19 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:34:10 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[5db6e503-ee50-4e1a-8f4a-ee93c7f28bfa]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:34:10 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1cccb138-f2f1-4798-a502-bafe90cc10f1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:34:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8ecec058-a02b-413c-acca-556f68fa0d19 -p 172.16.0.220:3260 --logout {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:34:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8ecec058-a02b-413c-acca-556f68fa0d19 -p 172.16.0.220:3260 --logout" returned: 0 in 0.079s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:34:11 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[49e2f31a-1762-4493-8e83-96d957d7d0d4]: (4, ('Logging out of session [sid: 45, target: iqn.2010-10.org.openstack:8ecec058-a02b-413c-acca-556f68fa0d19, portal: 172.16.0.220,3260]\nLogout of [sid: 45, target: iqn.2010-10.org.openstack:8ecec058-a02b-413c-acca-556f68fa0d19, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:34:11 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1cccb138-f2f1-4798-a502-bafe90cc10f1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 45, target: iqn.2010-10.org.openstack:8ecec058-a02b-413c-acca-556f68fa0d19, portal: 172.16.0.220,3260]
Aug 09 02:34:11 user nova-compute[72351]: Logout of [sid: 45, target: iqn.2010-10.org.openstack:8ecec058-a02b-413c-acca-556f68fa0d19, portal: 172.16.0.220,3260] successful.
Aug 09 02:34:11 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:34:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8ecec058-a02b-413c-acca-556f68fa0d19 -p 172.16.0.220:3260 --op delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:34:11 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:34:11 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance eb86cb49-2018-42c2-bd97-c662b829775d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:34:11 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:34:11 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance a7f3d076-864b-4b68-9207-214ce2c21444 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:34:11 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 47afce95-e5f7-4703-a99f-7ef8b28c4bd2 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:34:11 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 957c1510-1320-4492-9052-a58ff540319d actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:34:11 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance a03fbded-23ad-440b-97e7-bea07425e873 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:34:11 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:34:11 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 49a10075-0d3c-4cd6-bb43-10238cad3f37 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:34:11 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 7bef74f4-3bb3-4ad0-bff7-34156be31249 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:34:11 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 41b5e305-3973-4937-b262-d32db1091e20 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:34:11 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:34:11 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Migration 5464193d-df16-4ccd-b137-4df0859f7495 is active on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1709}}
Aug 09 02:34:11 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 80e87454-90b9-4168-bf00-b7d93b245526 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 2048, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:34:11 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance ad9b389e-ea11-4fd7-9233-a1be43052e60 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:34:11 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Total usable vcpus: 12, total allocated vcpus: 15 {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Aug 09 02:34:11 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Final resource view: name=user phys_ram=16011MB used_ram=9728MB phys_disk=40GB used_disk=9GB total_vcpus=12 used_vcpus=15 pci_stats=[] {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Aug 09 02:34:11 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Refreshing inventories for resource provider 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:804}}
Aug 09 02:34:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8ecec058-a02b-413c-acca-556f68fa0d19 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.036s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:34:11 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ce9366e1-68b9-426d-8fc6-29c10b94bbe3]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:34:11 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1cccb138-f2f1-4798-a502-bafe90cc10f1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:34:11 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-1cccb138-f2f1-4798-a502-bafe90cc10f1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.341s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:34:11 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1cccb138-f2f1-4798-a502-bafe90cc10f1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] <== disconnect_volume: return (343ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:34:11 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-1cccb138-f2f1-4798-a502-bafe90cc10f1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 02:34:11 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Updating ProviderTree inventory for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 from _refresh_and_get_inventory using data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) _refresh_and_get_inventory /opt/stack/nova/nova/scheduler/client/report.py:768}}
Aug 09 02:34:11 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Updating inventory in ProviderTree for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 with inventory: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:176}}
Aug 09 02:34:11 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Refreshing aggregate associations for resource provider 8eae9b37-1236-4e6d-91a5-b650de27b009, aggregates: None {{(pid=72351) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:813}}
Aug 09 02:34:11 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Refreshing trait associations for resource provider 8eae9b37-1236-4e6d-91a5-b650de27b009, traits: COMPUTE_VIOMMU_MODEL_INTEL,COMPUTE_STORAGE_BUS_IDE,COMPUTE_GRAPHICS_MODEL_NONE,COMPUTE_NET_VIF_MODEL_E1000E,COMPUTE_IMAGE_TYPE_AKI,COMPUTE_VOLUME_ATTACH_WITH_TAG,COMPUTE_NET_VIF_MODEL_VMXNET3,COMPUTE_SOCKET_PCI_NUMA_AFFINITY,COMPUTE_ACCELERATORS,HW_CPU_X86_SSE41,COMPUTE_NET_ATTACH_INTERFACE_WITH_TAG,COMPUTE_GRAPHICS_MODEL_VGA,COMPUTE_GRAPHICS_MODEL_QXL,COMPUTE_VIOMMU_MODEL_AUTO,COMPUTE_SECURITY_UEFI_SECURE_BOOT,COMPUTE_NET_VIF_MODEL_E1000,COMPUTE_TRUSTED_CERTS,COMPUTE_VOLUME_MULTI_ATTACH,COMPUTE_STORAGE_BUS_FDC,COMPUTE_VOLUME_EXTEND,COMPUTE_GRAPHICS_MODEL_CIRRUS,COMPUTE_GRAPHICS_MODEL_VMVGA,COMPUTE_GRAPHICS_MODEL_BOCHS,COMPUTE_DEVICE_TAGGING,COMPUTE_STORAGE_BUS_SCSI,COMPUTE_STORAGE_BUS_VIRTIO,HW_CPU_X86_SSSE3,HW_CPU_X86_SSE2,COMPUTE_IMAGE_TYPE_AMI,COMPUTE_STORAGE_BUS_USB,COMPUTE_NET_VIF_MODEL_PCNET,COMPUTE_NET_VIF_MODEL_VIRTIO,COMPUTE_IMAGE_TYPE_ISO,HW_CPU_X86_SSE42,HW_CPU_X86_SSE,COMPUTE_IMAGE_TYPE_ARI,COMPUTE_IMAGE_TYPE_RAW,COMPUTE_IMAGE_TYPE_QCOW2,COMPUTE_NET_VIF_MODEL_NE2K_PCI,COMPUTE_NODE,HW_CPU_X86_MMX,COMPUTE_NET_VIF_MODEL_RTL8139,COMPUTE_GRAPHICS_MODEL_VIRTIO,COMPUTE_NET_VIF_MODEL_SPAPR_VLAN,COMPUTE_STORAGE_BUS_SATA,COMPUTE_NET_ATTACH_INTERFACE,COMPUTE_RESCUE_BFV {{(pid=72351) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:825}}
Aug 09 02:34:11 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:34:11 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:34:11 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Compute_service record updated for user:user {{(pid=72351) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Aug 09 02:34:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.836s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:34:11 user nova-compute[72351]: DEBUG nova.objects.instance [None req-1cccb138-f2f1-4798-a502-bafe90cc10f1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lazy-loading 'flavor' on Instance uuid 7bef74f4-3bb3-4ad0-bff7-34156be31249 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:34:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1cccb138-f2f1-4798-a502-bafe90cc10f1 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "7bef74f4-3bb3-4ad0-bff7-34156be31249" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.941s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:34:12 user nova-compute[72351]: INFO nova.compute.manager [None req-b822faeb-8d7b-48d5-8f64-c8c6f56b6c33 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Unrescuing
Aug 09 02:34:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b822faeb-8d7b-48d5-8f64-c8c6f56b6c33 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Acquiring lock "refresh_cache-7b6f5d60-5f23-4ef1-b972-9ba7086f56cb" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:34:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b822faeb-8d7b-48d5-8f64-c8c6f56b6c33 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Acquired lock "refresh_cache-7b6f5d60-5f23-4ef1-b972-9ba7086f56cb" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:34:12 user nova-compute[72351]: DEBUG nova.network.neutron [None req-b822faeb-8d7b-48d5-8f64-c8c6f56b6c33 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:34:12 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:34:12 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:34:12 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Starting heal instance info cache {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9842}}
Aug 09 02:34:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "refresh_cache-47afce95-e5f7-4703-a99f-7ef8b28c4bd2" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:34:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquired lock "refresh_cache-47afce95-e5f7-4703-a99f-7ef8b28c4bd2" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:34:12 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Forcefully refreshing network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1996}}
Aug 09 02:34:12 user nova-compute[72351]: DEBUG nova.network.neutron [None req-b822faeb-8d7b-48d5-8f64-c8c6f56b6c33 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Updating instance_info_cache with network_info: [{"id": "c4a56310-1841-46ae-9a97-0421861e62c0", "address": "fa:16:3e:75:cf:ba", "network": {"id": "49e8751c-e119-430f-9110-3dc6069b1589", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-899275135-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.125", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3aea1e66805146f7a8cd1d97e64a24af", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc4a56310-18", "ovs_interfaceid": "c4a56310-1841-46ae-9a97-0421861e62c0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:34:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b822faeb-8d7b-48d5-8f64-c8c6f56b6c33 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Releasing lock "refresh_cache-7b6f5d60-5f23-4ef1-b972-9ba7086f56cb" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:34:12 user nova-compute[72351]: DEBUG nova.objects.instance [None req-b822faeb-8d7b-48d5-8f64-c8c6f56b6c33 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lazy-loading 'flavor' on Instance uuid 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:34:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:34:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:34:13 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Updating instance_info_cache with network_info: [{"id": "52e1842a-aa45-4804-9636-f0cfe1134eee", "address": "fa:16:3e:bd:df:05", "network": {"id": "9785f84c-cf45-464e-9127-332ce46abd11", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-87568186-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "eebc3a715ae04d48918b6c68b135935b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52e1842a-aa", "ovs_interfaceid": "52e1842a-aa45-4804-9636-f0cfe1134eee", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:34:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Releasing lock "refresh_cache-47afce95-e5f7-4703-a99f-7ef8b28c4bd2" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:34:13 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Updated the network info_cache for instance {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9913}}
Aug 09 02:34:13 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:34:13 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:34:13 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:34:13 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:34:13 user nova-compute[72351]: DEBUG nova.compute.manager [req-b16ebd1b-28ef-4a47-b0a0-a86ce95c47ac req-44552c1c-ec33-4cdc-b702-9da8f54837c0 service nova] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Received event network-vif-unplugged-c4a56310-1841-46ae-9a97-0421861e62c0 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:34:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-b16ebd1b-28ef-4a47-b0a0-a86ce95c47ac req-44552c1c-ec33-4cdc-b702-9da8f54837c0 service nova] Acquiring lock "7b6f5d60-5f23-4ef1-b972-9ba7086f56cb-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:34:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-b16ebd1b-28ef-4a47-b0a0-a86ce95c47ac req-44552c1c-ec33-4cdc-b702-9da8f54837c0 service nova] Lock "7b6f5d60-5f23-4ef1-b972-9ba7086f56cb-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:34:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-b16ebd1b-28ef-4a47-b0a0-a86ce95c47ac req-44552c1c-ec33-4cdc-b702-9da8f54837c0 service nova] Lock "7b6f5d60-5f23-4ef1-b972-9ba7086f56cb-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:34:13 user nova-compute[72351]: DEBUG nova.compute.manager [req-b16ebd1b-28ef-4a47-b0a0-a86ce95c47ac req-44552c1c-ec33-4cdc-b702-9da8f54837c0 service nova] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] No waiting events found dispatching network-vif-unplugged-c4a56310-1841-46ae-9a97-0421861e62c0 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:34:13 user nova-compute[72351]: WARNING nova.compute.manager [req-b16ebd1b-28ef-4a47-b0a0-a86ce95c47ac req-44552c1c-ec33-4cdc-b702-9da8f54837c0 service nova] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Received unexpected event network-vif-unplugged-c4a56310-1841-46ae-9a97-0421861e62c0 for instance with vm_state rescued and task_state unrescuing.
Aug 09 02:34:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:34:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:34:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:34:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:34:13 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Instance destroyed successfully.
Aug 09 02:34:14 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691548439.1647515, f37b7fb7-1b94-421b-be79-7066010efabf => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:34:14 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] VM Stopped (Lifecycle Event)
Aug 09 02:34:14 user nova-compute[72351]: DEBUG nova.compute.manager [None req-1718cfbb-85a9-4ae4-bf22-ba321abad982 None None] [instance: f37b7fb7-1b94-421b-be79-7066010efabf] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:34:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:34:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:34:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:34:15 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:34:15 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:34:15 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:34:15 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:34:15 user nova-compute[72351]: DEBUG nova.compute.manager [req-dc161e50-e66d-4b10-ba82-ed45707805bc req-b862cdec-775d-4da0-afd9-42b2464c4262 service nova] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Received event network-vif-plugged-c4a56310-1841-46ae-9a97-0421861e62c0 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:34:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-dc161e50-e66d-4b10-ba82-ed45707805bc req-b862cdec-775d-4da0-afd9-42b2464c4262 service nova] Acquiring lock "7b6f5d60-5f23-4ef1-b972-9ba7086f56cb-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:34:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-dc161e50-e66d-4b10-ba82-ed45707805bc req-b862cdec-775d-4da0-afd9-42b2464c4262 service nova] Lock "7b6f5d60-5f23-4ef1-b972-9ba7086f56cb-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:34:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-dc161e50-e66d-4b10-ba82-ed45707805bc req-b862cdec-775d-4da0-afd9-42b2464c4262 service nova] Lock "7b6f5d60-5f23-4ef1-b972-9ba7086f56cb-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:34:15 user nova-compute[72351]: DEBUG nova.compute.manager [req-dc161e50-e66d-4b10-ba82-ed45707805bc req-b862cdec-775d-4da0-afd9-42b2464c4262 service nova] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] No waiting events found dispatching network-vif-plugged-c4a56310-1841-46ae-9a97-0421861e62c0 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:34:15 user nova-compute[72351]: WARNING nova.compute.manager [req-dc161e50-e66d-4b10-ba82-ed45707805bc req-b862cdec-775d-4da0-afd9-42b2464c4262 service nova] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Received unexpected event network-vif-plugged-c4a56310-1841-46ae-9a97-0421861e62c0 for instance with vm_state rescued and task_state unrescuing.
Aug 09 02:34:15 user nova-compute[72351]: DEBUG nova.compute.manager [req-dc161e50-e66d-4b10-ba82-ed45707805bc req-b862cdec-775d-4da0-afd9-42b2464c4262 service nova] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Received event network-vif-plugged-c4a56310-1841-46ae-9a97-0421861e62c0 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:34:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-dc161e50-e66d-4b10-ba82-ed45707805bc req-b862cdec-775d-4da0-afd9-42b2464c4262 service nova] Acquiring lock "7b6f5d60-5f23-4ef1-b972-9ba7086f56cb-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:34:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-dc161e50-e66d-4b10-ba82-ed45707805bc req-b862cdec-775d-4da0-afd9-42b2464c4262 service nova] Lock "7b6f5d60-5f23-4ef1-b972-9ba7086f56cb-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:34:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-dc161e50-e66d-4b10-ba82-ed45707805bc req-b862cdec-775d-4da0-afd9-42b2464c4262 service nova] Lock "7b6f5d60-5f23-4ef1-b972-9ba7086f56cb-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:34:15 user nova-compute[72351]: DEBUG nova.compute.manager [req-dc161e50-e66d-4b10-ba82-ed45707805bc req-b862cdec-775d-4da0-afd9-42b2464c4262 service nova] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] No waiting events found dispatching network-vif-plugged-c4a56310-1841-46ae-9a97-0421861e62c0 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:34:15 user nova-compute[72351]: WARNING nova.compute.manager [req-dc161e50-e66d-4b10-ba82-ed45707805bc req-b862cdec-775d-4da0-afd9-42b2464c4262 service nova] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Received unexpected event network-vif-plugged-c4a56310-1841-46ae-9a97-0421861e62c0 for instance with vm_state rescued and task_state unrescuing.
Aug 09 02:34:17 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Removed pending event for 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb due to event {{(pid=72351) _event_emit_delayed /opt/stack/nova/nova/virt/libvirt/host.py:438}}
Aug 09 02:34:17 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548457.2931988, 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:34:17 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] VM Resumed (Lifecycle Event)
Aug 09 02:34:17 user nova-compute[72351]: DEBUG nova.compute.manager [None req-b822faeb-8d7b-48d5-8f64-c8c6f56b6c33 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:34:17 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:34:17 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: rescued, current task_state: unrescuing, current DB power_state: 1, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:34:17 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] During sync_power_state the instance has a pending task (unrescuing). Skip.
Aug 09 02:34:17 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548457.29544, 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:34:17 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] VM Started (Lifecycle Event)
Aug 09 02:34:17 user nova-compute[72351]: DEBUG nova.compute.manager [req-16e04fbe-159f-44b3-864c-8246c4648dba req-0d7ec700-defe-4f13-9110-0e5e92f61133 service nova] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Received event network-vif-plugged-c4a56310-1841-46ae-9a97-0421861e62c0 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:34:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-16e04fbe-159f-44b3-864c-8246c4648dba req-0d7ec700-defe-4f13-9110-0e5e92f61133 service nova] Acquiring lock "7b6f5d60-5f23-4ef1-b972-9ba7086f56cb-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:34:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-16e04fbe-159f-44b3-864c-8246c4648dba req-0d7ec700-defe-4f13-9110-0e5e92f61133 service nova] Lock "7b6f5d60-5f23-4ef1-b972-9ba7086f56cb-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:34:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-16e04fbe-159f-44b3-864c-8246c4648dba req-0d7ec700-defe-4f13-9110-0e5e92f61133 service nova] Lock "7b6f5d60-5f23-4ef1-b972-9ba7086f56cb-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:34:17 user nova-compute[72351]: DEBUG nova.compute.manager [req-16e04fbe-159f-44b3-864c-8246c4648dba req-0d7ec700-defe-4f13-9110-0e5e92f61133 service nova] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] No waiting events found dispatching network-vif-plugged-c4a56310-1841-46ae-9a97-0421861e62c0 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:34:17 user nova-compute[72351]: WARNING nova.compute.manager [req-16e04fbe-159f-44b3-864c-8246c4648dba req-0d7ec700-defe-4f13-9110-0e5e92f61133 service nova] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Received unexpected event network-vif-plugged-c4a56310-1841-46ae-9a97-0421861e62c0 for instance with vm_state rescued and task_state unrescuing.
Aug 09 02:34:17 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:34:17 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Synchronizing instance power state after lifecycle event "Started"; current vm_state: rescued, current task_state: unrescuing, current DB power_state: 1, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:34:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:34:19 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:34:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:34:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:34:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5003 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 02:34:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:34:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:34:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:34:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:34:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-573c77bf-0975-483d-95aa-c5dafd57f106 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Acquiring lock "47afce95-e5f7-4703-a99f-7ef8b28c4bd2" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:34:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-573c77bf-0975-483d-95aa-c5dafd57f106 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Lock "47afce95-e5f7-4703-a99f-7ef8b28c4bd2" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:34:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-573c77bf-0975-483d-95aa-c5dafd57f106 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Acquiring lock "47afce95-e5f7-4703-a99f-7ef8b28c4bd2-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:34:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-573c77bf-0975-483d-95aa-c5dafd57f106 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Lock "47afce95-e5f7-4703-a99f-7ef8b28c4bd2-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:34:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-573c77bf-0975-483d-95aa-c5dafd57f106 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Lock "47afce95-e5f7-4703-a99f-7ef8b28c4bd2-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:34:24 user nova-compute[72351]: INFO nova.compute.manager [None req-573c77bf-0975-483d-95aa-c5dafd57f106 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Terminating instance
Aug 09 02:34:24 user nova-compute[72351]: DEBUG nova.compute.manager [None req-573c77bf-0975-483d-95aa-c5dafd57f106 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Start destroying the instance on the hypervisor. {{(pid=72351) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3124}}
Aug 09 02:34:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:34:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:34:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:34:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:34:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:34:24 user nova-compute[72351]: DEBUG nova.compute.manager [req-6eea754b-e992-47cc-bdea-7d385fd35ed9 req-e5452d64-63eb-444c-a4bb-080102bf42eb service nova] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Received event network-vif-unplugged-52e1842a-aa45-4804-9636-f0cfe1134eee {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:34:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-6eea754b-e992-47cc-bdea-7d385fd35ed9 req-e5452d64-63eb-444c-a4bb-080102bf42eb service nova] Acquiring lock "47afce95-e5f7-4703-a99f-7ef8b28c4bd2-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:34:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-6eea754b-e992-47cc-bdea-7d385fd35ed9 req-e5452d64-63eb-444c-a4bb-080102bf42eb service nova] Lock "47afce95-e5f7-4703-a99f-7ef8b28c4bd2-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:34:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-6eea754b-e992-47cc-bdea-7d385fd35ed9 req-e5452d64-63eb-444c-a4bb-080102bf42eb service nova] Lock "47afce95-e5f7-4703-a99f-7ef8b28c4bd2-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:34:24 user nova-compute[72351]: DEBUG nova.compute.manager [req-6eea754b-e992-47cc-bdea-7d385fd35ed9 req-e5452d64-63eb-444c-a4bb-080102bf42eb service nova] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] No waiting events found dispatching network-vif-unplugged-52e1842a-aa45-4804-9636-f0cfe1134eee {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:34:24 user nova-compute[72351]: DEBUG nova.compute.manager [req-6eea754b-e992-47cc-bdea-7d385fd35ed9 req-e5452d64-63eb-444c-a4bb-080102bf42eb service nova] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Received event network-vif-unplugged-52e1842a-aa45-4804-9636-f0cfe1134eee for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:34:25 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Instance destroyed successfully.
Aug 09 02:34:25 user nova-compute[72351]: DEBUG nova.objects.instance [None req-573c77bf-0975-483d-95aa-c5dafd57f106 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Lazy-loading 'resources' on Instance uuid 47afce95-e5f7-4703-a99f-7ef8b28c4bd2 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:34:25 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-573c77bf-0975-483d-95aa-c5dafd57f106 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:25:27Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServersNegativeTestJSON-server-1699010745',display_name='tempest-ServersNegativeTestJSON-server-1699010745',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serversnegativetestjson-server-1699010745',id=26,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:25:35Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='eebc3a715ae04d48918b6c68b135935b',ramdisk_id='',reservation_id='r-lovurafx',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServersNegativeTestJSON-1773070288',owner_user_name='tempest-ServersNegativeTestJSON-1773070288-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:25:35Z,user_data=None,user_id='ebc62bdcafdd4131be9abb7fe4ed2422',uuid=47afce95-e5f7-4703-a99f-7ef8b28c4bd2,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "52e1842a-aa45-4804-9636-f0cfe1134eee", "address": "fa:16:3e:bd:df:05", "network": {"id": "9785f84c-cf45-464e-9127-332ce46abd11", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-87568186-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "eebc3a715ae04d48918b6c68b135935b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52e1842a-aa", "ovs_interfaceid": "52e1842a-aa45-4804-9636-f0cfe1134eee", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:34:25 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-573c77bf-0975-483d-95aa-c5dafd57f106 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Converting VIF {"id": "52e1842a-aa45-4804-9636-f0cfe1134eee", "address": "fa:16:3e:bd:df:05", "network": {"id": "9785f84c-cf45-464e-9127-332ce46abd11", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-87568186-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "eebc3a715ae04d48918b6c68b135935b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52e1842a-aa", "ovs_interfaceid": "52e1842a-aa45-4804-9636-f0cfe1134eee", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:34:25 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-573c77bf-0975-483d-95aa-c5dafd57f106 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:bd:df:05,bridge_name='br-int',has_traffic_filtering=True,id=52e1842a-aa45-4804-9636-f0cfe1134eee,network=Network(9785f84c-cf45-464e-9127-332ce46abd11),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap52e1842a-aa') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:34:25 user nova-compute[72351]: DEBUG os_vif [None req-573c77bf-0975-483d-95aa-c5dafd57f106 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:bd:df:05,bridge_name='br-int',has_traffic_filtering=True,id=52e1842a-aa45-4804-9636-f0cfe1134eee,network=Network(9785f84c-cf45-464e-9127-332ce46abd11),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap52e1842a-aa') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:34:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:34:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap52e1842a-aa, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:34:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:34:25 user nova-compute[72351]: INFO os_vif [None req-573c77bf-0975-483d-95aa-c5dafd57f106 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:bd:df:05,bridge_name='br-int',has_traffic_filtering=True,id=52e1842a-aa45-4804-9636-f0cfe1134eee,network=Network(9785f84c-cf45-464e-9127-332ce46abd11),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap52e1842a-aa')
Aug 09 02:34:25 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-573c77bf-0975-483d-95aa-c5dafd57f106 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Deleting instance files /opt/stack/data/nova/instances/47afce95-e5f7-4703-a99f-7ef8b28c4bd2_del
Aug 09 02:34:25 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-573c77bf-0975-483d-95aa-c5dafd57f106 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Deletion of /opt/stack/data/nova/instances/47afce95-e5f7-4703-a99f-7ef8b28c4bd2_del complete
Aug 09 02:34:25 user nova-compute[72351]: INFO nova.compute.manager [None req-573c77bf-0975-483d-95aa-c5dafd57f106 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Took 0.86 seconds to destroy the instance on the hypervisor.
Aug 09 02:34:25 user nova-compute[72351]: DEBUG oslo.service.loopingcall [None req-573c77bf-0975-483d-95aa-c5dafd57f106 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72351) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Aug 09 02:34:25 user nova-compute[72351]: DEBUG nova.compute.manager [-] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Deallocating network for instance {{(pid=72351) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2263}}
Aug 09 02:34:25 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] deallocate_for_instance() {{(pid=72351) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1795}}
Aug 09 02:34:26 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:34:26 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Took 0.53 seconds to deallocate network for instance.
Aug 09 02:34:26 user nova-compute[72351]: DEBUG nova.compute.manager [req-9b709456-f5a2-406a-845d-b8975c6ee54a req-fe7f943d-294e-469f-975f-1d5b740b7369 service nova] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Received event network-vif-deleted-52e1842a-aa45-4804-9636-f0cfe1134eee {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:34:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-573c77bf-0975-483d-95aa-c5dafd57f106 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:34:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-573c77bf-0975-483d-95aa-c5dafd57f106 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:34:26 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-573c77bf-0975-483d-95aa-c5dafd57f106 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:34:26 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-573c77bf-0975-483d-95aa-c5dafd57f106 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:34:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-573c77bf-0975-483d-95aa-c5dafd57f106 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.402s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:34:26 user nova-compute[72351]: INFO nova.scheduler.client.report [None req-573c77bf-0975-483d-95aa-c5dafd57f106 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Deleted allocations for instance 47afce95-e5f7-4703-a99f-7ef8b28c4bd2
Aug 09 02:34:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-573c77bf-0975-483d-95aa-c5dafd57f106 tempest-ServersNegativeTestJSON-1773070288 tempest-ServersNegativeTestJSON-1773070288-project-member] Lock "47afce95-e5f7-4703-a99f-7ef8b28c4bd2" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 1.995s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:34:27 user nova-compute[72351]: DEBUG nova.compute.manager [req-42da01f6-b8b8-4f5c-8fe4-b5eda113d0bd req-ff795b8e-6a52-4b25-be64-c738b597ae79 service nova] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Received event network-vif-plugged-52e1842a-aa45-4804-9636-f0cfe1134eee {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:34:27 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-42da01f6-b8b8-4f5c-8fe4-b5eda113d0bd req-ff795b8e-6a52-4b25-be64-c738b597ae79 service nova] Acquiring lock "47afce95-e5f7-4703-a99f-7ef8b28c4bd2-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:34:27 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-42da01f6-b8b8-4f5c-8fe4-b5eda113d0bd req-ff795b8e-6a52-4b25-be64-c738b597ae79 service nova] Lock "47afce95-e5f7-4703-a99f-7ef8b28c4bd2-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:34:27 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-42da01f6-b8b8-4f5c-8fe4-b5eda113d0bd req-ff795b8e-6a52-4b25-be64-c738b597ae79 service nova] Lock "47afce95-e5f7-4703-a99f-7ef8b28c4bd2-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:34:27 user nova-compute[72351]: DEBUG nova.compute.manager [req-42da01f6-b8b8-4f5c-8fe4-b5eda113d0bd req-ff795b8e-6a52-4b25-be64-c738b597ae79 service nova] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] No waiting events found dispatching network-vif-plugged-52e1842a-aa45-4804-9636-f0cfe1134eee {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:34:27 user nova-compute[72351]: WARNING nova.compute.manager [req-42da01f6-b8b8-4f5c-8fe4-b5eda113d0bd req-ff795b8e-6a52-4b25-be64-c738b597ae79 service nova] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Received unexpected event network-vif-plugged-52e1842a-aa45-4804-9636-f0cfe1134eee for instance with vm_state deleted and task_state None.
Aug 09 02:34:30 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:34:33 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:34:35 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:34:36 user nova-compute[72351]: DEBUG nova.compute.manager [req-4746a39c-926d-4a70-882d-adcb2b4b9b37 req-e9c3c265-472d-447a-a0bd-f571ca44f8b6 service nova] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Received event network-changed-00868cee-ead4-4f80-a2c7-1ec86fd0d78f {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:34:36 user nova-compute[72351]: DEBUG nova.compute.manager [req-4746a39c-926d-4a70-882d-adcb2b4b9b37 req-e9c3c265-472d-447a-a0bd-f571ca44f8b6 service nova] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Refreshing instance network info cache due to event network-changed-00868cee-ead4-4f80-a2c7-1ec86fd0d78f. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:34:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-4746a39c-926d-4a70-882d-adcb2b4b9b37 req-e9c3c265-472d-447a-a0bd-f571ca44f8b6 service nova] Acquiring lock "refresh_cache-8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:34:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-4746a39c-926d-4a70-882d-adcb2b4b9b37 req-e9c3c265-472d-447a-a0bd-f571ca44f8b6 service nova] Acquired lock "refresh_cache-8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:34:36 user nova-compute[72351]: DEBUG nova.network.neutron [req-4746a39c-926d-4a70-882d-adcb2b4b9b37 req-e9c3c265-472d-447a-a0bd-f571ca44f8b6 service nova] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Refreshing network info cache for port 00868cee-ead4-4f80-a2c7-1ec86fd0d78f {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:34:37 user nova-compute[72351]: DEBUG nova.network.neutron [req-4746a39c-926d-4a70-882d-adcb2b4b9b37 req-e9c3c265-472d-447a-a0bd-f571ca44f8b6 service nova] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Updated VIF entry in instance network info cache for port 00868cee-ead4-4f80-a2c7-1ec86fd0d78f. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:34:37 user nova-compute[72351]: DEBUG nova.network.neutron [req-4746a39c-926d-4a70-882d-adcb2b4b9b37 req-e9c3c265-472d-447a-a0bd-f571ca44f8b6 service nova] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Updating instance_info_cache with network_info: [{"id": "00868cee-ead4-4f80-a2c7-1ec86fd0d78f", "address": "fa:16:3e:20:ad:e5", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.113", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap00868cee-ea", "ovs_interfaceid": "00868cee-ead4-4f80-a2c7-1ec86fd0d78f", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:34:37 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-4746a39c-926d-4a70-882d-adcb2b4b9b37 req-e9c3c265-472d-447a-a0bd-f571ca44f8b6 service nova] Releasing lock "refresh_cache-8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:34:37 user nova-compute[72351]: DEBUG nova.compute.manager [req-697ded0a-f84e-4a57-a173-966173abaa14 req-81ed1269-7ab5-403f-82e1-f77444ea4dc4 service nova] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Received event network-changed-00868cee-ead4-4f80-a2c7-1ec86fd0d78f {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:34:37 user nova-compute[72351]: DEBUG nova.compute.manager [req-697ded0a-f84e-4a57-a173-966173abaa14 req-81ed1269-7ab5-403f-82e1-f77444ea4dc4 service nova] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Refreshing instance network info cache due to event network-changed-00868cee-ead4-4f80-a2c7-1ec86fd0d78f. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:34:37 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-697ded0a-f84e-4a57-a173-966173abaa14 req-81ed1269-7ab5-403f-82e1-f77444ea4dc4 service nova] Acquiring lock "refresh_cache-8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:34:37 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-697ded0a-f84e-4a57-a173-966173abaa14 req-81ed1269-7ab5-403f-82e1-f77444ea4dc4 service nova] Acquired lock "refresh_cache-8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:34:37 user nova-compute[72351]: DEBUG nova.network.neutron [req-697ded0a-f84e-4a57-a173-966173abaa14 req-81ed1269-7ab5-403f-82e1-f77444ea4dc4 service nova] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Refreshing network info cache for port 00868cee-ead4-4f80-a2c7-1ec86fd0d78f {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:34:37 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-3b481f2e-cc04-441a-845d-b56ae42bf857 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquiring lock "8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:34:37 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-3b481f2e-cc04-441a-845d-b56ae42bf857 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:34:37 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-3b481f2e-cc04-441a-845d-b56ae42bf857 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquiring lock "8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:34:37 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-3b481f2e-cc04-441a-845d-b56ae42bf857 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:34:37 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-3b481f2e-cc04-441a-845d-b56ae42bf857 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:34:37 user nova-compute[72351]: INFO nova.compute.manager [None req-3b481f2e-cc04-441a-845d-b56ae42bf857 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Terminating instance
Aug 09 02:34:37 user nova-compute[72351]: DEBUG nova.compute.manager [None req-3b481f2e-cc04-441a-845d-b56ae42bf857 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Start destroying the instance on the hypervisor. {{(pid=72351) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3124}}
Aug 09 02:34:38 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:34:38 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:34:38 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:34:38 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:34:38 user nova-compute[72351]: DEBUG nova.compute.manager [req-48bc48e4-8955-4cdb-8f69-68ca1d342ef9 req-0de2e5d6-bf71-4430-913a-2dbb9d80a685 service nova] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Received event network-vif-unplugged-00868cee-ead4-4f80-a2c7-1ec86fd0d78f {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:34:38 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-48bc48e4-8955-4cdb-8f69-68ca1d342ef9 req-0de2e5d6-bf71-4430-913a-2dbb9d80a685 service nova] Acquiring lock "8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:34:38 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-48bc48e4-8955-4cdb-8f69-68ca1d342ef9 req-0de2e5d6-bf71-4430-913a-2dbb9d80a685 service nova] Lock "8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:34:38 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-48bc48e4-8955-4cdb-8f69-68ca1d342ef9 req-0de2e5d6-bf71-4430-913a-2dbb9d80a685 service nova] Lock "8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:34:38 user nova-compute[72351]: DEBUG nova.compute.manager [req-48bc48e4-8955-4cdb-8f69-68ca1d342ef9 req-0de2e5d6-bf71-4430-913a-2dbb9d80a685 service nova] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] No waiting events found dispatching network-vif-unplugged-00868cee-ead4-4f80-a2c7-1ec86fd0d78f {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:34:38 user nova-compute[72351]: DEBUG nova.compute.manager [req-48bc48e4-8955-4cdb-8f69-68ca1d342ef9 req-0de2e5d6-bf71-4430-913a-2dbb9d80a685 service nova] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Received event network-vif-unplugged-00868cee-ead4-4f80-a2c7-1ec86fd0d78f for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:34:38 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:34:38 user nova-compute[72351]: DEBUG nova.network.neutron [req-697ded0a-f84e-4a57-a173-966173abaa14 req-81ed1269-7ab5-403f-82e1-f77444ea4dc4 service nova] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Updated VIF entry in instance network info cache for port 00868cee-ead4-4f80-a2c7-1ec86fd0d78f. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:34:38 user nova-compute[72351]: DEBUG nova.network.neutron [req-697ded0a-f84e-4a57-a173-966173abaa14 req-81ed1269-7ab5-403f-82e1-f77444ea4dc4 service nova] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Updating instance_info_cache with network_info: [{"id": "00868cee-ead4-4f80-a2c7-1ec86fd0d78f", "address": "fa:16:3e:20:ad:e5", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap00868cee-ea", "ovs_interfaceid": "00868cee-ead4-4f80-a2c7-1ec86fd0d78f", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:34:38 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-697ded0a-f84e-4a57-a173-966173abaa14 req-81ed1269-7ab5-403f-82e1-f77444ea4dc4 service nova] Releasing lock "refresh_cache-8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:34:38 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Instance destroyed successfully.
Aug 09 02:34:38 user nova-compute[72351]: DEBUG nova.objects.instance [None req-3b481f2e-cc04-441a-845d-b56ae42bf857 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lazy-loading 'resources' on Instance uuid 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:34:38 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-3b481f2e-cc04-441a-845d-b56ae42bf857 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:32:44Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-server-2049659607',display_name='tempest-TestVolumeBootPattern-server-2049659607',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testvolumebootpattern-server-2049659607',id=41,image_ref='',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBM+1oHdu/+hmVTCW73FChAt+AvG2bL++YbEuGYMvrUaUYAyZQrt9Vwv7j34dhFmoVuCI0Rk18KgJ1X07X8pPcTOnOW0XFALfY2BbRHpJcmwARDVbYh8uPoNVxqSRoIR2gA==',key_name='tempest-TestVolumeBootPattern-805107831',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:33:00Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='a4c6bbfd8d904ba1ac69ba5da4564513',ramdisk_id='',reservation_id='r-ng07e3t8',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TestVolumeBootPattern-350743460',owner_user_name='tempest-TestVolumeBootPattern-350743460-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:33:01Z,user_data=None,user_id='07d8d5a634274df19c2db769d2d27b29',uuid=8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "00868cee-ead4-4f80-a2c7-1ec86fd0d78f", "address": "fa:16:3e:20:ad:e5", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.113", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap00868cee-ea", "ovs_interfaceid": "00868cee-ead4-4f80-a2c7-1ec86fd0d78f", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:34:38 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-3b481f2e-cc04-441a-845d-b56ae42bf857 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Converting VIF {"id": "00868cee-ead4-4f80-a2c7-1ec86fd0d78f", "address": "fa:16:3e:20:ad:e5", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.113", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap00868cee-ea", "ovs_interfaceid": "00868cee-ead4-4f80-a2c7-1ec86fd0d78f", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:34:38 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-3b481f2e-cc04-441a-845d-b56ae42bf857 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:20:ad:e5,bridge_name='br-int',has_traffic_filtering=True,id=00868cee-ead4-4f80-a2c7-1ec86fd0d78f,network=Network(5ef428cf-e408-490c-8f60-70934f760f4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap00868cee-ea') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:34:38 user nova-compute[72351]: DEBUG os_vif [None req-3b481f2e-cc04-441a-845d-b56ae42bf857 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:20:ad:e5,bridge_name='br-int',has_traffic_filtering=True,id=00868cee-ead4-4f80-a2c7-1ec86fd0d78f,network=Network(5ef428cf-e408-490c-8f60-70934f760f4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap00868cee-ea') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:34:38 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:34:38 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap00868cee-ea, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:34:38 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:34:38 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:34:38 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:34:38 user nova-compute[72351]: INFO os_vif [None req-3b481f2e-cc04-441a-845d-b56ae42bf857 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:20:ad:e5,bridge_name='br-int',has_traffic_filtering=True,id=00868cee-ead4-4f80-a2c7-1ec86fd0d78f,network=Network(5ef428cf-e408-490c-8f60-70934f760f4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap00868cee-ea')
Aug 09 02:34:38 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-3b481f2e-cc04-441a-845d-b56ae42bf857 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 02:34:38 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-3b481f2e-cc04-441a-845d-b56ae42bf857 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'fHEsehNU', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc'}, None), 'kwargs': True}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:34:38 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-3b481f2e-cc04-441a-845d-b56ae42bf857 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:34:38 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-3b481f2e-cc04-441a-845d-b56ae42bf857 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:34:38 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-3b481f2e-cc04-441a-845d-b56ae42bf857 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:34:38 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-3b481f2e-cc04-441a-845d-b56ae42bf857 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 02:34:38 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:34:38 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:34:38 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[34e7113c-9368-4577-93f2-f49c20fd4fc9]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:34:38 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:34:38 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.029s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:34:38 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[952560c2-7169-4f92-9394-f793aecdeb25]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)\ntcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc (non-flash)\ntcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)\ntcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:34:38 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-3b481f2e-cc04-441a-845d-b56ae42bf857 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:34:38 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:34:38 user nova-compute[72351]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)
Aug 09 02:34:38 user nova-compute[72351]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc (non-flash)
Aug 09 02:34:38 user nova-compute[72351]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:34:38 user nova-compute[72351]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)
Aug 09 02:34:38 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:34:38 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-3b481f2e-cc04-441a-845d-b56ae42bf857 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:34:38 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:34:38 user nova-compute[72351]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)
Aug 09 02:34:38 user nova-compute[72351]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc (non-flash)
Aug 09 02:34:38 user nova-compute[72351]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:34:38 user nova-compute[72351]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)
Aug 09 02:34:38 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:34:38 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-3b481f2e-cc04-441a-845d-b56ae42bf857 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fcfbc7d5bd0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc'): ({'sdc'}, set())}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 02:34:38 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-3b481f2e-cc04-441a-845d-b56ae42bf857 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Removing single pathed devices sdc {{(pid=72351) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Aug 09 02:34:38 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:34:38 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.027s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:34:38 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[cf0b86cb-84d0-4574-b833-04af5cbdfbee]: (4, ('path checker states:\nup                  6\n\npaths: 0\nbusy: False\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:34:38 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdc {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:34:38 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdc" returned: 0 in 0.025s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:34:38 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e25be7af-96cd-4e5d-8df3-ceab861a95aa]: (4, ('ok\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:34:38 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-3b481f2e-cc04-441a-845d-b56ae42bf857 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Flushing IO for device /dev/sdc {{(pid=72351) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Aug 09 02:34:38 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdc {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:34:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdc" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:34:39 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e37fb88d-3f2d-467b-acb3-7cee71bf4820]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:34:39 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-3b481f2e-cc04-441a-845d-b56ae42bf857 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Remove SCSI device /dev/sdc with /sys/block/sdc/device/delete {{(pid=72351) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Aug 09 02:34:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdc/device/delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:34:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdc/device/delete" returned: 0 in 0.049s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:34:39 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[65d83663-ca5c-4609-9ab2-d693374fc911]: (4, ('1', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:34:39 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-3b481f2e-cc04-441a-845d-b56ae42bf857 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Checking to see if SCSI volumes sdc have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Aug 09 02:34:39 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-3b481f2e-cc04-441a-845d-b56ae42bf857 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] SCSI volumes sdc have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Aug 09 02:34:39 user nova-compute[72351]: DEBUG os_brick.privileged.rootwrap [-] Unlinking ('/dev/disk/by-id/scsi-1SCST_BIOc88193fd-c6743eb939c82122', '/dev/disk/by-id/scsi-SSCST_BIO_Storage_c88193fd', '/dev/disk/by-id/scsi-26336373433656239') {{(pid=83366) unlink_root /usr/local/lib/python3.10/dist-packages/os_brick/privileged/rootwrap.py:219}}
Aug 09 02:34:39 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[9f4d7d23-a79e-4c40-a45d-549d22160124]: (4, None) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:34:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-3b481f2e-cc04-441a-845d-b56ae42bf857 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc')] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 02:34:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:34:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.027s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:34:39 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d621d277-b1c4-4c29-9584-851dde3ed248]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:34:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-3b481f2e-cc04-441a-845d-b56ae42bf857 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:34:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc -p 172.16.0.220:3260 --logout {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:34:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc -p 172.16.0.220:3260 --logout" returned: 0 in 0.061s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:34:39 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[baf4f154-e0c3-4add-bd5e-c2ce20281c24]: (4, ('Logging out of session [sid: 41, target: iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc, portal: 172.16.0.220,3260]\nLogout of [sid: 41, target: iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:34:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-3b481f2e-cc04-441a-845d-b56ae42bf857 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 41, target: iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc, portal: 172.16.0.220,3260]
Aug 09 02:34:39 user nova-compute[72351]: Logout of [sid: 41, target: iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc, portal: 172.16.0.220,3260] successful.
Aug 09 02:34:39 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:34:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc -p 172.16.0.220:3260 --op delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:34:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bc935627-c165-4eca-b63b-394ec763b6fc -p 172.16.0.220:3260 --op delete" returned: 0 in 0.031s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:34:39 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[601fde57-f5a2-4be0-880c-10936f4361ea]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:34:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-3b481f2e-cc04-441a-845d-b56ae42bf857 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:34:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-3b481f2e-cc04-441a-845d-b56ae42bf857 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.349s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:34:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-3b481f2e-cc04-441a-845d-b56ae42bf857 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] <== disconnect_volume: return (349ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:34:39 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-3b481f2e-cc04-441a-845d-b56ae42bf857 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 02:34:39 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-3b481f2e-cc04-441a-845d-b56ae42bf857 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Deleting instance files /opt/stack/data/nova/instances/8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb_del
Aug 09 02:34:39 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-3b481f2e-cc04-441a-845d-b56ae42bf857 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Deletion of /opt/stack/data/nova/instances/8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb_del complete
Aug 09 02:34:39 user nova-compute[72351]: INFO nova.compute.manager [None req-3b481f2e-cc04-441a-845d-b56ae42bf857 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Took 1.29 seconds to destroy the instance on the hypervisor.
Aug 09 02:34:39 user nova-compute[72351]: DEBUG oslo.service.loopingcall [None req-3b481f2e-cc04-441a-845d-b56ae42bf857 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72351) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Aug 09 02:34:39 user nova-compute[72351]: DEBUG nova.compute.manager [-] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Deallocating network for instance {{(pid=72351) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2263}}
Aug 09 02:34:39 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] deallocate_for_instance() {{(pid=72351) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1795}}
Aug 09 02:34:39 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:34:39 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Took 0.62 seconds to deallocate network for instance.
Aug 09 02:34:40 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691548465.3982239, 47afce95-e5f7-4703-a99f-7ef8b28c4bd2 => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:34:40 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] VM Stopped (Lifecycle Event)
Aug 09 02:34:40 user nova-compute[72351]: DEBUG nova.compute.manager [None req-e39bcede-0c95-4c02-8e47-b9ce4d198721 None None] [instance: 47afce95-e5f7-4703-a99f-7ef8b28c4bd2] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:34:40 user nova-compute[72351]: DEBUG nova.compute.manager [req-31d916e4-ad96-455f-9ec7-cf0b275c4652 req-b424ac4e-a629-4ad5-accd-9624013e0121 service nova] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Received event network-vif-plugged-00868cee-ead4-4f80-a2c7-1ec86fd0d78f {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:34:40 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-31d916e4-ad96-455f-9ec7-cf0b275c4652 req-b424ac4e-a629-4ad5-accd-9624013e0121 service nova] Acquiring lock "8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:34:40 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-31d916e4-ad96-455f-9ec7-cf0b275c4652 req-b424ac4e-a629-4ad5-accd-9624013e0121 service nova] Lock "8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:34:40 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-31d916e4-ad96-455f-9ec7-cf0b275c4652 req-b424ac4e-a629-4ad5-accd-9624013e0121 service nova] Lock "8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:34:40 user nova-compute[72351]: DEBUG nova.compute.manager [req-31d916e4-ad96-455f-9ec7-cf0b275c4652 req-b424ac4e-a629-4ad5-accd-9624013e0121 service nova] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] No waiting events found dispatching network-vif-plugged-00868cee-ead4-4f80-a2c7-1ec86fd0d78f {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:34:40 user nova-compute[72351]: WARNING nova.compute.manager [req-31d916e4-ad96-455f-9ec7-cf0b275c4652 req-b424ac4e-a629-4ad5-accd-9624013e0121 service nova] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Received unexpected event network-vif-plugged-00868cee-ead4-4f80-a2c7-1ec86fd0d78f for instance with vm_state active and task_state deleting.
Aug 09 02:34:40 user nova-compute[72351]: DEBUG nova.compute.manager [req-31d916e4-ad96-455f-9ec7-cf0b275c4652 req-b424ac4e-a629-4ad5-accd-9624013e0121 service nova] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Received event network-vif-deleted-00868cee-ead4-4f80-a2c7-1ec86fd0d78f {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:34:40 user nova-compute[72351]: INFO nova.compute.manager [None req-3b481f2e-cc04-441a-845d-b56ae42bf857 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Took 0.75 seconds to detach 1 volumes for instance.
Aug 09 02:34:40 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-3b481f2e-cc04-441a-845d-b56ae42bf857 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:34:40 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-3b481f2e-cc04-441a-845d-b56ae42bf857 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:34:41 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-3b481f2e-cc04-441a-845d-b56ae42bf857 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:34:41 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-3b481f2e-cc04-441a-845d-b56ae42bf857 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:34:41 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-3b481f2e-cc04-441a-845d-b56ae42bf857 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.395s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:34:41 user nova-compute[72351]: INFO nova.scheduler.client.report [None req-3b481f2e-cc04-441a-845d-b56ae42bf857 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Deleted allocations for instance 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb
Aug 09 02:34:41 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-3b481f2e-cc04-441a-845d-b56ae42bf857 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 3.239s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:34:43 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:34:43 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:34:48 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:34:48 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:34:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Acquiring lock "8e970bdd-54a1-4058-b09d-784cb6b5b1a4" by "nova.compute.manager.ComputeManager.unshelve_instance.<locals>.do_unshelve_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:34:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Lock "8e970bdd-54a1-4058-b09d-784cb6b5b1a4" acquired by "nova.compute.manager.ComputeManager.unshelve_instance.<locals>.do_unshelve_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:34:52 user nova-compute[72351]: INFO nova.compute.manager [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Unshelving
Aug 09 02:34:53 user nova-compute[72351]: INFO nova.virt.block_device [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Booting with volume bc173cc3-1a94-467d-8d0b-d0f50936cee3 at /dev/vda
Aug 09 02:34:53 user nova-compute[72351]: DEBUG os_brick.utils [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:34:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:34:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.016s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:34:53 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[f50923f5-732a-4ce3-88d5-dabfb70f9834]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:a52269602ff4\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:34:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:34:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.015s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:34:53 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d18fa768-9052-4c07-a5fa-c24926a14fcb]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:34:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:34:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:34:53 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[9d97ee90-9b43-4597-a0b5-a3371ac9e0d4]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:34:53 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[bfad8198-3944-4bde-af70-722214697607]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:34:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Running cmd (subprocess): nvme version {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:34:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] 'nvme version' failed. Not Retrying. {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:34:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.nvmeof [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] nvme not present on system {{(pid=72351) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Aug 09 02:34:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:34:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:34:53 user nova-compute[72351]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Aug 09 02:34:53 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[c1e17bb6-55cf-4c29-92ce-fd5fe2a70a81]: (4, '') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:34:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72351) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Aug 09 02:34:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Aug 09 02:34:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] LIGHTOS: no hostnqn found. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Aug 09 02:34:53 user nova-compute[72351]: DEBUG os_brick.utils [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] <== get_connector_properties: return (128ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:a52269602ff4', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:34:53 user nova-compute[72351]: DEBUG nova.virt.block_device [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Updating existing volume attachment record: a153defe-4185-43c8-9f4e-50f86f960ffe {{(pid=72351) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Aug 09 02:34:53 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691548478.7968414, 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:34:53 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] VM Stopped (Lifecycle Event)
Aug 09 02:34:53 user nova-compute[72351]: DEBUG nova.compute.manager [None req-f284a795-820a-46c9-995e-4d51dfcdc23a None None] [instance: 8650d06e-7063-4c5a-9b0f-fdbe8a1cdbcb] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:34:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:34:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:34:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 02:34:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:34:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:34:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:34:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Acquiring lock "bde1ef1e-163c-40eb-abb5-962b9064b7ab" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:34:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "bde1ef1e-163c-40eb-abb5-962b9064b7ab" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:34:57 user nova-compute[72351]: DEBUG nova.compute.manager [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Starting instance... {{(pid=72351) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2406}}
Aug 09 02:34:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-09bd7bab-74c7-45d1-931f-26eb0115e946 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Acquiring lock "80e87454-90b9-4168-bf00-b7d93b245526" by "nova.compute.manager.ComputeManager.confirm_resize.<locals>.do_confirm_resize" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:34:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-09bd7bab-74c7-45d1-931f-26eb0115e946 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "80e87454-90b9-4168-bf00-b7d93b245526" acquired by "nova.compute.manager.ComputeManager.confirm_resize.<locals>.do_confirm_resize" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:34:57 user nova-compute[72351]: DEBUG nova.compute.manager [None req-09bd7bab-74c7-45d1-931f-26eb0115e946 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Going to confirm migration 2 {{(pid=72351) do_confirm_resize /opt/stack/nova/nova/compute/manager.py:4694}}
Aug 09 02:34:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:34:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:34:57 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72351) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Aug 09 02:34:57 user nova-compute[72351]: INFO nova.compute.claims [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Claim successful on node user
Aug 09 02:34:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-09bd7bab-74c7-45d1-931f-26eb0115e946 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Acquiring lock "refresh_cache-80e87454-90b9-4168-bf00-b7d93b245526" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:34:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-09bd7bab-74c7-45d1-931f-26eb0115e946 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Acquired lock "refresh_cache-80e87454-90b9-4168-bf00-b7d93b245526" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:34:57 user nova-compute[72351]: DEBUG nova.network.neutron [None req-09bd7bab-74c7-45d1-931f-26eb0115e946 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:34:57 user nova-compute[72351]: DEBUG nova.objects.instance [None req-09bd7bab-74c7-45d1-931f-26eb0115e946 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lazy-loading 'info_cache' on Instance uuid 80e87454-90b9-4168-bf00-b7d93b245526 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:34:57 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:34:57 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:34:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.584s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:34:57 user nova-compute[72351]: DEBUG nova.compute.manager [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Start building networks asynchronously for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2803}}
Aug 09 02:34:57 user nova-compute[72351]: DEBUG nova.compute.manager [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Allocating IP information in the background. {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1956}}
Aug 09 02:34:57 user nova-compute[72351]: DEBUG nova.network.neutron [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] allocate_for_instance() {{(pid=72351) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Aug 09 02:34:57 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Aug 09 02:34:57 user nova-compute[72351]: DEBUG nova.compute.manager [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Start building block device mappings for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2838}}
Aug 09 02:34:57 user nova-compute[72351]: DEBUG nova.network.neutron [None req-09bd7bab-74c7-45d1-931f-26eb0115e946 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Updating instance_info_cache with network_info: [{"id": "fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4", "address": "fa:16:3e:7a:77:69", "network": {"id": "b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-210689817-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.41", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "69d584c230b94ff8aae722eb2f364c5c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfcb0ceaf-ff", "ovs_interfaceid": "fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:34:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-09bd7bab-74c7-45d1-931f-26eb0115e946 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Releasing lock "refresh_cache-80e87454-90b9-4168-bf00-b7d93b245526" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:34:57 user nova-compute[72351]: DEBUG nova.objects.instance [None req-09bd7bab-74c7-45d1-931f-26eb0115e946 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lazy-loading 'migration_context' on Instance uuid 80e87454-90b9-4168-bf00-b7d93b245526 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:34:57 user nova-compute[72351]: INFO nova.virt.block_device [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Booting with volume-backed-image 18ccda7f-31d3-4978-ab9d-4ed2f74eff94 at /dev/vda
Aug 09 02:34:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-09bd7bab-74c7-45d1-931f-26eb0115e946 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.drop_move_claim_at_source" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:34:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-09bd7bab-74c7-45d1-931f-26eb0115e946 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.drop_move_claim_at_source" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:34:57 user nova-compute[72351]: DEBUG nova.policy [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '73d05c48be5640c28455e8ba8b05ef31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '321111f935dd4e8aab4a083206ea9a7e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72351) authorize /opt/stack/nova/nova/policy.py:203}}
Aug 09 02:34:58 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-09bd7bab-74c7-45d1-931f-26eb0115e946 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:34:58 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-09bd7bab-74c7-45d1-931f-26eb0115e946 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:34:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-09bd7bab-74c7-45d1-931f-26eb0115e946 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.drop_move_claim_at_source" :: held 0.741s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:34:58 user nova-compute[72351]: INFO nova.scheduler.client.report [None req-09bd7bab-74c7-45d1-931f-26eb0115e946 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Deleted allocation for migration 5464193d-df16-4ccd-b137-4df0859f7495
Aug 09 02:34:58 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:34:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-09bd7bab-74c7-45d1-931f-26eb0115e946 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "80e87454-90b9-4168-bf00-b7d93b245526" "released" by "nova.compute.manager.ComputeManager.confirm_resize.<locals>.do_confirm_resize" :: held 1.896s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:34:59 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:34:59 user nova-compute[72351]: DEBUG nova.network.neutron [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Successfully created port: 8424518d-87b7-4fc4-8a58-1b4f094a5ca0 {{(pid=72351) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Aug 09 02:34:59 user nova-compute[72351]: DEBUG nova.compute.manager [req-6ef343af-bdbf-42d8-8092-8575bb2c03b1 req-e68520a8-2c3b-45e7-8748-99606d422592 service nova] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Received event network-changed-e58740da-bc24-4309-8e35-48fc70bece21 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:34:59 user nova-compute[72351]: DEBUG nova.compute.manager [req-6ef343af-bdbf-42d8-8092-8575bb2c03b1 req-e68520a8-2c3b-45e7-8748-99606d422592 service nova] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Refreshing instance network info cache due to event network-changed-e58740da-bc24-4309-8e35-48fc70bece21. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:34:59 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-6ef343af-bdbf-42d8-8092-8575bb2c03b1 req-e68520a8-2c3b-45e7-8748-99606d422592 service nova] Acquiring lock "refresh_cache-a03fbded-23ad-440b-97e7-bea07425e873" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:34:59 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-6ef343af-bdbf-42d8-8092-8575bb2c03b1 req-e68520a8-2c3b-45e7-8748-99606d422592 service nova] Acquired lock "refresh_cache-a03fbded-23ad-440b-97e7-bea07425e873" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:34:59 user nova-compute[72351]: DEBUG nova.network.neutron [req-6ef343af-bdbf-42d8-8092-8575bb2c03b1 req-e68520a8-2c3b-45e7-8748-99606d422592 service nova] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Refreshing network info cache for port e58740da-bc24-4309-8e35-48fc70bece21 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:34:59 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-20a02ef8-c4bd-4cdd-ae48-7a28855685d0 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Acquiring lock "a03fbded-23ad-440b-97e7-bea07425e873" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:34:59 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-20a02ef8-c4bd-4cdd-ae48-7a28855685d0 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lock "a03fbded-23ad-440b-97e7-bea07425e873" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:34:59 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-20a02ef8-c4bd-4cdd-ae48-7a28855685d0 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Acquiring lock "a03fbded-23ad-440b-97e7-bea07425e873-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:34:59 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-20a02ef8-c4bd-4cdd-ae48-7a28855685d0 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lock "a03fbded-23ad-440b-97e7-bea07425e873-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:34:59 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-20a02ef8-c4bd-4cdd-ae48-7a28855685d0 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lock "a03fbded-23ad-440b-97e7-bea07425e873-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:34:59 user nova-compute[72351]: INFO nova.compute.manager [None req-20a02ef8-c4bd-4cdd-ae48-7a28855685d0 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Terminating instance
Aug 09 02:34:59 user nova-compute[72351]: DEBUG nova.compute.manager [None req-20a02ef8-c4bd-4cdd-ae48-7a28855685d0 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Start destroying the instance on the hypervisor. {{(pid=72351) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3124}}
Aug 09 02:34:59 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:34:59 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:34:59 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:00 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:00 user nova-compute[72351]: DEBUG nova.network.neutron [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Successfully updated port: 8424518d-87b7-4fc4-8a58-1b4f094a5ca0 {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:35:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Acquiring lock "refresh_cache-bde1ef1e-163c-40eb-abb5-962b9064b7ab" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:35:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Acquired lock "refresh_cache-bde1ef1e-163c-40eb-abb5-962b9064b7ab" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:35:00 user nova-compute[72351]: DEBUG nova.network.neutron [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:35:00 user nova-compute[72351]: DEBUG nova.compute.manager [req-76b58108-7bf0-47a8-8feb-120a8917b44f req-56e33520-510a-4395-8345-ebcfc0183f85 service nova] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Received event network-changed-8424518d-87b7-4fc4-8a58-1b4f094a5ca0 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:35:00 user nova-compute[72351]: DEBUG nova.compute.manager [req-76b58108-7bf0-47a8-8feb-120a8917b44f req-56e33520-510a-4395-8345-ebcfc0183f85 service nova] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Refreshing instance network info cache due to event network-changed-8424518d-87b7-4fc4-8a58-1b4f094a5ca0. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:35:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-76b58108-7bf0-47a8-8feb-120a8917b44f req-56e33520-510a-4395-8345-ebcfc0183f85 service nova] Acquiring lock "refresh_cache-bde1ef1e-163c-40eb-abb5-962b9064b7ab" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:35:00 user nova-compute[72351]: DEBUG nova.network.neutron [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:35:00 user nova-compute[72351]: DEBUG nova.network.neutron [req-6ef343af-bdbf-42d8-8092-8575bb2c03b1 req-e68520a8-2c3b-45e7-8748-99606d422592 service nova] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Updated VIF entry in instance network info cache for port e58740da-bc24-4309-8e35-48fc70bece21. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:35:00 user nova-compute[72351]: DEBUG nova.network.neutron [req-6ef343af-bdbf-42d8-8092-8575bb2c03b1 req-e68520a8-2c3b-45e7-8748-99606d422592 service nova] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Updating instance_info_cache with network_info: [{"id": "e58740da-bc24-4309-8e35-48fc70bece21", "address": "fa:16:3e:f3:4e:5e", "network": {"id": "49bfc70e-93c7-4607-9507-4711bda1a686", "bridge": "br-int", "label": "tempest-TestStampPattern-1656229455-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fc12f9037cd54bb6a2fdff3071132dd4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape58740da-bc", "ovs_interfaceid": "e58740da-bc24-4309-8e35-48fc70bece21", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:35:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-6ef343af-bdbf-42d8-8092-8575bb2c03b1 req-e68520a8-2c3b-45e7-8748-99606d422592 service nova] Releasing lock "refresh_cache-a03fbded-23ad-440b-97e7-bea07425e873" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:35:00 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Instance destroyed successfully.
Aug 09 02:35:00 user nova-compute[72351]: DEBUG nova.objects.instance [None req-20a02ef8-c4bd-4cdd-ae48-7a28855685d0 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lazy-loading 'resources' on Instance uuid a03fbded-23ad-440b-97e7-bea07425e873 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:35:00 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-20a02ef8-c4bd-4cdd-ae48-7a28855685d0 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:27:37Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestStampPattern-server-356325522',display_name='tempest-TestStampPattern-server-356325522',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-teststamppattern-server-356325522',id=30,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBPzyrlB54Gi/8YMv0Cfs/esdmr1xNOdiloZGKR/XYFcg8Nos/w3Zm8ZOseVT8bKKFqeq3DP2KQSbuWg07AAvODzdTY0X+5ZdNLlP+cvLLuP4ef0UvshCJkUtWTANxgqmUw==',key_name='tempest-TestStampPattern-2061411239',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:27:44Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='fc12f9037cd54bb6a2fdff3071132dd4',ramdisk_id='',reservation_id='r-9r7jqe2t',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TestStampPattern-1244733964',owner_user_name='tempest-TestStampPattern-1244733964-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:30:49Z,user_data=None,user_id='a103ebb9b63f4a259163ec25a33a0dcc',uuid=a03fbded-23ad-440b-97e7-bea07425e873,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "e58740da-bc24-4309-8e35-48fc70bece21", "address": "fa:16:3e:f3:4e:5e", "network": {"id": "49bfc70e-93c7-4607-9507-4711bda1a686", "bridge": "br-int", "label": "tempest-TestStampPattern-1656229455-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.74", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fc12f9037cd54bb6a2fdff3071132dd4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape58740da-bc", "ovs_interfaceid": "e58740da-bc24-4309-8e35-48fc70bece21", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:35:00 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-20a02ef8-c4bd-4cdd-ae48-7a28855685d0 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Converting VIF {"id": "e58740da-bc24-4309-8e35-48fc70bece21", "address": "fa:16:3e:f3:4e:5e", "network": {"id": "49bfc70e-93c7-4607-9507-4711bda1a686", "bridge": "br-int", "label": "tempest-TestStampPattern-1656229455-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.74", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "fc12f9037cd54bb6a2fdff3071132dd4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape58740da-bc", "ovs_interfaceid": "e58740da-bc24-4309-8e35-48fc70bece21", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:35:00 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-20a02ef8-c4bd-4cdd-ae48-7a28855685d0 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:f3:4e:5e,bridge_name='br-int',has_traffic_filtering=True,id=e58740da-bc24-4309-8e35-48fc70bece21,network=Network(49bfc70e-93c7-4607-9507-4711bda1a686),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape58740da-bc') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:35:00 user nova-compute[72351]: DEBUG os_vif [None req-20a02ef8-c4bd-4cdd-ae48-7a28855685d0 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:f3:4e:5e,bridge_name='br-int',has_traffic_filtering=True,id=e58740da-bc24-4309-8e35-48fc70bece21,network=Network(49bfc70e-93c7-4607-9507-4711bda1a686),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape58740da-bc') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:35:00 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:00 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tape58740da-bc, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:35:00 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:00 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:35:00 user nova-compute[72351]: INFO os_vif [None req-20a02ef8-c4bd-4cdd-ae48-7a28855685d0 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:f3:4e:5e,bridge_name='br-int',has_traffic_filtering=True,id=e58740da-bc24-4309-8e35-48fc70bece21,network=Network(49bfc70e-93c7-4607-9507-4711bda1a686),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape58740da-bc')
Aug 09 02:35:00 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-20a02ef8-c4bd-4cdd-ae48-7a28855685d0 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Deleting instance files /opt/stack/data/nova/instances/a03fbded-23ad-440b-97e7-bea07425e873_del
Aug 09 02:35:00 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-20a02ef8-c4bd-4cdd-ae48-7a28855685d0 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Deletion of /opt/stack/data/nova/instances/a03fbded-23ad-440b-97e7-bea07425e873_del complete
Aug 09 02:35:00 user nova-compute[72351]: DEBUG nova.network.neutron [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Updating instance_info_cache with network_info: [{"id": "8424518d-87b7-4fc4-8a58-1b4f094a5ca0", "address": "fa:16:3e:08:61:83", "network": {"id": "9a452414-d61f-4045-a6cc-9be93c892c4c", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "321111f935dd4e8aab4a083206ea9a7e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8424518d-87", "ovs_interfaceid": "8424518d-87b7-4fc4-8a58-1b4f094a5ca0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:35:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Releasing lock "refresh_cache-bde1ef1e-163c-40eb-abb5-962b9064b7ab" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:35:00 user nova-compute[72351]: DEBUG nova.compute.manager [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Instance network_info: |[{"id": "8424518d-87b7-4fc4-8a58-1b4f094a5ca0", "address": "fa:16:3e:08:61:83", "network": {"id": "9a452414-d61f-4045-a6cc-9be93c892c4c", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "321111f935dd4e8aab4a083206ea9a7e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8424518d-87", "ovs_interfaceid": "8424518d-87b7-4fc4-8a58-1b4f094a5ca0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1971}}
Aug 09 02:35:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-76b58108-7bf0-47a8-8feb-120a8917b44f req-56e33520-510a-4395-8345-ebcfc0183f85 service nova] Acquired lock "refresh_cache-bde1ef1e-163c-40eb-abb5-962b9064b7ab" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:35:00 user nova-compute[72351]: DEBUG nova.network.neutron [req-76b58108-7bf0-47a8-8feb-120a8917b44f req-56e33520-510a-4395-8345-ebcfc0183f85 service nova] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Refreshing network info cache for port 8424518d-87b7-4fc4-8a58-1b4f094a5ca0 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:35:00 user nova-compute[72351]: INFO nova.compute.manager [None req-20a02ef8-c4bd-4cdd-ae48-7a28855685d0 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Took 0.90 seconds to destroy the instance on the hypervisor.
Aug 09 02:35:00 user nova-compute[72351]: DEBUG oslo.service.loopingcall [None req-20a02ef8-c4bd-4cdd-ae48-7a28855685d0 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72351) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Aug 09 02:35:00 user nova-compute[72351]: DEBUG nova.compute.manager [-] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Deallocating network for instance {{(pid=72351) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2263}}
Aug 09 02:35:00 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: a03fbded-23ad-440b-97e7-bea07425e873] deallocate_for_instance() {{(pid=72351) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1795}}
Aug 09 02:35:01 user nova-compute[72351]: DEBUG nova.network.neutron [req-76b58108-7bf0-47a8-8feb-120a8917b44f req-56e33520-510a-4395-8345-ebcfc0183f85 service nova] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Updated VIF entry in instance network info cache for port 8424518d-87b7-4fc4-8a58-1b4f094a5ca0. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:35:01 user nova-compute[72351]: DEBUG nova.network.neutron [req-76b58108-7bf0-47a8-8feb-120a8917b44f req-56e33520-510a-4395-8345-ebcfc0183f85 service nova] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Updating instance_info_cache with network_info: [{"id": "8424518d-87b7-4fc4-8a58-1b4f094a5ca0", "address": "fa:16:3e:08:61:83", "network": {"id": "9a452414-d61f-4045-a6cc-9be93c892c4c", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "321111f935dd4e8aab4a083206ea9a7e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8424518d-87", "ovs_interfaceid": "8424518d-87b7-4fc4-8a58-1b4f094a5ca0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:35:01 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-76b58108-7bf0-47a8-8feb-120a8917b44f req-56e33520-510a-4395-8345-ebcfc0183f85 service nova] Releasing lock "refresh_cache-bde1ef1e-163c-40eb-abb5-962b9064b7ab" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:35:01 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:35:01 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Took 0.63 seconds to deallocate network for instance.
Aug 09 02:35:01 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-20a02ef8-c4bd-4cdd-ae48-7a28855685d0 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:35:01 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-20a02ef8-c4bd-4cdd-ae48-7a28855685d0 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:35:01 user nova-compute[72351]: DEBUG nova.compute.manager [req-83a74ca6-b160-449a-bebb-5dbdfe3f5643 req-4d9462ff-d5cf-484c-8810-9dcbfdf71ddf service nova] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Received event network-vif-unplugged-e58740da-bc24-4309-8e35-48fc70bece21 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:35:01 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-83a74ca6-b160-449a-bebb-5dbdfe3f5643 req-4d9462ff-d5cf-484c-8810-9dcbfdf71ddf service nova] Acquiring lock "a03fbded-23ad-440b-97e7-bea07425e873-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:35:01 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-83a74ca6-b160-449a-bebb-5dbdfe3f5643 req-4d9462ff-d5cf-484c-8810-9dcbfdf71ddf service nova] Lock "a03fbded-23ad-440b-97e7-bea07425e873-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:35:01 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-83a74ca6-b160-449a-bebb-5dbdfe3f5643 req-4d9462ff-d5cf-484c-8810-9dcbfdf71ddf service nova] Lock "a03fbded-23ad-440b-97e7-bea07425e873-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:35:01 user nova-compute[72351]: DEBUG nova.compute.manager [req-83a74ca6-b160-449a-bebb-5dbdfe3f5643 req-4d9462ff-d5cf-484c-8810-9dcbfdf71ddf service nova] [instance: a03fbded-23ad-440b-97e7-bea07425e873] No waiting events found dispatching network-vif-unplugged-e58740da-bc24-4309-8e35-48fc70bece21 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:35:01 user nova-compute[72351]: WARNING nova.compute.manager [req-83a74ca6-b160-449a-bebb-5dbdfe3f5643 req-4d9462ff-d5cf-484c-8810-9dcbfdf71ddf service nova] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Received unexpected event network-vif-unplugged-e58740da-bc24-4309-8e35-48fc70bece21 for instance with vm_state deleted and task_state None.
Aug 09 02:35:01 user nova-compute[72351]: DEBUG nova.compute.manager [req-83a74ca6-b160-449a-bebb-5dbdfe3f5643 req-4d9462ff-d5cf-484c-8810-9dcbfdf71ddf service nova] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Received event network-vif-plugged-e58740da-bc24-4309-8e35-48fc70bece21 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:35:01 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-83a74ca6-b160-449a-bebb-5dbdfe3f5643 req-4d9462ff-d5cf-484c-8810-9dcbfdf71ddf service nova] Acquiring lock "a03fbded-23ad-440b-97e7-bea07425e873-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:35:01 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-83a74ca6-b160-449a-bebb-5dbdfe3f5643 req-4d9462ff-d5cf-484c-8810-9dcbfdf71ddf service nova] Lock "a03fbded-23ad-440b-97e7-bea07425e873-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:35:01 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-83a74ca6-b160-449a-bebb-5dbdfe3f5643 req-4d9462ff-d5cf-484c-8810-9dcbfdf71ddf service nova] Lock "a03fbded-23ad-440b-97e7-bea07425e873-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:35:01 user nova-compute[72351]: DEBUG nova.compute.manager [req-83a74ca6-b160-449a-bebb-5dbdfe3f5643 req-4d9462ff-d5cf-484c-8810-9dcbfdf71ddf service nova] [instance: a03fbded-23ad-440b-97e7-bea07425e873] No waiting events found dispatching network-vif-plugged-e58740da-bc24-4309-8e35-48fc70bece21 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:35:01 user nova-compute[72351]: WARNING nova.compute.manager [req-83a74ca6-b160-449a-bebb-5dbdfe3f5643 req-4d9462ff-d5cf-484c-8810-9dcbfdf71ddf service nova] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Received unexpected event network-vif-plugged-e58740da-bc24-4309-8e35-48fc70bece21 for instance with vm_state deleted and task_state None.
Aug 09 02:35:01 user nova-compute[72351]: DEBUG nova.compute.manager [req-83a74ca6-b160-449a-bebb-5dbdfe3f5643 req-4d9462ff-d5cf-484c-8810-9dcbfdf71ddf service nova] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Received event network-vif-deleted-e58740da-bc24-4309-8e35-48fc70bece21 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:35:01 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-20a02ef8-c4bd-4cdd-ae48-7a28855685d0 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:35:01 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-20a02ef8-c4bd-4cdd-ae48-7a28855685d0 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:35:01 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-20a02ef8-c4bd-4cdd-ae48-7a28855685d0 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.445s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:35:01 user nova-compute[72351]: INFO nova.scheduler.client.report [None req-20a02ef8-c4bd-4cdd-ae48-7a28855685d0 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Deleted allocations for instance a03fbded-23ad-440b-97e7-bea07425e873
Aug 09 02:35:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-20a02ef8-c4bd-4cdd-ae48-7a28855685d0 tempest-TestStampPattern-1244733964 tempest-TestStampPattern-1244733964-project-member] Lock "a03fbded-23ad-440b-97e7-bea07425e873" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.158s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:35:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:35:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:35:02 user nova-compute[72351]: DEBUG nova.objects.instance [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Lazy-loading 'pci_requests' on Instance uuid 8e970bdd-54a1-4058-b09d-784cb6b5b1a4 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:35:02 user nova-compute[72351]: DEBUG nova.objects.instance [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Lazy-loading 'numa_topology' on Instance uuid 8e970bdd-54a1-4058-b09d-784cb6b5b1a4 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:35:02 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72351) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Aug 09 02:35:02 user nova-compute[72351]: INFO nova.compute.claims [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Claim successful on node user
Aug 09 02:35:03 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:35:03 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:35:03 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.506s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:35:03 user nova-compute[72351]: INFO nova.network.neutron [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Updating port a83afb1f-511f-406c-b925-eb70b32bed90 with attributes {'binding:host_id': 'user', 'device_owner': 'compute:nova'}
Aug 09 02:35:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-adc3b121-6c45-4913-b281-cb74fc4b9036 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Acquiring lock "7bef74f4-3bb3-4ad0-bff7-34156be31249" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:35:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-adc3b121-6c45-4913-b281-cb74fc4b9036 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "7bef74f4-3bb3-4ad0-bff7-34156be31249" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:35:04 user nova-compute[72351]: DEBUG nova.objects.instance [None req-adc3b121-6c45-4913-b281-cb74fc4b9036 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lazy-loading 'flavor' on Instance uuid 7bef74f4-3bb3-4ad0-bff7-34156be31249 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:35:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-adc3b121-6c45-4913-b281-cb74fc4b9036 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "7bef74f4-3bb3-4ad0-bff7-34156be31249" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.060s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:35:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-adc3b121-6c45-4913-b281-cb74fc4b9036 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Acquiring lock "7bef74f4-3bb3-4ad0-bff7-34156be31249" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:35:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-adc3b121-6c45-4913-b281-cb74fc4b9036 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "7bef74f4-3bb3-4ad0-bff7-34156be31249" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:35:04 user nova-compute[72351]: INFO nova.compute.manager [None req-adc3b121-6c45-4913-b281-cb74fc4b9036 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Attaching volume 387cac3d-74d5-4a21-9997-bf342fecc112 to /dev/vdb
Aug 09 02:35:04 user nova-compute[72351]: DEBUG os_brick.utils [None req-adc3b121-6c45-4913-b281-cb74fc4b9036 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:35:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:04 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[bfa86b79-d333-483e-a4d8-999dd67c6321]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:a52269602ff4\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Acquiring lock "refresh_cache-8e970bdd-54a1-4058-b09d-784cb6b5b1a4" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:35:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Acquired lock "refresh_cache-8e970bdd-54a1-4058-b09d-784cb6b5b1a4" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:35:04 user nova-compute[72351]: DEBUG nova.network.neutron [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:35:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.016s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:04 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[1dd86b85-798f-4127-9f44-bb27f0d2d9c9]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:04 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[422e9d5b-0f4d-46b0-9aca-db35e371a55c]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:04 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[8aa434e3-4576-4cb1-a692-9df65018c614]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-adc3b121-6c45-4913-b281-cb74fc4b9036 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Running cmd (subprocess): nvme version {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-adc3b121-6c45-4913-b281-cb74fc4b9036 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] 'nvme version' failed. Not Retrying. {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:35:04 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.nvmeof [None req-adc3b121-6c45-4913-b281-cb74fc4b9036 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] nvme not present on system {{(pid=72351) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Aug 09 02:35:04 user nova-compute[72351]: DEBUG nova.compute.manager [req-9d6ca6e7-df87-41c2-bca6-e31949816670 req-2791bf49-458a-4ff0-89d6-8acd2583dcb5 service nova] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Received event network-changed-a83afb1f-511f-406c-b925-eb70b32bed90 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:35:04 user nova-compute[72351]: DEBUG nova.compute.manager [req-9d6ca6e7-df87-41c2-bca6-e31949816670 req-2791bf49-458a-4ff0-89d6-8acd2583dcb5 service nova] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Refreshing instance network info cache due to event network-changed-a83afb1f-511f-406c-b925-eb70b32bed90. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:35:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-9d6ca6e7-df87-41c2-bca6-e31949816670 req-2791bf49-458a-4ff0-89d6-8acd2583dcb5 service nova] Acquiring lock "refresh_cache-8e970bdd-54a1-4058-b09d-784cb6b5b1a4" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:35:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:35:04 user nova-compute[72351]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Aug 09 02:35:04 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ffbd3506-47c0-443e-8b62-f85455b82608]: (4, '') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:04 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-adc3b121-6c45-4913-b281-cb74fc4b9036 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72351) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Aug 09 02:35:04 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-adc3b121-6c45-4913-b281-cb74fc4b9036 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Aug 09 02:35:04 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-adc3b121-6c45-4913-b281-cb74fc4b9036 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] LIGHTOS: no hostnqn found. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Aug 09 02:35:04 user nova-compute[72351]: DEBUG os_brick.utils [None req-adc3b121-6c45-4913-b281-cb74fc4b9036 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] <== get_connector_properties: return (135ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:a52269602ff4', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:35:04 user nova-compute[72351]: DEBUG nova.virt.block_device [None req-adc3b121-6c45-4913-b281-cb74fc4b9036 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Updating existing volume attachment record: b8d1670a-30c7-4a9a-b559-aff12fc94778 {{(pid=72351) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Aug 09 02:35:04 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:35:04 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:35:04 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72351) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10461}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG nova.network.neutron [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Updating instance_info_cache with network_info: [{"id": "a83afb1f-511f-406c-b925-eb70b32bed90", "address": "fa:16:3e:16:83:e9", "network": {"id": "0b3a34b6-6d32-4414-9135-22bd2a1dae20", "bridge": "br-int", "label": "tempest-TestShelveInstance-2024538322-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.128", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4d0cc057c220422a9541bcb9fa7f598b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa83afb1f-51", "ovs_interfaceid": "a83afb1f-511f-406c-b925-eb70b32bed90", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Releasing lock "refresh_cache-8e970bdd-54a1-4058-b09d-784cb6b5b1a4" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Creating instance directory {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4717}}
Aug 09 02:35:05 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Creating image(s)
Aug 09 02:35:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Did not create local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4853}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Ensure instance console log exists: /opt/stack/data/nova/instances/8e970bdd-54a1-4058-b09d-784cb6b5b1a4/console.log {{(pid=72351) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4603}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Start _get_guest_xml network_info=[{"id": "a83afb1f-511f-406c-b925-eb70b32bed90", "address": "fa:16:3e:16:83:e9", "network": {"id": "0b3a34b6-6d32-4414-9135-22bd2a1dae20", "bridge": "br-int", "label": "tempest-TestShelveInstance-2024538322-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.128", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4d0cc057c220422a9541bcb9fa7f598b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa83afb1f-51", "ovs_interfaceid": "a83afb1f-511f-406c-b925-eb70b32bed90", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=1,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=<?>,status=<?>,tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'device_type': 'disk', 'guest_format': None, 'disk_bus': 'virtio', 'mount_device': '/dev/vda', 'boot_index': 0, 'attachment_id': 'a153defe-4185-43c8-9f4e-50f86f960ffe', 'delete_on_termination': True, 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqns': ['iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'EalQALYR', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '8e970bdd-54a1-4058-b09d-784cb6b5b1a4', 'attached_at': '', 'detached_at': '', 'volume_id': 'bc173cc3-1a94-467d-8d0b-d0f50936cee3', 'serial': 'bc173cc3-1a94-467d-8d0b-d0f50936cee3'}, 'volume_type': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-9d6ca6e7-df87-41c2-bca6-e31949816670 req-2791bf49-458a-4ff0-89d6-8acd2583dcb5 service nova] Acquired lock "refresh_cache-8e970bdd-54a1-4058-b09d-784cb6b5b1a4" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG nova.network.neutron [req-9d6ca6e7-df87-41c2-bca6-e31949816670 req-2791bf49-458a-4ff0-89d6-8acd2583dcb5 service nova] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Refreshing network info cache for port a83afb1f-511f-406c-b925-eb70b32bed90 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:35:05 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:35:05 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:35:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=1,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=<?>,status=<?>,tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG nova.objects.instance [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Lazy-loading 'vcpu_model' on Instance uuid 8e970bdd-54a1-4058-b09d-784cb6b5b1a4 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'EalQALYR', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:35:05 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:35:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:35:05 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:35:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:35:05 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 02:35:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 -p 172.16.0.220:3260" returned: 21 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[037dbc4f-9e7a-48c3-847d-95d28656915c]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Aug 09 02:35:05 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 -p 172.16.0.220:3260 --interface default --op new {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[f235b895-509c-41ce-a6a9-3e0da97e3f6c]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3] added\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3] added
Aug 09 02:35:05 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[2ad6386b-01d6-4a10-b422-695a223660d3]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[82824443-bc3f-4448-ac8b-350ae1a3997d]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v EalQALYR {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v EalQALYR" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a7575a05-a7ad-4b76-9d87-c7defb94fda1]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'EalQALYR'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[0358addf-3cdf-411d-ac7e-6014a59c504a]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[9f8e0fc0-6731-4783-955b-a884ad76b808]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)\ntcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)\ntcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:35:05 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:35:05 user nova-compute[72351]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)
Aug 09 02:35:05 user nova-compute[72351]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:35:05 user nova-compute[72351]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)
Aug 09 02:35:05 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:35:05 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:35:05 user nova-compute[72351]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)
Aug 09 02:35:05 user nova-compute[72351]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:35:05 user nova-compute[72351]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)
Aug 09 02:35:05 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 -p 172.16.0.220:3260 --login {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 -p 172.16.0.220:3260 --login" returned: 0 in 0.078s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[2631062b-ddd1-4e0e-931e-4fe3023561be]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3, portal: 172.16.0.220,3260]
Aug 09 02:35:05 user nova-compute[72351]: Login to [iface: default, target: iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3, portal: 172.16.0.220,3260] successful.
Aug 09 02:35:05 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.024s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[bcbd4733-7e75-486d-a4cd-1ecfc707510a]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.034s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[0f0db556-980c-40f0-a990-d3ee2cf86463]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)\ntcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)\ntcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)\ntcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:35:05 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:35:05 user nova-compute[72351]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)
Aug 09 02:35:05 user nova-compute[72351]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:35:05 user nova-compute[72351]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)
Aug 09 02:35:05 user nova-compute[72351]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)
Aug 09 02:35:05 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:35:05 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:35:05 user nova-compute[72351]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)
Aug 09 02:35:05 user nova-compute[72351]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:35:05 user nova-compute[72351]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)
Aug 09 02:35:05 user nova-compute[72351]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)
Aug 09 02:35:05 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.318s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] HCTL ('34', '-', '-', 0) found on session 46 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Scanning host 34 c: -, t: -, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ce69bd71-ba60-48d2-bbc1-eca2a66aa007]: (4, ('- - 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Searching for a device in session 46 and hctl ['34', '*', '*', 0] yield: None {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG nova.network.neutron [req-9d6ca6e7-df87-41c2-bca6-e31949816670 req-2791bf49-458a-4ff0-89d6-8acd2583dcb5 service nova] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Updated VIF entry in instance network info cache for port a83afb1f-511f-406c-b925-eb70b32bed90. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG nova.network.neutron [req-9d6ca6e7-df87-41c2-bca6-e31949816670 req-2791bf49-458a-4ff0-89d6-8acd2583dcb5 service nova] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Updating instance_info_cache with network_info: [{"id": "a83afb1f-511f-406c-b925-eb70b32bed90", "address": "fa:16:3e:16:83:e9", "network": {"id": "0b3a34b6-6d32-4414-9135-22bd2a1dae20", "bridge": "br-int", "label": "tempest-TestShelveInstance-2024538322-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.128", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4d0cc057c220422a9541bcb9fa7f598b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa83afb1f-51", "ovs_interfaceid": "a83afb1f-511f-406c-b925-eb70b32bed90", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-9d6ca6e7-df87-41c2-bca6-e31949816670 req-2791bf49-458a-4ff0-89d6-8acd2583dcb5 service nova] Releasing lock "refresh_cache-8e970bdd-54a1-4058-b09d-784cb6b5b1a4" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:35:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:06 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Searching for a device in session 46 and hctl ['34', '*', '*', 0] yield: sdc {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:35:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Connected to sdc using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'EalQALYR', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 02:35:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.349s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:35:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] <== connect_volume: return (1351ms) {'type': 'block', 'scsi_wwn': '26661326636383833', 'path': '/dev/sdc'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:35:06 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26661326636383833', 'path': '/dev/sdc'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 02:35:06 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,compute_id=1,config_drive='',created_at=2023-08-09T02:31:07Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestShelveInstance-server-805905709',display_name='tempest-TestShelveInstance-server-805905709',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testshelveinstance-server-805905709',id=37,image_ref='',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name='tempest-TestShelveInstance-826835257',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:31:26Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=4,progress=0,project_id='4d0cc057c220422a9541bcb9fa7f598b',ramdisk_id='',reservation_id='r-ndibkwoj',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TestShelveInstance-1998695689',owner_user_name='tempest-TestShelveInstance-1998695689-project-member'},tags=<?>,task_state='spawning',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:34:53Z,user_data=None,user_id='0aa403f4ead84aecb78a538c2ced64e9',uuid=8e970bdd-54a1-4058-b09d-784cb6b5b1a4,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='shelved_offloaded') vif={"id": "a83afb1f-511f-406c-b925-eb70b32bed90", "address": "fa:16:3e:16:83:e9", "network": {"id": "0b3a34b6-6d32-4414-9135-22bd2a1dae20", "bridge": "br-int", "label": "tempest-TestShelveInstance-2024538322-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.128", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4d0cc057c220422a9541bcb9fa7f598b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa83afb1f-51", "ovs_interfaceid": "a83afb1f-511f-406c-b925-eb70b32bed90", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:35:06 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Converting VIF {"id": "a83afb1f-511f-406c-b925-eb70b32bed90", "address": "fa:16:3e:16:83:e9", "network": {"id": "0b3a34b6-6d32-4414-9135-22bd2a1dae20", "bridge": "br-int", "label": "tempest-TestShelveInstance-2024538322-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.128", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4d0cc057c220422a9541bcb9fa7f598b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa83afb1f-51", "ovs_interfaceid": "a83afb1f-511f-406c-b925-eb70b32bed90", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:35:06 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:16:83:e9,bridge_name='br-int',has_traffic_filtering=True,id=a83afb1f-511f-406c-b925-eb70b32bed90,network=Network(0b3a34b6-6d32-4414-9135-22bd2a1dae20),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa83afb1f-51') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:35:06 user nova-compute[72351]: DEBUG nova.objects.instance [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Lazy-loading 'pci_devices' on Instance uuid 8e970bdd-54a1-4058-b09d-784cb6b5b1a4 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:35:06 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:35:06 user nova-compute[72351]:   <uuid>8e970bdd-54a1-4058-b09d-784cb6b5b1a4</uuid>
Aug 09 02:35:06 user nova-compute[72351]:   <name>instance-00000025</name>
Aug 09 02:35:06 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:35:06 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:35:06 user nova-compute[72351]:   <metadata>
Aug 09 02:35:06 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:35:06 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:35:06 user nova-compute[72351]:       <nova:name>tempest-TestShelveInstance-server-805905709</nova:name>
Aug 09 02:35:06 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:35:05</nova:creationTime>
Aug 09 02:35:06 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:35:06 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:35:06 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:35:06 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:35:06 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:35:06 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:35:06 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:35:06 user nova-compute[72351]:       <nova:owner>
Aug 09 02:35:06 user nova-compute[72351]:         <nova:user uuid="0aa403f4ead84aecb78a538c2ced64e9">tempest-TestShelveInstance-1998695689-project-member</nova:user>
Aug 09 02:35:06 user nova-compute[72351]:         <nova:project uuid="4d0cc057c220422a9541bcb9fa7f598b">tempest-TestShelveInstance-1998695689</nova:project>
Aug 09 02:35:06 user nova-compute[72351]:       </nova:owner>
Aug 09 02:35:06 user nova-compute[72351]:       <nova:ports>
Aug 09 02:35:06 user nova-compute[72351]:         <nova:port uuid="a83afb1f-511f-406c-b925-eb70b32bed90">
Aug 09 02:35:06 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.6" ipVersion="4"/>
Aug 09 02:35:06 user nova-compute[72351]:         </nova:port>
Aug 09 02:35:06 user nova-compute[72351]:       </nova:ports>
Aug 09 02:35:06 user nova-compute[72351]:     </nova:instance>
Aug 09 02:35:06 user nova-compute[72351]:   </metadata>
Aug 09 02:35:06 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:35:06 user nova-compute[72351]:     <system>
Aug 09 02:35:06 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:35:06 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:35:06 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:35:06 user nova-compute[72351]:       <entry name="serial">8e970bdd-54a1-4058-b09d-784cb6b5b1a4</entry>
Aug 09 02:35:06 user nova-compute[72351]:       <entry name="uuid">8e970bdd-54a1-4058-b09d-784cb6b5b1a4</entry>
Aug 09 02:35:06 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:35:06 user nova-compute[72351]:     </system>
Aug 09 02:35:06 user nova-compute[72351]:   </sysinfo>
Aug 09 02:35:06 user nova-compute[72351]:   <os>
Aug 09 02:35:06 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:35:06 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:35:06 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:35:06 user nova-compute[72351]:   </os>
Aug 09 02:35:06 user nova-compute[72351]:   <features>
Aug 09 02:35:06 user nova-compute[72351]:     <acpi/>
Aug 09 02:35:06 user nova-compute[72351]:     <apic/>
Aug 09 02:35:06 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:35:06 user nova-compute[72351]:   </features>
Aug 09 02:35:06 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:35:06 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:35:06 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:35:06 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:35:06 user nova-compute[72351]:   </clock>
Aug 09 02:35:06 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:35:06 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:35:06 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:35:06 user nova-compute[72351]:   </cpu>
Aug 09 02:35:06 user nova-compute[72351]:   <devices>
Aug 09 02:35:06 user nova-compute[72351]:     <disk type="block" device="disk">
Aug 09 02:35:06 user nova-compute[72351]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:35:06 user nova-compute[72351]:       <source dev="/dev/sdc"/>
Aug 09 02:35:06 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:35:06 user nova-compute[72351]:       <serial>bc173cc3-1a94-467d-8d0b-d0f50936cee3</serial>
Aug 09 02:35:06 user nova-compute[72351]:     </disk>
Aug 09 02:35:06 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:35:06 user nova-compute[72351]:       <mac address="fa:16:3e:16:83:e9"/>
Aug 09 02:35:06 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:35:06 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:35:06 user nova-compute[72351]:       <target dev="tapa83afb1f-51"/>
Aug 09 02:35:06 user nova-compute[72351]:     </interface>
Aug 09 02:35:06 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:35:06 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/8e970bdd-54a1-4058-b09d-784cb6b5b1a4/console.log" append="off"/>
Aug 09 02:35:06 user nova-compute[72351]:     </serial>
Aug 09 02:35:06 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:35:06 user nova-compute[72351]:     <video>
Aug 09 02:35:06 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:35:06 user nova-compute[72351]:     </video>
Aug 09 02:35:06 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:35:06 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:35:06 user nova-compute[72351]:     </rng>
Aug 09 02:35:06 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:35:06 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:35:06 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:35:06 user nova-compute[72351]:     </memballoon>
Aug 09 02:35:06 user nova-compute[72351]:   </devices>
Aug 09 02:35:06 user nova-compute[72351]: </domain>
Aug 09 02:35:06 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:35:06 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,compute_id=1,config_drive='',created_at=2023-08-09T02:31:07Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestShelveInstance-server-805905709',display_name='tempest-TestShelveInstance-server-805905709',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testshelveinstance-server-805905709',id=37,image_ref='',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name='tempest-TestShelveInstance-826835257',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:31:26Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=4,progress=0,project_id='4d0cc057c220422a9541bcb9fa7f598b',ramdisk_id='',reservation_id='r-ndibkwoj',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TestShelveInstance-1998695689',owner_user_name='tempest-TestShelveInstance-1998695689-project-member'},tags=<?>,task_state='spawning',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:34:53Z,user_data=None,user_id='0aa403f4ead84aecb78a538c2ced64e9',uuid=8e970bdd-54a1-4058-b09d-784cb6b5b1a4,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='shelved_offloaded') vif={"id": "a83afb1f-511f-406c-b925-eb70b32bed90", "address": "fa:16:3e:16:83:e9", "network": {"id": "0b3a34b6-6d32-4414-9135-22bd2a1dae20", "bridge": "br-int", "label": "tempest-TestShelveInstance-2024538322-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.128", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4d0cc057c220422a9541bcb9fa7f598b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa83afb1f-51", "ovs_interfaceid": "a83afb1f-511f-406c-b925-eb70b32bed90", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:35:06 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Converting VIF {"id": "a83afb1f-511f-406c-b925-eb70b32bed90", "address": "fa:16:3e:16:83:e9", "network": {"id": "0b3a34b6-6d32-4414-9135-22bd2a1dae20", "bridge": "br-int", "label": "tempest-TestShelveInstance-2024538322-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.128", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4d0cc057c220422a9541bcb9fa7f598b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa83afb1f-51", "ovs_interfaceid": "a83afb1f-511f-406c-b925-eb70b32bed90", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:35:06 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:16:83:e9,bridge_name='br-int',has_traffic_filtering=True,id=a83afb1f-511f-406c-b925-eb70b32bed90,network=Network(0b3a34b6-6d32-4414-9135-22bd2a1dae20),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa83afb1f-51') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:35:06 user nova-compute[72351]: DEBUG os_vif [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:16:83:e9,bridge_name='br-int',has_traffic_filtering=True,id=a83afb1f-511f-406c-b925-eb70b32bed90,network=Network(0b3a34b6-6d32-4414-9135-22bd2a1dae20),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa83afb1f-51') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:35:06 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:06 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:35:06 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:35:06 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:06 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapa83afb1f-51, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:35:06 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapa83afb1f-51, col_values=(('external_ids', {'iface-id': 'a83afb1f-511f-406c-b925-eb70b32bed90', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:16:83:e9', 'vm-uuid': '8e970bdd-54a1-4058-b09d-784cb6b5b1a4'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:35:06 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:35:06 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:06 user nova-compute[72351]: INFO os_vif [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:16:83:e9,bridge_name='br-int',has_traffic_filtering=True,id=a83afb1f-511f-406c-b925-eb70b32bed90,network=Network(0b3a34b6-6d32-4414-9135-22bd2a1dae20),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa83afb1f-51')
Aug 09 02:35:06 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:35:06 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] No VIF found with MAC fa:16:3e:16:83:e9, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:35:07 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:07 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:08 user nova-compute[72351]: DEBUG nova.compute.manager [req-65389a40-f578-49b9-9127-a9d79df10fc4 req-0b92a5bd-dc9b-4304-a130-4524ee717bf9 service nova] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Received event network-vif-plugged-a83afb1f-511f-406c-b925-eb70b32bed90 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:35:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-65389a40-f578-49b9-9127-a9d79df10fc4 req-0b92a5bd-dc9b-4304-a130-4524ee717bf9 service nova] Acquiring lock "8e970bdd-54a1-4058-b09d-784cb6b5b1a4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:35:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-65389a40-f578-49b9-9127-a9d79df10fc4 req-0b92a5bd-dc9b-4304-a130-4524ee717bf9 service nova] Lock "8e970bdd-54a1-4058-b09d-784cb6b5b1a4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:35:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-65389a40-f578-49b9-9127-a9d79df10fc4 req-0b92a5bd-dc9b-4304-a130-4524ee717bf9 service nova] Lock "8e970bdd-54a1-4058-b09d-784cb6b5b1a4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:35:08 user nova-compute[72351]: DEBUG nova.compute.manager [req-65389a40-f578-49b9-9127-a9d79df10fc4 req-0b92a5bd-dc9b-4304-a130-4524ee717bf9 service nova] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] No waiting events found dispatching network-vif-plugged-a83afb1f-511f-406c-b925-eb70b32bed90 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:35:08 user nova-compute[72351]: WARNING nova.compute.manager [req-65389a40-f578-49b9-9127-a9d79df10fc4 req-0b92a5bd-dc9b-4304-a130-4524ee717bf9 service nova] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Received unexpected event network-vif-plugged-a83afb1f-511f-406c-b925-eb70b32bed90 for instance with vm_state shelved_offloaded and task_state spawning.
Aug 09 02:35:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:08 user nova-compute[72351]: DEBUG os_brick.utils [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:35:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.016s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:08 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[27da07d1-a898-4aec-ab57-357868f79470]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:a52269602ff4\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.016s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:08 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a814b68a-afac-4b97-aeab-95616e831869]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:08 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:35:08 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:35:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:35:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:35:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:35:08 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Auditing locally available compute resources for user (node: user) {{(pid=72351) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Aug 09 02:35:08 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ec33417c-ea6f-4045-8fd0-f0b5ea53cec6]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:08 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ffbcea80-4273-4fa5-9df7-0f01c952de42]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Running cmd (subprocess): nvme version {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] 'nvme version' failed. Not Retrying. {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:35:08 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.nvmeof [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] nvme not present on system {{(pid=72351) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Aug 09 02:35:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:09 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:35:09 user nova-compute[72351]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Aug 09 02:35:09 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[b494e82d-4aef-4c48-8a4c-f0f52fac94a9]: (4, '') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:09 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72351) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Aug 09 02:35:09 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Aug 09 02:35:09 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] LIGHTOS: no hostnqn found. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Aug 09 02:35:09 user nova-compute[72351]: DEBUG os_brick.utils [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] <== get_connector_properties: return (141ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:a52269602ff4', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:35:09 user nova-compute[72351]: DEBUG nova.virt.block_device [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Updating existing volume attachment record: 62241c55-0d08-4cc4-8b91-0e840a1e3282 {{(pid=72351) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Aug 09 02:35:09 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7bef74f4-3bb3-4ad0-bff7-34156be31249/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:09 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7bef74f4-3bb3-4ad0-bff7-34156be31249/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:09 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7bef74f4-3bb3-4ad0-bff7-34156be31249/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:09 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7bef74f4-3bb3-4ad0-bff7-34156be31249/disk --force-share --output=json" returned: 0 in 0.140s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:09 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/41b5e305-3973-4937-b262-d32db1091e20/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:09 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/41b5e305-3973-4937-b262-d32db1091e20/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:09 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/41b5e305-3973-4937-b262-d32db1091e20/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:09 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/41b5e305-3973-4937-b262-d32db1091e20/disk --force-share --output=json" returned: 0 in 0.146s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:09 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sde (vda) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:35:09 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:09 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:09 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:10 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548510.0237288, 8e970bdd-54a1-4058-b09d-784cb6b5b1a4 => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:35:10 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] VM Resumed (Lifecycle Event)
Aug 09 02:35:10 user nova-compute[72351]: DEBUG nova.compute.manager [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:35:10 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Guest created on hypervisor {{(pid=72351) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4411}}
Aug 09 02:35:10 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Instance spawned successfully.
Aug 09 02:35:10 user nova-compute[72351]: DEBUG nova.compute.manager [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:35:10 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:35:10 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: shelved_offloaded, current task_state: spawning, current DB power_state: 4, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:35:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json" returned: 0 in 0.147s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:10 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdf (vda) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:35:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.rescue --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:10 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:35:10 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548510.037247, 8e970bdd-54a1-4058-b09d-784cb6b5b1a4 => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:35:10 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] VM Started (Lifecycle Event)
Aug 09 02:35:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-3c544167-ade4-42f3-9e78-6463d8b7f4ee tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Lock "8e970bdd-54a1-4058-b09d-784cb6b5b1a4" "released" by "nova.compute.manager.ComputeManager.unshelve_instance.<locals>.do_unshelve_instance" :: held 17.177s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:35:10 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:35:10 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Synchronizing instance power state after lifecycle event "Started"; current vm_state: active, current task_state: None, current DB power_state: 1, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:35:10 user nova-compute[72351]: DEBUG nova.compute.manager [req-5c8f6be1-ec36-4895-a409-6840da9e728b req-40cbe20e-a627-418f-a2bc-52aef36db6fb service nova] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Received event network-vif-plugged-a83afb1f-511f-406c-b925-eb70b32bed90 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:35:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-5c8f6be1-ec36-4895-a409-6840da9e728b req-40cbe20e-a627-418f-a2bc-52aef36db6fb service nova] Acquiring lock "8e970bdd-54a1-4058-b09d-784cb6b5b1a4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:35:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-5c8f6be1-ec36-4895-a409-6840da9e728b req-40cbe20e-a627-418f-a2bc-52aef36db6fb service nova] Lock "8e970bdd-54a1-4058-b09d-784cb6b5b1a4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:35:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-5c8f6be1-ec36-4895-a409-6840da9e728b req-40cbe20e-a627-418f-a2bc-52aef36db6fb service nova] Lock "8e970bdd-54a1-4058-b09d-784cb6b5b1a4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:35:10 user nova-compute[72351]: DEBUG nova.compute.manager [req-5c8f6be1-ec36-4895-a409-6840da9e728b req-40cbe20e-a627-418f-a2bc-52aef36db6fb service nova] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] No waiting events found dispatching network-vif-plugged-a83afb1f-511f-406c-b925-eb70b32bed90 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:35:10 user nova-compute[72351]: WARNING nova.compute.manager [req-5c8f6be1-ec36-4895-a409-6840da9e728b req-40cbe20e-a627-418f-a2bc-52aef36db6fb service nova] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Received unexpected event network-vif-plugged-a83afb1f-511f-406c-b925-eb70b32bed90 for instance with vm_state active and task_state None.
Aug 09 02:35:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.rescue --force-share --output=json" returned: 0 in 0.141s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.rescue --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.rescue --force-share --output=json" returned: 0 in 0.136s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json" returned: 0 in 0.171s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:10 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdc (vda) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:35:10 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdg (vda) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:35:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7b6f5d60-5f23-4ef1-b972-9ba7086f56cb/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7b6f5d60-5f23-4ef1-b972-9ba7086f56cb/disk --force-share --output=json" returned: 0 in 0.140s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7b6f5d60-5f23-4ef1-b972-9ba7086f56cb/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7b6f5d60-5f23-4ef1-b972-9ba7086f56cb/disk --force-share --output=json" returned: 0 in 0.140s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:10 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdb (vdb) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:35:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json" returned: 0 in 0.172s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:11 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdd (vda) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:35:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json" returned: 0 in 0.161s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:11 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json" returned: 0 in 0.139s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:12 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:35:12 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:35:12 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Hypervisor/Node resource view: name=user free_ram=7109MB free_disk=25.147613525390625GB free_vcpus=0 pci_devices=[{"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72351) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Aug 09 02:35:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:35:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:35:12 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:35:12 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance eb86cb49-2018-42c2-bd97-c662b829775d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:35:12 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:35:12 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance a7f3d076-864b-4b68-9207-214ce2c21444 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:35:12 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 957c1510-1320-4492-9052-a58ff540319d actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:35:12 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:35:12 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 49a10075-0d3c-4cd6-bb43-10238cad3f37 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:35:12 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 7bef74f4-3bb3-4ad0-bff7-34156be31249 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:35:12 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 41b5e305-3973-4937-b262-d32db1091e20 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:35:12 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 80e87454-90b9-4168-bf00-b7d93b245526 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 2048, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:35:12 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance ad9b389e-ea11-4fd7-9233-a1be43052e60 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:35:12 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 8e970bdd-54a1-4058-b09d-784cb6b5b1a4 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:35:12 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance bde1ef1e-163c-40eb-abb5-962b9064b7ab actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:35:12 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Total usable vcpus: 12, total allocated vcpus: 13 {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Aug 09 02:35:12 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Final resource view: name=user phys_ram=16011MB used_ram=8704MB phys_disk=40GB used_disk=7GB total_vcpus=12 used_vcpus=13 pci_stats=[] {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Aug 09 02:35:12 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:35:12 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:35:12 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Compute_service record updated for user:user {{(pid=72351) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Aug 09 02:35:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.580s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:35:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:14 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-adc3b121-6c45-4913-b281-cb74fc4b9036 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 02:35:14 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-adc3b121-6c45-4913-b281-cb74fc4b9036 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:387cac3d-74d5-4a21-9997-bf342fecc112'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'OUjBvElF', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:35:14 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-adc3b121-6c45-4913-b281-cb74fc4b9036 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:35:14 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-adc3b121-6c45-4913-b281-cb74fc4b9036 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:35:14 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-adc3b121-6c45-4913-b281-cb74fc4b9036 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:35:14 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-adc3b121-6c45-4913-b281-cb74fc4b9036 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:35:14 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-adc3b121-6c45-4913-b281-cb74fc4b9036 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:387cac3d-74d5-4a21-9997-bf342fecc112" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:35:14 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-adc3b121-6c45-4913-b281-cb74fc4b9036 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:387cac3d-74d5-4a21-9997-bf342fecc112" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:35:14 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-adc3b121-6c45-4913-b281-cb74fc4b9036 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 02:35:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:387cac3d-74d5-4a21-9997-bf342fecc112 -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:387cac3d-74d5-4a21-9997-bf342fecc112 -p 172.16.0.220:3260" returned: 21 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:14 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[2d0bfcd9-790b-4a6e-8a49-a815bda4ba6f]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:14 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-adc3b121-6c45-4913-b281-cb74fc4b9036 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Aug 09 02:35:14 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:35:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:387cac3d-74d5-4a21-9997-bf342fecc112 -p 172.16.0.220:3260 --interface default --op new {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:387cac3d-74d5-4a21-9997-bf342fecc112 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:14 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[6b6eea37-9e46-43c5-8c84-63b275414276]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:387cac3d-74d5-4a21-9997-bf342fecc112] added\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:14 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-adc3b121-6c45-4913-b281-cb74fc4b9036 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:387cac3d-74d5-4a21-9997-bf342fecc112] added
Aug 09 02:35:14 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:35:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:387cac3d-74d5-4a21-9997-bf342fecc112 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:387cac3d-74d5-4a21-9997-bf342fecc112 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:14 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[32a06af1-255e-49ff-8861-b304c51e8444]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:14 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-adc3b121-6c45-4913-b281-cb74fc4b9036 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:35:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:387cac3d-74d5-4a21-9997-bf342fecc112 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:387cac3d-74d5-4a21-9997-bf342fecc112 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:14 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[307d1ff3-9696-4f00-b8a6-a4c7a0d640d9]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:14 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-adc3b121-6c45-4913-b281-cb74fc4b9036 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:35:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:387cac3d-74d5-4a21-9997-bf342fecc112 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v OUjBvElF {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:387cac3d-74d5-4a21-9997-bf342fecc112 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v OUjBvElF" returned: 0 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:14 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[95aab9ee-f1db-4c7b-b798-cf0e9aa95e8d]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:14 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-adc3b121-6c45-4913-b281-cb74fc4b9036 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'OUjBvElF'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:35:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:387cac3d-74d5-4a21-9997-bf342fecc112 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:387cac3d-74d5-4a21-9997-bf342fecc112 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:14 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[3a6dab9b-08d8-4a05-a3d8-50329b30df34]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:14 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-adc3b121-6c45-4913-b281-cb74fc4b9036 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:35:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.034s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:14 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[947a2201-ab07-4bb7-b6b7-9ad5825b8308]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)\ntcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)\ntcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)\ntcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:14 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-adc3b121-6c45-4913-b281-cb74fc4b9036 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:35:14 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:35:14 user nova-compute[72351]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)
Aug 09 02:35:14 user nova-compute[72351]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:35:14 user nova-compute[72351]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)
Aug 09 02:35:14 user nova-compute[72351]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)
Aug 09 02:35:14 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:35:14 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-adc3b121-6c45-4913-b281-cb74fc4b9036 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:35:14 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:35:14 user nova-compute[72351]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)
Aug 09 02:35:14 user nova-compute[72351]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:35:14 user nova-compute[72351]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)
Aug 09 02:35:14 user nova-compute[72351]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)
Aug 09 02:35:14 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:35:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:387cac3d-74d5-4a21-9997-bf342fecc112 -p 172.16.0.220:3260 --login {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:387cac3d-74d5-4a21-9997-bf342fecc112 -p 172.16.0.220:3260 --login" returned: 0 in 0.083s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:14 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[c309c49d-fcc4-4ad7-8d25-a04de168d4a5]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:387cac3d-74d5-4a21-9997-bf342fecc112, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:387cac3d-74d5-4a21-9997-bf342fecc112, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:14 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-adc3b121-6c45-4913-b281-cb74fc4b9036 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:387cac3d-74d5-4a21-9997-bf342fecc112, portal: 172.16.0.220,3260]
Aug 09 02:35:14 user nova-compute[72351]: Login to [iface: default, target: iqn.2010-10.org.openstack:387cac3d-74d5-4a21-9997-bf342fecc112, portal: 172.16.0.220,3260] successful.
Aug 09 02:35:14 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:35:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:387cac3d-74d5-4a21-9997-bf342fecc112 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:387cac3d-74d5-4a21-9997-bf342fecc112 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.024s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:14 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a2488660-a8a8-4247-9c89-d02e2f9c2406]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:14 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-adc3b121-6c45-4913-b281-cb74fc4b9036 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:35:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.033s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:14 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[b0cbdd35-a6bc-46d0-b929-b03e6bdb9dd7]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)\ntcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)\ntcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)\ntcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)\ntcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:387cac3d-74d5-4a21-9997-bf342fecc112 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:14 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-adc3b121-6c45-4913-b281-cb74fc4b9036 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:35:14 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:35:14 user nova-compute[72351]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)
Aug 09 02:35:14 user nova-compute[72351]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:35:14 user nova-compute[72351]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)
Aug 09 02:35:14 user nova-compute[72351]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)
Aug 09 02:35:14 user nova-compute[72351]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:387cac3d-74d5-4a21-9997-bf342fecc112 (non-flash)
Aug 09 02:35:14 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:35:14 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-adc3b121-6c45-4913-b281-cb74fc4b9036 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:35:14 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:35:14 user nova-compute[72351]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)
Aug 09 02:35:14 user nova-compute[72351]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:35:14 user nova-compute[72351]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)
Aug 09 02:35:14 user nova-compute[72351]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)
Aug 09 02:35:14 user nova-compute[72351]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:387cac3d-74d5-4a21-9997-bf342fecc112 (non-flash)
Aug 09 02:35:14 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:35:14 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-adc3b121-6c45-4913-b281-cb74fc4b9036 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:387cac3d-74d5-4a21-9997-bf342fecc112" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.343s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:35:14 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-adc3b121-6c45-4913-b281-cb74fc4b9036 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 02:35:14 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-adc3b121-6c45-4913-b281-cb74fc4b9036 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] HCTL ('39', '-', '-', 0) found on session 47 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 02:35:14 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-adc3b121-6c45-4913-b281-cb74fc4b9036 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Scanning host 39 c: -, t: -, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 02:35:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host39/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host39/scan" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:14 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d57c08d0-a2c7-4434-a0a2-aed8c7a88bbb]: (4, ('- - 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:14 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-adc3b121-6c45-4913-b281-cb74fc4b9036 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Searching for a device in session 47 and hctl ['39', '*', '*', 0] yield: sdh {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:35:14 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-adc3b121-6c45-4913-b281-cb74fc4b9036 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Connected to sdh using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'OUjBvElF', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 02:35:14 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:35:14 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:35:14 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:35:14 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Starting heal instance info cache {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9842}}
Aug 09 02:35:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "refresh_cache-957c1510-1320-4492-9052-a58ff540319d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:35:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquired lock "refresh_cache-957c1510-1320-4492-9052-a58ff540319d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:35:14 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 957c1510-1320-4492-9052-a58ff540319d] Forcefully refreshing network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1996}}
Aug 09 02:35:15 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 957c1510-1320-4492-9052-a58ff540319d] Updating instance_info_cache with network_info: [{"id": "e679cfd9-123f-4b88-8fd2-397e4c8616cc", "address": "fa:16:3e:7f:d6:1b", "network": {"id": "9a452414-d61f-4045-a6cc-9be93c892c4c", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "321111f935dd4e8aab4a083206ea9a7e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape679cfd9-12", "ovs_interfaceid": "e679cfd9-123f-4b88-8fd2-397e4c8616cc", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:35:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Releasing lock "refresh_cache-957c1510-1320-4492-9052-a58ff540319d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:35:15 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 957c1510-1320-4492-9052-a58ff540319d] Updated the network info_cache for instance {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9913}}
Aug 09 02:35:15 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:35:15 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:35:15 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:35:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-adc3b121-6c45-4913-b281-cb74fc4b9036 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.383s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:35:15 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-adc3b121-6c45-4913-b281-cb74fc4b9036 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] <== connect_volume: return (1384ms) {'type': 'block', 'scsi_wwn': '23232663432646162', 'path': '/dev/sdh'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:35:15 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-adc3b121-6c45-4913-b281-cb74fc4b9036 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23232663432646162', 'path': '/dev/sdh'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 02:35:15 user nova-compute[72351]: DEBUG nova.objects.instance [None req-adc3b121-6c45-4913-b281-cb74fc4b9036 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lazy-loading 'flavor' on Instance uuid 7bef74f4-3bb3-4ad0-bff7-34156be31249 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:35:15 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-adc3b121-6c45-4913-b281-cb74fc4b9036 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] attach device xml: <disk type="block" device="disk">
Aug 09 02:35:15 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:35:15 user nova-compute[72351]:   <source dev="/dev/sdh"/>
Aug 09 02:35:15 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:35:15 user nova-compute[72351]:   <serial>387cac3d-74d5-4a21-9997-bf342fecc112</serial>
Aug 09 02:35:15 user nova-compute[72351]: </disk>
Aug 09 02:35:15 user nova-compute[72351]:  {{(pid=72351) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Aug 09 02:35:15 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691548500.650399, a03fbded-23ad-440b-97e7-bea07425e873 => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:35:15 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: a03fbded-23ad-440b-97e7-bea07425e873] VM Stopped (Lifecycle Event)
Aug 09 02:35:15 user nova-compute[72351]: DEBUG nova.compute.manager [None req-dce5bb22-8e12-4d4b-a4e3-12c44b12d278 None None] [instance: a03fbded-23ad-440b-97e7-bea07425e873] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:35:15 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-adc3b121-6c45-4913-b281-cb74fc4b9036 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:35:15 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-adc3b121-6c45-4913-b281-cb74fc4b9036 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:35:15 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-adc3b121-6c45-4913-b281-cb74fc4b9036 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] No VIF found with MAC fa:16:3e:b4:8f:2e, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:35:16 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-adc3b121-6c45-4913-b281-cb74fc4b9036 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "7bef74f4-3bb3-4ad0-bff7-34156be31249" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 11.618s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:35:16 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:17 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-da231063-70f4-43e9-a5fd-214c07b509cd tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Acquiring lock "7bef74f4-3bb3-4ad0-bff7-34156be31249" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:35:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-da231063-70f4-43e9-a5fd-214c07b509cd tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "7bef74f4-3bb3-4ad0-bff7-34156be31249" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:35:17 user nova-compute[72351]: INFO nova.compute.manager [None req-da231063-70f4-43e9-a5fd-214c07b509cd tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Detaching volume 387cac3d-74d5-4a21-9997-bf342fecc112
Aug 09 02:35:18 user nova-compute[72351]: INFO nova.virt.block_device [None req-da231063-70f4-43e9-a5fd-214c07b509cd tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Attempting to driver detach volume 387cac3d-74d5-4a21-9997-bf342fecc112 from mountpoint /dev/vdb
Aug 09 02:35:18 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-da231063-70f4-43e9-a5fd-214c07b509cd tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Attempting to detach device vdb from instance 7bef74f4-3bb3-4ad0-bff7-34156be31249 from the persistent domain config. {{(pid=72351) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2477}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-da231063-70f4-43e9-a5fd-214c07b509cd tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 02:35:18 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:35:18 user nova-compute[72351]:   <source dev="/dev/sdh"/>
Aug 09 02:35:18 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:35:18 user nova-compute[72351]:   <serial>387cac3d-74d5-4a21-9997-bf342fecc112</serial>
Aug 09 02:35:18 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Aug 09 02:35:18 user nova-compute[72351]: </disk>
Aug 09 02:35:18 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 02:35:18 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-da231063-70f4-43e9-a5fd-214c07b509cd tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Successfully detached device vdb from instance 7bef74f4-3bb3-4ad0-bff7-34156be31249 from the persistent domain config.
Aug 09 02:35:18 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-da231063-70f4-43e9-a5fd-214c07b509cd tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 7bef74f4-3bb3-4ad0-bff7-34156be31249 from the live domain config. {{(pid=72351) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2513}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-da231063-70f4-43e9-a5fd-214c07b509cd tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 02:35:18 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:35:18 user nova-compute[72351]:   <source dev="/dev/sdh"/>
Aug 09 02:35:18 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:35:18 user nova-compute[72351]:   <serial>387cac3d-74d5-4a21-9997-bf342fecc112</serial>
Aug 09 02:35:18 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Aug 09 02:35:18 user nova-compute[72351]: </disk>
Aug 09 02:35:18 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Received event <DeviceRemovedEvent: 1691548518.495267, 7bef74f4-3bb3-4ad0-bff7-34156be31249 => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=72351) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2360}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-da231063-70f4-43e9-a5fd-214c07b509cd tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 7bef74f4-3bb3-4ad0-bff7-34156be31249 {{(pid=72351) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2589}}
Aug 09 02:35:18 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-da231063-70f4-43e9-a5fd-214c07b509cd tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Successfully detached device vdb from instance 7bef74f4-3bb3-4ad0-bff7-34156be31249 from the live domain config.
Aug 09 02:35:18 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-da231063-70f4-43e9-a5fd-214c07b509cd tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-da231063-70f4-43e9-a5fd-214c07b509cd tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:387cac3d-74d5-4a21-9997-bf342fecc112'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'OUjBvElF', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdh'}, None), 'kwargs': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:35:18 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-da231063-70f4-43e9-a5fd-214c07b509cd tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:35:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-da231063-70f4-43e9-a5fd-214c07b509cd tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-da231063-70f4-43e9-a5fd-214c07b509cd tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-da231063-70f4-43e9-a5fd-214c07b509cd tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:387cac3d-74d5-4a21-9997-bf342fecc112', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.026s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[3594dfdd-1994-429b-a8bf-24572156ec92]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:387cac3d-74d5-4a21-9997-bf342fecc112\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.032s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[c43d75e1-2e7e-4a31-b364-0a43bcdd19cf]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)\ntcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)\ntcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)\ntcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)\ntcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:387cac3d-74d5-4a21-9997-bf342fecc112 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-da231063-70f4-43e9-a5fd-214c07b509cd tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:35:18 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:35:18 user nova-compute[72351]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)
Aug 09 02:35:18 user nova-compute[72351]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:35:18 user nova-compute[72351]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)
Aug 09 02:35:18 user nova-compute[72351]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)
Aug 09 02:35:18 user nova-compute[72351]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:387cac3d-74d5-4a21-9997-bf342fecc112 (non-flash)
Aug 09 02:35:18 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-da231063-70f4-43e9-a5fd-214c07b509cd tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:35:18 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:35:18 user nova-compute[72351]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)
Aug 09 02:35:18 user nova-compute[72351]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:35:18 user nova-compute[72351]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)
Aug 09 02:35:18 user nova-compute[72351]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)
Aug 09 02:35:18 user nova-compute[72351]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:387cac3d-74d5-4a21-9997-bf342fecc112 (non-flash)
Aug 09 02:35:18 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-da231063-70f4-43e9-a5fd-214c07b509cd tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fcfbc7d44c0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:387cac3d-74d5-4a21-9997-bf342fecc112'): ({'sdh'}, set())}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-da231063-70f4-43e9-a5fd-214c07b509cd tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Removing single pathed devices sdh {{(pid=72351) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ad92c795-49ca-4715-8214-434d6935ed6f]: (4, ('path checker states:\nup                  7\n\npaths: 0\nbusy: False\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdh {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdh" returned: 0 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d40ecb64-4ffc-42f2-b940-ff6d835bf38d]: (4, ('ok\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-da231063-70f4-43e9-a5fd-214c07b509cd tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Flushing IO for device /dev/sdh {{(pid=72351) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdh {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdh" returned: 0 in 0.029s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[0dedb291-1518-41a1-aed7-f64c7f833b96]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-da231063-70f4-43e9-a5fd-214c07b509cd tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Remove SCSI device /dev/sdh with /sys/block/sdh/device/delete {{(pid=72351) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdh/device/delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdh/device/delete" returned: 0 in 0.042s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[1e984509-33a5-4c79-94ef-bec5d100c032]: (4, ('1', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-da231063-70f4-43e9-a5fd-214c07b509cd tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Checking to see if SCSI volumes sdh have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-da231063-70f4-43e9-a5fd-214c07b509cd tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] SCSI volumes sdh have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-da231063-70f4-43e9-a5fd-214c07b509cd tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:387cac3d-74d5-4a21-9997-bf342fecc112')] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:387cac3d-74d5-4a21-9997-bf342fecc112 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:387cac3d-74d5-4a21-9997-bf342fecc112 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d3037bc5-6f31-40be-915e-6afce88e8fce]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-da231063-70f4-43e9-a5fd-214c07b509cd tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:387cac3d-74d5-4a21-9997-bf342fecc112 -p 172.16.0.220:3260 --logout {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:387cac3d-74d5-4a21-9997-bf342fecc112 -p 172.16.0.220:3260 --logout" returned: 0 in 0.075s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ef2dde31-1f8a-49a2-9d48-02569cdb3fdd]: (4, ('Logging out of session [sid: 47, target: iqn.2010-10.org.openstack:387cac3d-74d5-4a21-9997-bf342fecc112, portal: 172.16.0.220,3260]\nLogout of [sid: 47, target: iqn.2010-10.org.openstack:387cac3d-74d5-4a21-9997-bf342fecc112, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-da231063-70f4-43e9-a5fd-214c07b509cd tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 47, target: iqn.2010-10.org.openstack:387cac3d-74d5-4a21-9997-bf342fecc112, portal: 172.16.0.220,3260]
Aug 09 02:35:18 user nova-compute[72351]: Logout of [sid: 47, target: iqn.2010-10.org.openstack:387cac3d-74d5-4a21-9997-bf342fecc112, portal: 172.16.0.220,3260] successful.
Aug 09 02:35:18 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:387cac3d-74d5-4a21-9997-bf342fecc112 -p 172.16.0.220:3260 --op delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:387cac3d-74d5-4a21-9997-bf342fecc112 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.034s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[0180b0fe-eb4a-4367-9872-f16dc2902f60]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-da231063-70f4-43e9-a5fd-214c07b509cd tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-da231063-70f4-43e9-a5fd-214c07b509cd tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.363s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-da231063-70f4-43e9-a5fd-214c07b509cd tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] <== disconnect_volume: return (363ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-da231063-70f4-43e9-a5fd-214c07b509cd tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG nova.compute.manager [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Start spawning the instance on the hypervisor. {{(pid=72351) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2612}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Creating instance directory {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4717}}
Aug 09 02:35:18 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Creating image(s)
Aug 09 02:35:18 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Did not create local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4853}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Ensure instance console log exists: /opt/stack/data/nova/instances/bde1ef1e-163c-40eb-abb5-962b9064b7ab/console.log {{(pid=72351) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4603}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Start _get_guest_xml network_info=[{"id": "8424518d-87b7-4fc4-8a58-1b4f094a5ca0", "address": "fa:16:3e:08:61:83", "network": {"id": "9a452414-d61f-4045-a6cc-9be93c892c4c", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "321111f935dd4e8aab4a083206ea9a7e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8424518d-87", "ovs_interfaceid": "8424518d-87b7-4fc4-8a58-1b4f094a5ca0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'device_type': 'disk', 'guest_format': None, 'disk_bus': 'virtio', 'mount_device': '/dev/vda', 'boot_index': 0, 'attachment_id': '62241c55-0d08-4cc4-8b91-0e840a1e3282', 'delete_on_termination': False, 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqns': ['iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'LNfcKqMo', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': 'bde1ef1e-163c-40eb-abb5-962b9064b7ab', 'attached_at': '', 'detached_at': '', 'volume_id': 'b5789e9a-6f06-4239-8523-0e8607f350df', 'serial': 'b5789e9a-6f06-4239-8523-0e8607f350df'}, 'volume_type': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:35:18 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:35:18 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:35:18 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'LNfcKqMo', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:35:18 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:35:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:35:18 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:35:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:35:18 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 02:35:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df -p 172.16.0.220:3260" returned: 21 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:18 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e7659dd3-1d9b-4059-95c7-ee8a29912fbc]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:19 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Aug 09 02:35:19 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:35:19 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df -p 172.16.0.220:3260 --interface default --op new {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:19 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:19 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[860c4a5c-f7fb-4003-b53f-4a3f5f39b107]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df] added\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:19 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df] added
Aug 09 02:35:19 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:35:19 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:19 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.029s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:19 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[382681df-1b74-4a39-96b1-519ff1e93b35]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:19 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:35:19 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:19 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.025s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:19 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[1eeac24b-8b2b-4334-acc3-a082a325555d]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:19 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:35:19 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df -p 172.16.0.220:3260 --op update -n node.session.auth.username -v LNfcKqMo {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:19 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df -p 172.16.0.220:3260 --op update -n node.session.auth.username -v LNfcKqMo" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:19 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a0949519-c433-499a-a1db-b402c4754629]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:19 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'LNfcKqMo'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:35:19 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:19 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:19 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[afbaf5a2-5125-4a43-8895-e6fc718d6951]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:19 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:35:19 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:19 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.048s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:19 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[705cb9f7-7b51-41df-91c3-68f05094ceeb]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)\ntcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)\ntcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)\ntcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:19 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:35:19 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:35:19 user nova-compute[72351]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)
Aug 09 02:35:19 user nova-compute[72351]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:35:19 user nova-compute[72351]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)
Aug 09 02:35:19 user nova-compute[72351]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)
Aug 09 02:35:19 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:35:19 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:35:19 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:35:19 user nova-compute[72351]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)
Aug 09 02:35:19 user nova-compute[72351]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:35:19 user nova-compute[72351]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)
Aug 09 02:35:19 user nova-compute[72351]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)
Aug 09 02:35:19 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:35:19 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df -p 172.16.0.220:3260 --login {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:19 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df -p 172.16.0.220:3260 --login" returned: 0 in 0.211s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:19 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[096bb2f4-7e70-44af-81fd-e58be12aed8a]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:19 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df, portal: 172.16.0.220,3260]
Aug 09 02:35:19 user nova-compute[72351]: Login to [iface: default, target: iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df, portal: 172.16.0.220,3260] successful.
Aug 09 02:35:19 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:35:19 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:19 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.025s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:19 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[1bc0a3de-836e-4f69-9f17-f5059b85887d]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:19 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:35:19 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:19 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.043s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:19 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[6d318f7d-d450-4dab-9b72-fd82ce3a619e]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)\ntcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)\ntcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)\ntcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)\ntcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:19 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:35:19 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:35:19 user nova-compute[72351]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)
Aug 09 02:35:19 user nova-compute[72351]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:35:19 user nova-compute[72351]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)
Aug 09 02:35:19 user nova-compute[72351]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)
Aug 09 02:35:19 user nova-compute[72351]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df (non-flash)
Aug 09 02:35:19 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:35:19 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:35:19 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:35:19 user nova-compute[72351]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)
Aug 09 02:35:19 user nova-compute[72351]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:35:19 user nova-compute[72351]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)
Aug 09 02:35:19 user nova-compute[72351]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)
Aug 09 02:35:19 user nova-compute[72351]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df (non-flash)
Aug 09 02:35:19 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:35:19 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.511s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:35:19 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 02:35:19 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] HCTL ('39', '-', '-', 0) found on session 48 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 02:35:19 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Scanning host 39 c: -, t: -, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 02:35:19 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host39/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:19 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host39/scan" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:19 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[8089e2e7-efea-405e-bca1-ed58516c9108]: (4, ('- - 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:19 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Searching for a device in session 48 and hctl ['39', '*', '*', 0] yield: sdh {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:35:19 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Connected to sdh using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'LNfcKqMo', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 02:35:19 user nova-compute[72351]: DEBUG nova.compute.manager [req-fb4cd95a-6372-4892-a49b-a472b2acb6ab req-f6b9bb23-c38f-4a0e-8831-78cb44443355 service nova] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Received event network-changed-64d8b550-334e-42c4-b57f-145ccc73517c {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:35:19 user nova-compute[72351]: DEBUG nova.compute.manager [req-fb4cd95a-6372-4892-a49b-a472b2acb6ab req-f6b9bb23-c38f-4a0e-8831-78cb44443355 service nova] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Refreshing instance network info cache due to event network-changed-64d8b550-334e-42c4-b57f-145ccc73517c. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:35:19 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-fb4cd95a-6372-4892-a49b-a472b2acb6ab req-f6b9bb23-c38f-4a0e-8831-78cb44443355 service nova] Acquiring lock "refresh_cache-ad9b389e-ea11-4fd7-9233-a1be43052e60" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:35:19 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-fb4cd95a-6372-4892-a49b-a472b2acb6ab req-f6b9bb23-c38f-4a0e-8831-78cb44443355 service nova] Acquired lock "refresh_cache-ad9b389e-ea11-4fd7-9233-a1be43052e60" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:35:19 user nova-compute[72351]: DEBUG nova.network.neutron [req-fb4cd95a-6372-4892-a49b-a472b2acb6ab req-f6b9bb23-c38f-4a0e-8831-78cb44443355 service nova] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Refreshing network info cache for port 64d8b550-334e-42c4-b57f-145ccc73517c {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:35:19 user nova-compute[72351]: DEBUG nova.objects.instance [None req-da231063-70f4-43e9-a5fd-214c07b509cd tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lazy-loading 'flavor' on Instance uuid 7bef74f4-3bb3-4ad0-bff7-34156be31249 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:35:19 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-da231063-70f4-43e9-a5fd-214c07b509cd tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "7bef74f4-3bb3-4ad0-bff7-34156be31249" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.990s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:35:20 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.560s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:35:20 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] <== connect_volume: return (1562ms) {'type': 'block', 'scsi_wwn': '23333363661396463', 'path': '/dev/sdh'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:35:20 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23333363661396463', 'path': '/dev/sdh'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 02:35:20 user nova-compute[72351]: DEBUG nova.network.neutron [req-fb4cd95a-6372-4892-a49b-a472b2acb6ab req-f6b9bb23-c38f-4a0e-8831-78cb44443355 service nova] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Updated VIF entry in instance network info cache for port 64d8b550-334e-42c4-b57f-145ccc73517c. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:35:20 user nova-compute[72351]: DEBUG nova.network.neutron [req-fb4cd95a-6372-4892-a49b-a472b2acb6ab req-f6b9bb23-c38f-4a0e-8831-78cb44443355 service nova] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Updating instance_info_cache with network_info: [{"id": "64d8b550-334e-42c4-b57f-145ccc73517c", "address": "fa:16:3e:52:34:4d", "network": {"id": "f7042dd1-90f3-4ee1-99cd-e52f9264dc87", "bridge": "br-int", "label": "tempest-ServerActionsV293TestJSON-1595239201-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.209", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "624e34ea80f944509fde2cb5472ed203", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap64d8b550-33", "ovs_interfaceid": "64d8b550-334e-42c4-b57f-145ccc73517c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:35:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-fb4cd95a-6372-4892-a49b-a472b2acb6ab req-f6b9bb23-c38f-4a0e-8831-78cb44443355 service nova] Releasing lock "refresh_cache-ad9b389e-ea11-4fd7-9233-a1be43052e60" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:35:20 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:34:56Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-1202876429',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-1202876429',id=43,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='321111f935dd4e8aab4a083206ea9a7e',ramdisk_id='',reservation_id='r-p0t08coj',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-2108329302',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:34:58Z,user_data=None,user_id='73d05c48be5640c28455e8ba8b05ef31',uuid=bde1ef1e-163c-40eb-abb5-962b9064b7ab,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "8424518d-87b7-4fc4-8a58-1b4f094a5ca0", "address": "fa:16:3e:08:61:83", "network": {"id": "9a452414-d61f-4045-a6cc-9be93c892c4c", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "321111f935dd4e8aab4a083206ea9a7e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8424518d-87", "ovs_interfaceid": "8424518d-87b7-4fc4-8a58-1b4f094a5ca0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:35:20 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Converting VIF {"id": "8424518d-87b7-4fc4-8a58-1b4f094a5ca0", "address": "fa:16:3e:08:61:83", "network": {"id": "9a452414-d61f-4045-a6cc-9be93c892c4c", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "321111f935dd4e8aab4a083206ea9a7e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8424518d-87", "ovs_interfaceid": "8424518d-87b7-4fc4-8a58-1b4f094a5ca0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:35:20 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:08:61:83,bridge_name='br-int',has_traffic_filtering=True,id=8424518d-87b7-4fc4-8a58-1b4f094a5ca0,network=Network(9a452414-d61f-4045-a6cc-9be93c892c4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8424518d-87') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:35:20 user nova-compute[72351]: DEBUG nova.objects.instance [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lazy-loading 'pci_devices' on Instance uuid bde1ef1e-163c-40eb-abb5-962b9064b7ab {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:35:20 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:35:20 user nova-compute[72351]:   <uuid>bde1ef1e-163c-40eb-abb5-962b9064b7ab</uuid>
Aug 09 02:35:20 user nova-compute[72351]:   <name>instance-0000002b</name>
Aug 09 02:35:20 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:35:20 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:35:20 user nova-compute[72351]:   <metadata>
Aug 09 02:35:20 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:35:20 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:35:20 user nova-compute[72351]:       <nova:name>tempest-ServerBootFromVolumeStableRescueTest-server-1202876429</nova:name>
Aug 09 02:35:20 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:35:18</nova:creationTime>
Aug 09 02:35:20 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:35:20 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:35:20 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:35:20 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:35:20 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:35:20 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:35:20 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:35:20 user nova-compute[72351]:       <nova:owner>
Aug 09 02:35:20 user nova-compute[72351]:         <nova:user uuid="73d05c48be5640c28455e8ba8b05ef31">tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member</nova:user>
Aug 09 02:35:20 user nova-compute[72351]:         <nova:project uuid="321111f935dd4e8aab4a083206ea9a7e">tempest-ServerBootFromVolumeStableRescueTest-2108329302</nova:project>
Aug 09 02:35:20 user nova-compute[72351]:       </nova:owner>
Aug 09 02:35:20 user nova-compute[72351]:       <nova:root type="image" uuid="18ccda7f-31d3-4978-ab9d-4ed2f74eff94"/>
Aug 09 02:35:20 user nova-compute[72351]:       <nova:ports>
Aug 09 02:35:20 user nova-compute[72351]:         <nova:port uuid="8424518d-87b7-4fc4-8a58-1b4f094a5ca0">
Aug 09 02:35:20 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.3" ipVersion="4"/>
Aug 09 02:35:20 user nova-compute[72351]:         </nova:port>
Aug 09 02:35:20 user nova-compute[72351]:       </nova:ports>
Aug 09 02:35:20 user nova-compute[72351]:     </nova:instance>
Aug 09 02:35:20 user nova-compute[72351]:   </metadata>
Aug 09 02:35:20 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:35:20 user nova-compute[72351]:     <system>
Aug 09 02:35:20 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:35:20 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:35:20 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:35:20 user nova-compute[72351]:       <entry name="serial">bde1ef1e-163c-40eb-abb5-962b9064b7ab</entry>
Aug 09 02:35:20 user nova-compute[72351]:       <entry name="uuid">bde1ef1e-163c-40eb-abb5-962b9064b7ab</entry>
Aug 09 02:35:20 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:35:20 user nova-compute[72351]:     </system>
Aug 09 02:35:20 user nova-compute[72351]:   </sysinfo>
Aug 09 02:35:20 user nova-compute[72351]:   <os>
Aug 09 02:35:20 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:35:20 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:35:20 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:35:20 user nova-compute[72351]:   </os>
Aug 09 02:35:20 user nova-compute[72351]:   <features>
Aug 09 02:35:20 user nova-compute[72351]:     <acpi/>
Aug 09 02:35:20 user nova-compute[72351]:     <apic/>
Aug 09 02:35:20 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:35:20 user nova-compute[72351]:   </features>
Aug 09 02:35:20 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:35:20 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:35:20 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:35:20 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:35:20 user nova-compute[72351]:   </clock>
Aug 09 02:35:20 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:35:20 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:35:20 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:35:20 user nova-compute[72351]:   </cpu>
Aug 09 02:35:20 user nova-compute[72351]:   <devices>
Aug 09 02:35:20 user nova-compute[72351]:     <disk type="block" device="disk">
Aug 09 02:35:20 user nova-compute[72351]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:35:20 user nova-compute[72351]:       <source dev="/dev/sdh"/>
Aug 09 02:35:20 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:35:20 user nova-compute[72351]:       <serial>b5789e9a-6f06-4239-8523-0e8607f350df</serial>
Aug 09 02:35:20 user nova-compute[72351]:     </disk>
Aug 09 02:35:20 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:35:20 user nova-compute[72351]:       <mac address="fa:16:3e:08:61:83"/>
Aug 09 02:35:20 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:35:20 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:35:20 user nova-compute[72351]:       <target dev="tap8424518d-87"/>
Aug 09 02:35:20 user nova-compute[72351]:     </interface>
Aug 09 02:35:20 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:35:20 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/bde1ef1e-163c-40eb-abb5-962b9064b7ab/console.log" append="off"/>
Aug 09 02:35:20 user nova-compute[72351]:     </serial>
Aug 09 02:35:20 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:35:20 user nova-compute[72351]:     <video>
Aug 09 02:35:20 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:35:20 user nova-compute[72351]:     </video>
Aug 09 02:35:20 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:35:20 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:35:20 user nova-compute[72351]:     </rng>
Aug 09 02:35:20 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:35:20 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:35:20 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:35:20 user nova-compute[72351]:     </memballoon>
Aug 09 02:35:20 user nova-compute[72351]:   </devices>
Aug 09 02:35:20 user nova-compute[72351]: </domain>
Aug 09 02:35:20 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:35:20 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:34:56Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-1202876429',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-1202876429',id=43,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='321111f935dd4e8aab4a083206ea9a7e',ramdisk_id='',reservation_id='r-p0t08coj',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-2108329302',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:34:58Z,user_data=None,user_id='73d05c48be5640c28455e8ba8b05ef31',uuid=bde1ef1e-163c-40eb-abb5-962b9064b7ab,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "8424518d-87b7-4fc4-8a58-1b4f094a5ca0", "address": "fa:16:3e:08:61:83", "network": {"id": "9a452414-d61f-4045-a6cc-9be93c892c4c", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "321111f935dd4e8aab4a083206ea9a7e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8424518d-87", "ovs_interfaceid": "8424518d-87b7-4fc4-8a58-1b4f094a5ca0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:35:20 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Converting VIF {"id": "8424518d-87b7-4fc4-8a58-1b4f094a5ca0", "address": "fa:16:3e:08:61:83", "network": {"id": "9a452414-d61f-4045-a6cc-9be93c892c4c", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "321111f935dd4e8aab4a083206ea9a7e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8424518d-87", "ovs_interfaceid": "8424518d-87b7-4fc4-8a58-1b4f094a5ca0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:35:20 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:08:61:83,bridge_name='br-int',has_traffic_filtering=True,id=8424518d-87b7-4fc4-8a58-1b4f094a5ca0,network=Network(9a452414-d61f-4045-a6cc-9be93c892c4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8424518d-87') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:35:20 user nova-compute[72351]: DEBUG os_vif [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:08:61:83,bridge_name='br-int',has_traffic_filtering=True,id=8424518d-87b7-4fc4-8a58-1b4f094a5ca0,network=Network(9a452414-d61f-4045-a6cc-9be93c892c4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8424518d-87') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:35:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:35:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:35:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap8424518d-87, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:35:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap8424518d-87, col_values=(('external_ids', {'iface-id': '8424518d-87b7-4fc4-8a58-1b4f094a5ca0', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:08:61:83', 'vm-uuid': 'bde1ef1e-163c-40eb-abb5-962b9064b7ab'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:35:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:35:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:20 user nova-compute[72351]: INFO os_vif [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:08:61:83,bridge_name='br-int',has_traffic_filtering=True,id=8424518d-87b7-4fc4-8a58-1b4f094a5ca0,network=Network(9a452414-d61f-4045-a6cc-9be93c892c4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8424518d-87')
Aug 09 02:35:20 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:35:20 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] No VIF found with MAC fa:16:3e:08:61:83, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:35:20 user nova-compute[72351]: INFO nova.compute.manager [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Rebuilding instance
Aug 09 02:35:20 user nova-compute[72351]: DEBUG nova.objects.instance [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Lazy-loading 'trusted_certs' on Instance uuid ad9b389e-ea11-4fd7-9233-a1be43052e60 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:35:20 user nova-compute[72351]: DEBUG nova.compute.manager [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:35:20 user nova-compute[72351]: DEBUG nova.objects.instance [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Lazy-loading 'pci_requests' on Instance uuid ad9b389e-ea11-4fd7-9233-a1be43052e60 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:35:20 user nova-compute[72351]: DEBUG nova.objects.instance [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Lazy-loading 'pci_devices' on Instance uuid ad9b389e-ea11-4fd7-9233-a1be43052e60 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:35:20 user nova-compute[72351]: DEBUG nova.objects.instance [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Lazy-loading 'resources' on Instance uuid ad9b389e-ea11-4fd7-9233-a1be43052e60 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:35:20 user nova-compute[72351]: DEBUG nova.objects.instance [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Lazy-loading 'migration_context' on Instance uuid ad9b389e-ea11-4fd7-9233-a1be43052e60 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:35:20 user nova-compute[72351]: DEBUG nova.objects.instance [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Trying to apply a migration context that does not seem to be set for this instance {{(pid=72351) apply_migration_context /opt/stack/nova/nova/objects/instance.py:1067}}
Aug 09 02:35:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:21 user nova-compute[72351]: DEBUG nova.compute.manager [req-cb1b95c0-660a-45a3-9ad0-1a7dc76a568b req-4217c820-ff97-4e36-ba76-28db9de3ef68 service nova] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Received event network-vif-unplugged-64d8b550-334e-42c4-b57f-145ccc73517c {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:35:21 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-cb1b95c0-660a-45a3-9ad0-1a7dc76a568b req-4217c820-ff97-4e36-ba76-28db9de3ef68 service nova] Acquiring lock "ad9b389e-ea11-4fd7-9233-a1be43052e60-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:35:21 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-cb1b95c0-660a-45a3-9ad0-1a7dc76a568b req-4217c820-ff97-4e36-ba76-28db9de3ef68 service nova] Lock "ad9b389e-ea11-4fd7-9233-a1be43052e60-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:35:21 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-cb1b95c0-660a-45a3-9ad0-1a7dc76a568b req-4217c820-ff97-4e36-ba76-28db9de3ef68 service nova] Lock "ad9b389e-ea11-4fd7-9233-a1be43052e60-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:35:21 user nova-compute[72351]: DEBUG nova.compute.manager [req-cb1b95c0-660a-45a3-9ad0-1a7dc76a568b req-4217c820-ff97-4e36-ba76-28db9de3ef68 service nova] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] No waiting events found dispatching network-vif-unplugged-64d8b550-334e-42c4-b57f-145ccc73517c {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:35:21 user nova-compute[72351]: WARNING nova.compute.manager [req-cb1b95c0-660a-45a3-9ad0-1a7dc76a568b req-4217c820-ff97-4e36-ba76-28db9de3ef68 service nova] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Received unexpected event network-vif-unplugged-64d8b550-334e-42c4-b57f-145ccc73517c for instance with vm_state active and task_state rebuilding.
Aug 09 02:35:21 user nova-compute[72351]: DEBUG nova.compute.manager [req-cb1b95c0-660a-45a3-9ad0-1a7dc76a568b req-4217c820-ff97-4e36-ba76-28db9de3ef68 service nova] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Received event network-vif-plugged-64d8b550-334e-42c4-b57f-145ccc73517c {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:35:21 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-cb1b95c0-660a-45a3-9ad0-1a7dc76a568b req-4217c820-ff97-4e36-ba76-28db9de3ef68 service nova] Acquiring lock "ad9b389e-ea11-4fd7-9233-a1be43052e60-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:35:21 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-cb1b95c0-660a-45a3-9ad0-1a7dc76a568b req-4217c820-ff97-4e36-ba76-28db9de3ef68 service nova] Lock "ad9b389e-ea11-4fd7-9233-a1be43052e60-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:35:21 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-cb1b95c0-660a-45a3-9ad0-1a7dc76a568b req-4217c820-ff97-4e36-ba76-28db9de3ef68 service nova] Lock "ad9b389e-ea11-4fd7-9233-a1be43052e60-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:35:21 user nova-compute[72351]: DEBUG nova.compute.manager [req-cb1b95c0-660a-45a3-9ad0-1a7dc76a568b req-4217c820-ff97-4e36-ba76-28db9de3ef68 service nova] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] No waiting events found dispatching network-vif-plugged-64d8b550-334e-42c4-b57f-145ccc73517c {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:35:21 user nova-compute[72351]: WARNING nova.compute.manager [req-cb1b95c0-660a-45a3-9ad0-1a7dc76a568b req-4217c820-ff97-4e36-ba76-28db9de3ef68 service nova] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Received unexpected event network-vif-plugged-64d8b550-334e-42c4-b57f-145ccc73517c for instance with vm_state active and task_state rebuilding.
Aug 09 02:35:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:22 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Instance destroyed successfully.
Aug 09 02:35:22 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Instance destroyed successfully.
Aug 09 02:35:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:33:27Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-ServerActionsV293TestJSON-server-2038847230',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serveractionsv293testjson-server-657819557',id=42,image_ref='',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBEunnOgCeXfv3GBp09pSPyhwCL3D6Q1n187gHEnIiV9PA3lK8lSVc8G3tp5ZU9m6m0OxdZGYGPGwrp313iNid8wPNsg46CMpUwvg2Jv3lbvp6Ufeyxq5cu1dosjL2Qlv6A==',key_name='tempest-keypair-461212954',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:33:43Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={rebuild='server'},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='624e34ea80f944509fde2cb5472ed203',ramdisk_id='',reservation_id='r-wzsh0h7v',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerActionsV293TestJSON-475025801',owner_user_name='tempest-ServerActionsV293TestJSON-475025801-project-member'},tags=<?>,task_state='rebuilding',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:35:21Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='801c60b1fab94f38b4d40222e9946f89',uuid=ad9b389e-ea11-4fd7-9233-a1be43052e60,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "64d8b550-334e-42c4-b57f-145ccc73517c", "address": "fa:16:3e:52:34:4d", "network": {"id": "f7042dd1-90f3-4ee1-99cd-e52f9264dc87", "bridge": "br-int", "label": "tempest-ServerActionsV293TestJSON-1595239201-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.209", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "624e34ea80f944509fde2cb5472ed203", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap64d8b550-33", "ovs_interfaceid": "64d8b550-334e-42c4-b57f-145ccc73517c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Converting VIF {"id": "64d8b550-334e-42c4-b57f-145ccc73517c", "address": "fa:16:3e:52:34:4d", "network": {"id": "f7042dd1-90f3-4ee1-99cd-e52f9264dc87", "bridge": "br-int", "label": "tempest-ServerActionsV293TestJSON-1595239201-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.209", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "624e34ea80f944509fde2cb5472ed203", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap64d8b550-33", "ovs_interfaceid": "64d8b550-334e-42c4-b57f-145ccc73517c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:52:34:4d,bridge_name='br-int',has_traffic_filtering=True,id=64d8b550-334e-42c4-b57f-145ccc73517c,network=Network(f7042dd1-90f3-4ee1-99cd-e52f9264dc87),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap64d8b550-33') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG os_vif [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:52:34:4d,bridge_name='br-int',has_traffic_filtering=True,id=64d8b550-334e-42c4-b57f-145ccc73517c,network=Network(f7042dd1-90f3-4ee1-99cd-e52f9264dc87),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap64d8b550-33') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap64d8b550-33, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:22 user nova-compute[72351]: INFO os_vif [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:52:34:4d,bridge_name='br-int',has_traffic_filtering=True,id=64d8b550-334e-42c4-b57f-145ccc73517c,network=Network(f7042dd1-90f3-4ee1-99cd-e52f9264dc87),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap64d8b550-33')
Aug 09 02:35:22 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'arRsVtvH', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sde'}, None), 'kwargs': True}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:35:22 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release: NotImplementedError
Aug 09 02:35:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[86be5b62-6ffb-4fb6-b31c-7d64c98f425b]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.035s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[1b3b09c0-8b4c-447b-9a85-8f1efcceeacd]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)\ntcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)\ntcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)\ntcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)\ntcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:35:22 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:35:22 user nova-compute[72351]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)
Aug 09 02:35:22 user nova-compute[72351]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:35:22 user nova-compute[72351]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)
Aug 09 02:35:22 user nova-compute[72351]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)
Aug 09 02:35:22 user nova-compute[72351]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df (non-flash)
Aug 09 02:35:22 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:35:22 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:35:22 user nova-compute[72351]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)
Aug 09 02:35:22 user nova-compute[72351]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:35:22 user nova-compute[72351]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)
Aug 09 02:35:22 user nova-compute[72351]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)
Aug 09 02:35:22 user nova-compute[72351]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df (non-flash)
Aug 09 02:35:22 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fcfdc1dce50>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d'): ({'sde'}, set())}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Removing single pathed devices sde {{(pid=72351) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[90ec05dc-c247-4ec5-8f26-323ce69aaadf]: (4, ('path checker states:\nup                  7\n\npaths: 0\nbusy: False\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sde {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sde" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[873cbe71-23d6-4563-b06a-9e8dcbbf9f33]: (4, ('ok\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Flushing IO for device /dev/sde {{(pid=72351) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sde {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sde" returned: 0 in 0.034s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[dd534fa1-d68b-45c5-84bd-ddc152cfb7c2]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Remove SCSI device /dev/sde with /sys/block/sde/device/delete {{(pid=72351) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sde/device/delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG nova.compute.manager [req-5f2f193d-c876-429f-919b-1381985081f0 req-6f5e6a4d-d86a-41d6-933e-25b22e58181a service nova] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Received event network-vif-plugged-8424518d-87b7-4fc4-8a58-1b4f094a5ca0 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-5f2f193d-c876-429f-919b-1381985081f0 req-6f5e6a4d-d86a-41d6-933e-25b22e58181a service nova] Acquiring lock "bde1ef1e-163c-40eb-abb5-962b9064b7ab-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-5f2f193d-c876-429f-919b-1381985081f0 req-6f5e6a4d-d86a-41d6-933e-25b22e58181a service nova] Lock "bde1ef1e-163c-40eb-abb5-962b9064b7ab-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-5f2f193d-c876-429f-919b-1381985081f0 req-6f5e6a4d-d86a-41d6-933e-25b22e58181a service nova] Lock "bde1ef1e-163c-40eb-abb5-962b9064b7ab-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG nova.compute.manager [req-5f2f193d-c876-429f-919b-1381985081f0 req-6f5e6a4d-d86a-41d6-933e-25b22e58181a service nova] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] No waiting events found dispatching network-vif-plugged-8424518d-87b7-4fc4-8a58-1b4f094a5ca0 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:35:22 user nova-compute[72351]: WARNING nova.compute.manager [req-5f2f193d-c876-429f-919b-1381985081f0 req-6f5e6a4d-d86a-41d6-933e-25b22e58181a service nova] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Received unexpected event network-vif-plugged-8424518d-87b7-4fc4-8a58-1b4f094a5ca0 for instance with vm_state building and task_state spawning.
Aug 09 02:35:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sde/device/delete" returned: 0 in 0.049s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[86aa3360-abdc-4bc5-b3fc-7d61803b449c]: (4, ('1', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Checking to see if SCSI volumes sde have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] SCSI volumes sde have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d')] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[330e8e93-961e-46c7-a72d-9e7544459d30]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d -p 172.16.0.220:3260 --logout {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d -p 172.16.0.220:3260 --logout" returned: 0 in 0.104s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a2142ca8-15d4-4618-b309-ae79ce9e6cf8]: (4, ('Logging out of session [sid: 44, target: iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d, portal: 172.16.0.220,3260]\nLogout of [sid: 44, target: iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 44, target: iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d, portal: 172.16.0.220,3260]
Aug 09 02:35:22 user nova-compute[72351]: Logout of [sid: 44, target: iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d, portal: 172.16.0.220,3260] successful.
Aug 09 02:35:22 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d -p 172.16.0.220:3260 --op delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d -p 172.16.0.220:3260 --op delete" returned: 0 in 0.046s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[30b8c963-2c0e-4038-b1c8-f419811bd82a]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.425s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] <== disconnect_volume: return (427ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:35:22 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 02:35:22 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Deleting instance files /opt/stack/data/nova/instances/ad9b389e-ea11-4fd7-9233-a1be43052e60_del
Aug 09 02:35:22 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Deletion of /opt/stack/data/nova/instances/ad9b389e-ea11-4fd7-9233-a1be43052e60_del complete
Aug 09 02:35:23 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] During detach_volume, instance disappeared.: nova.exception.InstanceNotFound: Instance ad9b389e-ea11-4fd7-9233-a1be43052e60 could not be found.
Aug 09 02:35:23 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 02:35:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'arRsVtvH', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sde'}, None), 'kwargs': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:35:23 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release: NotImplementedError
Aug 09 02:35:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:35:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:35:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 02:35:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:23 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[0b3e310a-5fdd-40ff-a865-cce8fd066507]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.026s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:23 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[b994d197-49a8-4b32-a55c-cc864de5b0e0]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)\ntcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)\ntcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)\ntcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:35:23 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:35:23 user nova-compute[72351]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)
Aug 09 02:35:23 user nova-compute[72351]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:35:23 user nova-compute[72351]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)
Aug 09 02:35:23 user nova-compute[72351]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df (non-flash)
Aug 09 02:35:23 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:35:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:35:23 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:35:23 user nova-compute[72351]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)
Aug 09 02:35:23 user nova-compute[72351]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:35:23 user nova-compute[72351]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)
Aug 09 02:35:23 user nova-compute[72351]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df (non-flash)
Aug 09 02:35:23 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:35:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fcfdc1dcc10>, {}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 02:35:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Disconnecting from: [] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 02:35:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.067s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:35:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] <== disconnect_volume: return (68ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:35:23 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 02:35:23 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:23 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:24 user nova-compute[72351]: DEBUG nova.compute.manager [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Preparing to wait for external event volume-reimaged-75a3424d-ab51-409b-b1e1-f702f4a84d3d {{(pid=72351) prepare_for_instance_event /opt/stack/nova/nova/compute/manager.py:283}}
Aug 09 02:35:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Acquiring lock "ad9b389e-ea11-4fd7-9233-a1be43052e60-events" by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.<locals>._create_or_get_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:35:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Lock "ad9b389e-ea11-4fd7-9233-a1be43052e60-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.<locals>._create_or_get_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:35:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Lock "ad9b389e-ea11-4fd7-9233-a1be43052e60-events" "released" by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.<locals>._create_or_get_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:35:24 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548524.5454164, bde1ef1e-163c-40eb-abb5-962b9064b7ab => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:35:24 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] VM Resumed (Lifecycle Event)
Aug 09 02:35:24 user nova-compute[72351]: DEBUG nova.compute.manager [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:35:24 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Guest created on hypervisor {{(pid=72351) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4411}}
Aug 09 02:35:24 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Instance spawned successfully.
Aug 09 02:35:24 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:894}}
Aug 09 02:35:24 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:35:24 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:35:24 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Found default for hw_cdrom_bus of ide {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:35:24 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Found default for hw_disk_bus of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:35:24 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Found default for hw_input_bus of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:35:24 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Found default for hw_pointer_model of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:35:24 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Found default for hw_video_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:35:24 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Found default for hw_vif_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:35:24 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:35:24 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548524.5491223, bde1ef1e-163c-40eb-abb5-962b9064b7ab => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:35:24 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] VM Started (Lifecycle Event)
Aug 09 02:35:24 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:35:24 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:35:24 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:35:24 user nova-compute[72351]: DEBUG nova.compute.manager [req-6d6d5ba3-089b-4ec7-bf3c-659ea9685eb1 req-6a941f8a-c02a-4604-9791-12423f05262f service nova] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Received event network-vif-plugged-8424518d-87b7-4fc4-8a58-1b4f094a5ca0 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:35:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-6d6d5ba3-089b-4ec7-bf3c-659ea9685eb1 req-6a941f8a-c02a-4604-9791-12423f05262f service nova] Acquiring lock "bde1ef1e-163c-40eb-abb5-962b9064b7ab-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:35:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-6d6d5ba3-089b-4ec7-bf3c-659ea9685eb1 req-6a941f8a-c02a-4604-9791-12423f05262f service nova] Lock "bde1ef1e-163c-40eb-abb5-962b9064b7ab-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:35:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-6d6d5ba3-089b-4ec7-bf3c-659ea9685eb1 req-6a941f8a-c02a-4604-9791-12423f05262f service nova] Lock "bde1ef1e-163c-40eb-abb5-962b9064b7ab-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:35:24 user nova-compute[72351]: DEBUG nova.compute.manager [req-6d6d5ba3-089b-4ec7-bf3c-659ea9685eb1 req-6a941f8a-c02a-4604-9791-12423f05262f service nova] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] No waiting events found dispatching network-vif-plugged-8424518d-87b7-4fc4-8a58-1b4f094a5ca0 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:35:24 user nova-compute[72351]: WARNING nova.compute.manager [req-6d6d5ba3-089b-4ec7-bf3c-659ea9685eb1 req-6a941f8a-c02a-4604-9791-12423f05262f service nova] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Received unexpected event network-vif-plugged-8424518d-87b7-4fc4-8a58-1b4f094a5ca0 for instance with vm_state building and task_state spawning.
Aug 09 02:35:24 user nova-compute[72351]: INFO nova.compute.manager [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Took 5.76 seconds to spawn the instance on the hypervisor.
Aug 09 02:35:24 user nova-compute[72351]: DEBUG nova.compute.manager [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:35:24 user nova-compute[72351]: INFO nova.compute.manager [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Took 27.66 seconds to build instance.
Aug 09 02:35:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f9f5430d-e5ba-4ad2-8426-1da5143a8423 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "bde1ef1e-163c-40eb-abb5-962b9064b7ab" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 27.763s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:35:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:28 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:32 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:33 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:37 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691548522.3319883, ad9b389e-ea11-4fd7-9233-a1be43052e60 => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:35:37 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] VM Stopped (Lifecycle Event)
Aug 09 02:35:37 user nova-compute[72351]: DEBUG nova.compute.manager [None req-efd75ceb-a46a-4e99-8461-cdde3e5d15a7 None None] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:35:37 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:38 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:41 user nova-compute[72351]: DEBUG nova.compute.manager [req-478b0d03-7ccd-4e50-b342-ed353bd33c96 req-42b31578-a043-4d23-9a82-847e77881566 service nova] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Received event network-changed-2dcfe7b8-34fe-4860-b820-f3de7c00df62 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:35:41 user nova-compute[72351]: DEBUG nova.compute.manager [req-478b0d03-7ccd-4e50-b342-ed353bd33c96 req-42b31578-a043-4d23-9a82-847e77881566 service nova] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Refreshing instance network info cache due to event network-changed-2dcfe7b8-34fe-4860-b820-f3de7c00df62. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:35:41 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-478b0d03-7ccd-4e50-b342-ed353bd33c96 req-42b31578-a043-4d23-9a82-847e77881566 service nova] Acquiring lock "refresh_cache-49a10075-0d3c-4cd6-bb43-10238cad3f37" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:35:41 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-478b0d03-7ccd-4e50-b342-ed353bd33c96 req-42b31578-a043-4d23-9a82-847e77881566 service nova] Acquired lock "refresh_cache-49a10075-0d3c-4cd6-bb43-10238cad3f37" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:35:41 user nova-compute[72351]: DEBUG nova.network.neutron [req-478b0d03-7ccd-4e50-b342-ed353bd33c96 req-42b31578-a043-4d23-9a82-847e77881566 service nova] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Refreshing network info cache for port 2dcfe7b8-34fe-4860-b820-f3de7c00df62 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:35:41 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-8175e910-d6bf-444c-ba81-0ddb1b0cfec0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquiring lock "49a10075-0d3c-4cd6-bb43-10238cad3f37" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:35:41 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-8175e910-d6bf-444c-ba81-0ddb1b0cfec0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "49a10075-0d3c-4cd6-bb43-10238cad3f37" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:35:41 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-8175e910-d6bf-444c-ba81-0ddb1b0cfec0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquiring lock "49a10075-0d3c-4cd6-bb43-10238cad3f37-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:35:41 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-8175e910-d6bf-444c-ba81-0ddb1b0cfec0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "49a10075-0d3c-4cd6-bb43-10238cad3f37-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:35:41 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-8175e910-d6bf-444c-ba81-0ddb1b0cfec0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "49a10075-0d3c-4cd6-bb43-10238cad3f37-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:35:41 user nova-compute[72351]: INFO nova.compute.manager [None req-8175e910-d6bf-444c-ba81-0ddb1b0cfec0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Terminating instance
Aug 09 02:35:41 user nova-compute[72351]: DEBUG nova.compute.manager [None req-8175e910-d6bf-444c-ba81-0ddb1b0cfec0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Start destroying the instance on the hypervisor. {{(pid=72351) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3124}}
Aug 09 02:35:41 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:41 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:41 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:41 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:41 user nova-compute[72351]: DEBUG nova.compute.manager [req-a0d6e475-31fe-41a2-9e74-0b1cac5d4686 req-79de3a17-713e-482e-bd7f-5bf5f8588b63 service nova] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Received event network-vif-unplugged-2dcfe7b8-34fe-4860-b820-f3de7c00df62 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:35:41 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a0d6e475-31fe-41a2-9e74-0b1cac5d4686 req-79de3a17-713e-482e-bd7f-5bf5f8588b63 service nova] Acquiring lock "49a10075-0d3c-4cd6-bb43-10238cad3f37-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:35:41 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a0d6e475-31fe-41a2-9e74-0b1cac5d4686 req-79de3a17-713e-482e-bd7f-5bf5f8588b63 service nova] Lock "49a10075-0d3c-4cd6-bb43-10238cad3f37-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:35:41 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a0d6e475-31fe-41a2-9e74-0b1cac5d4686 req-79de3a17-713e-482e-bd7f-5bf5f8588b63 service nova] Lock "49a10075-0d3c-4cd6-bb43-10238cad3f37-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:35:41 user nova-compute[72351]: DEBUG nova.compute.manager [req-a0d6e475-31fe-41a2-9e74-0b1cac5d4686 req-79de3a17-713e-482e-bd7f-5bf5f8588b63 service nova] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] No waiting events found dispatching network-vif-unplugged-2dcfe7b8-34fe-4860-b820-f3de7c00df62 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:35:41 user nova-compute[72351]: DEBUG nova.compute.manager [req-a0d6e475-31fe-41a2-9e74-0b1cac5d4686 req-79de3a17-713e-482e-bd7f-5bf5f8588b63 service nova] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Received event network-vif-unplugged-2dcfe7b8-34fe-4860-b820-f3de7c00df62 for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:35:41 user nova-compute[72351]: DEBUG nova.network.neutron [req-478b0d03-7ccd-4e50-b342-ed353bd33c96 req-42b31578-a043-4d23-9a82-847e77881566 service nova] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Updated VIF entry in instance network info cache for port 2dcfe7b8-34fe-4860-b820-f3de7c00df62. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:35:41 user nova-compute[72351]: DEBUG nova.network.neutron [req-478b0d03-7ccd-4e50-b342-ed353bd33c96 req-42b31578-a043-4d23-9a82-847e77881566 service nova] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Updating instance_info_cache with network_info: [{"id": "2dcfe7b8-34fe-4860-b820-f3de7c00df62", "address": "fa:16:3e:01:0f:c9", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2dcfe7b8-34", "ovs_interfaceid": "2dcfe7b8-34fe-4860-b820-f3de7c00df62", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:35:41 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-478b0d03-7ccd-4e50-b342-ed353bd33c96 req-42b31578-a043-4d23-9a82-847e77881566 service nova] Releasing lock "refresh_cache-49a10075-0d3c-4cd6-bb43-10238cad3f37" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:35:42 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Instance destroyed successfully.
Aug 09 02:35:42 user nova-compute[72351]: DEBUG nova.objects.instance [None req-8175e910-d6bf-444c-ba81-0ddb1b0cfec0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lazy-loading 'resources' on Instance uuid 49a10075-0d3c-4cd6-bb43-10238cad3f37 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-8175e910-d6bf-444c-ba81-0ddb1b0cfec0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:30:37Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-server-1123873481',display_name='tempest-TestVolumeBootPattern-server-1123873481',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testvolumebootpattern-server-1123873481',id=35,image_ref='',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBM+1oHdu/+hmVTCW73FChAt+AvG2bL++YbEuGYMvrUaUYAyZQrt9Vwv7j34dhFmoVuCI0Rk18KgJ1X07X8pPcTOnOW0XFALfY2BbRHpJcmwARDVbYh8uPoNVxqSRoIR2gA==',key_name='tempest-TestVolumeBootPattern-805107831',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:31:05Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='a4c6bbfd8d904ba1ac69ba5da4564513',ramdisk_id='',reservation_id='r-tfy77r2o',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TestVolumeBootPattern-350743460',owner_user_name='tempest-TestVolumeBootPattern-350743460-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:31:05Z,user_data=None,user_id='07d8d5a634274df19c2db769d2d27b29',uuid=49a10075-0d3c-4cd6-bb43-10238cad3f37,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "2dcfe7b8-34fe-4860-b820-f3de7c00df62", "address": "fa:16:3e:01:0f:c9", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.21", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2dcfe7b8-34", "ovs_interfaceid": "2dcfe7b8-34fe-4860-b820-f3de7c00df62", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-8175e910-d6bf-444c-ba81-0ddb1b0cfec0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Converting VIF {"id": "2dcfe7b8-34fe-4860-b820-f3de7c00df62", "address": "fa:16:3e:01:0f:c9", "network": {"id": "5ef428cf-e408-490c-8f60-70934f760f4c", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1319785616-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.21", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a4c6bbfd8d904ba1ac69ba5da4564513", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2dcfe7b8-34", "ovs_interfaceid": "2dcfe7b8-34fe-4860-b820-f3de7c00df62", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-8175e910-d6bf-444c-ba81-0ddb1b0cfec0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:01:0f:c9,bridge_name='br-int',has_traffic_filtering=True,id=2dcfe7b8-34fe-4860-b820-f3de7c00df62,network=Network(5ef428cf-e408-490c-8f60-70934f760f4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2dcfe7b8-34') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG os_vif [None req-8175e910-d6bf-444c-ba81-0ddb1b0cfec0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:01:0f:c9,bridge_name='br-int',has_traffic_filtering=True,id=2dcfe7b8-34fe-4860-b820-f3de7c00df62,network=Network(5ef428cf-e408-490c-8f60-70934f760f4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2dcfe7b8-34') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap2dcfe7b8-34, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:35:42 user nova-compute[72351]: INFO os_vif [None req-8175e910-d6bf-444c-ba81-0ddb1b0cfec0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:01:0f:c9,bridge_name='br-int',has_traffic_filtering=True,id=2dcfe7b8-34fe-4860-b820-f3de7c00df62,network=Network(5ef428cf-e408-490c-8f60-70934f760f4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2dcfe7b8-34')
Aug 09 02:35:42 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-8175e910-d6bf-444c-ba81-0ddb1b0cfec0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-8175e910-d6bf-444c-ba81-0ddb1b0cfec0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'CdzJhfwW', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdf'}, None), 'kwargs': True}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:35:42 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-8175e910-d6bf-444c-ba81-0ddb1b0cfec0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:35:42 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-8175e910-d6bf-444c-ba81-0ddb1b0cfec0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-8175e910-d6bf-444c-ba81-0ddb1b0cfec0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-8175e910-d6bf-444c-ba81-0ddb1b0cfec0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[f3ba62a0-2938-4545-b9f9-34afc98ad33d]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.030s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[55d8f1f1-e37a-4f82-a78c-666bf89252ad]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)\ntcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)\ntcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)\ntcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-8175e910-d6bf-444c-ba81-0ddb1b0cfec0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:35:42 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:35:42 user nova-compute[72351]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)
Aug 09 02:35:42 user nova-compute[72351]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:35:42 user nova-compute[72351]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)
Aug 09 02:35:42 user nova-compute[72351]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df (non-flash)
Aug 09 02:35:42 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-8175e910-d6bf-444c-ba81-0ddb1b0cfec0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:35:42 user nova-compute[72351]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 (non-flash)
Aug 09 02:35:42 user nova-compute[72351]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)
Aug 09 02:35:42 user nova-compute[72351]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:35:42 user nova-compute[72351]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)
Aug 09 02:35:42 user nova-compute[72351]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df (non-flash)
Aug 09 02:35:42 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-8175e910-d6bf-444c-ba81-0ddb1b0cfec0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fcfbc63c280>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448'): ({'sdf'}, set())}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-8175e910-d6bf-444c-ba81-0ddb1b0cfec0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Removing single pathed devices sdf {{(pid=72351) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.032s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[45ed6d69-3563-44fd-bf54-e6a3fbeed7b1]: (4, ('path checker states:\nup                  6\n\npaths: 0\nbusy: False\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdf {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdf" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[b533c9fe-c2e2-4e4d-9dba-a7018e7ba624]: (4, ('ok\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-8175e910-d6bf-444c-ba81-0ddb1b0cfec0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Flushing IO for device /dev/sdf {{(pid=72351) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdf {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdf" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e26e40ef-8264-44f0-8f04-b23b7d71e1b8]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-8175e910-d6bf-444c-ba81-0ddb1b0cfec0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Remove SCSI device /dev/sdf with /sys/block/sdf/device/delete {{(pid=72351) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdf/device/delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdf/device/delete" returned: 0 in 0.044s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[3f8c0089-ddf4-4e43-a14b-2956e2b6cf42]: (4, ('1', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-8175e910-d6bf-444c-ba81-0ddb1b0cfec0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Checking to see if SCSI volumes sdf have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-8175e910-d6bf-444c-ba81-0ddb1b0cfec0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] SCSI volumes sdf have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG os_brick.privileged.rootwrap [-] Unlinking ('/dev/disk/by-id/scsi-SSCST_BIO_Storage_26913bcd', '/dev/disk/by-id/scsi-23931306338316130', '/dev/disk/by-id/scsi-1SCST_BIO26913bcd-910c81a04590072a') {{(pid=83366) unlink_root /usr/local/lib/python3.10/dist-packages/os_brick/privileged/rootwrap.py:219}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[23847cd6-3bf7-42dc-8381-3f7bb2da7f10]: (4, None) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-8175e910-d6bf-444c-ba81-0ddb1b0cfec0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448')] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[32a3e9e6-0c20-494b-9b9d-927880461442]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-8175e910-d6bf-444c-ba81-0ddb1b0cfec0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 -p 172.16.0.220:3260 --logout {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 -p 172.16.0.220:3260 --logout" returned: 0 in 0.084s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[cfb0d8f3-722a-4ea9-a31e-83e8dbbb4e61]: (4, ('Logging out of session [sid: 37, target: iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448, portal: 172.16.0.220,3260]\nLogout of [sid: 37, target: iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-8175e910-d6bf-444c-ba81-0ddb1b0cfec0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 37, target: iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448, portal: 172.16.0.220,3260]
Aug 09 02:35:42 user nova-compute[72351]: Logout of [sid: 37, target: iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448, portal: 172.16.0.220,3260] successful.
Aug 09 02:35:42 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 -p 172.16.0.220:3260 --op delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4113a563-77a5-4b68-9198-9d0434737448 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.038s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d9a5dbc9-0654-4bd3-b927-2ad0f528350f]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-8175e910-d6bf-444c-ba81-0ddb1b0cfec0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-8175e910-d6bf-444c-ba81-0ddb1b0cfec0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.362s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-8175e910-d6bf-444c-ba81-0ddb1b0cfec0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] <== disconnect_volume: return (364ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-8175e910-d6bf-444c-ba81-0ddb1b0cfec0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 02:35:42 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-8175e910-d6bf-444c-ba81-0ddb1b0cfec0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Deleting instance files /opt/stack/data/nova/instances/49a10075-0d3c-4cd6-bb43-10238cad3f37_del
Aug 09 02:35:42 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-8175e910-d6bf-444c-ba81-0ddb1b0cfec0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Deletion of /opt/stack/data/nova/instances/49a10075-0d3c-4cd6-bb43-10238cad3f37_del complete
Aug 09 02:35:42 user nova-compute[72351]: DEBUG nova.compute.manager [req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 req-8790d50b-d782-46f9-98f0-211688702cad service nova] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Received event volume-reimaged-75a3424d-ab51-409b-b1e1-f702f4a84d3d {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 req-8790d50b-d782-46f9-98f0-211688702cad service nova] Acquiring lock "ad9b389e-ea11-4fd7-9233-a1be43052e60-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 req-8790d50b-d782-46f9-98f0-211688702cad service nova] Lock "ad9b389e-ea11-4fd7-9233-a1be43052e60-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 req-8790d50b-d782-46f9-98f0-211688702cad service nova] Lock "ad9b389e-ea11-4fd7-9233-a1be43052e60-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG nova.compute.manager [req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 req-8790d50b-d782-46f9-98f0-211688702cad service nova] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Processing event volume-reimaged-75a3424d-ab51-409b-b1e1-f702f4a84d3d {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10792}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG nova.compute.manager [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Instance event wait completed in 18 seconds for volume-reimaged {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:35:42 user nova-compute[72351]: INFO nova.compute.manager [None req-8175e910-d6bf-444c-ba81-0ddb1b0cfec0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Took 1.26 seconds to destroy the instance on the hypervisor.
Aug 09 02:35:42 user nova-compute[72351]: DEBUG oslo.service.loopingcall [None req-8175e910-d6bf-444c-ba81-0ddb1b0cfec0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72351) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG nova.compute.manager [-] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Deallocating network for instance {{(pid=72351) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2263}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] deallocate_for_instance() {{(pid=72351) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1795}}
Aug 09 02:35:42 user nova-compute[72351]: INFO nova.virt.block_device [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Booting with volume 75a3424d-ab51-409b-b1e1-f702f4a84d3d at /dev/vda
Aug 09 02:35:42 user nova-compute[72351]: DEBUG os_brick.utils [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ead4d31c-0ece-4069-a0eb-9d0e31432557]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:a52269602ff4\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d1ac6d0f-ed88-4059-a37c-52a950ea8e81]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.016s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[b94283e9-f216-475c-b461-18439e00f415]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e7bebe9b-da2a-4773-9930-f12eea13ba41]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Running cmd (subprocess): nvme version {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] 'nvme version' failed. Not Retrying. {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.nvmeof [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] nvme not present on system {{(pid=72351) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:35:42 user nova-compute[72351]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Aug 09 02:35:42 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[501d7f13-a55f-41b6-b924-78ed61f96c60]: (4, '') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72351) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] LIGHTOS: no hostnqn found. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG os_brick.utils [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] <== get_connector_properties: return (127ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:a52269602ff4', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:35:42 user nova-compute[72351]: DEBUG nova.virt.block_device [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Updating existing volume attachment record: 204ad121-a9fb-436b-a519-df7d39df12ec {{(pid=72351) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Aug 09 02:35:43 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:35:43 user nova-compute[72351]: DEBUG nova.compute.manager [req-d3daf24b-700a-45e3-8108-8a2d82d70a2a req-ded5199e-ca15-445c-b98c-2615795f2034 service nova] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Received event network-vif-deleted-2dcfe7b8-34fe-4860-b820-f3de7c00df62 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:35:43 user nova-compute[72351]: INFO nova.compute.manager [req-d3daf24b-700a-45e3-8108-8a2d82d70a2a req-ded5199e-ca15-445c-b98c-2615795f2034 service nova] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Neutron deleted interface 2dcfe7b8-34fe-4860-b820-f3de7c00df62; detaching it from the instance and deleting it from the info cache
Aug 09 02:35:43 user nova-compute[72351]: DEBUG nova.network.neutron [req-d3daf24b-700a-45e3-8108-8a2d82d70a2a req-ded5199e-ca15-445c-b98c-2615795f2034 service nova] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:35:43 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Took 0.58 seconds to deallocate network for instance.
Aug 09 02:35:43 user nova-compute[72351]: DEBUG nova.compute.manager [req-d3daf24b-700a-45e3-8108-8a2d82d70a2a req-ded5199e-ca15-445c-b98c-2615795f2034 service nova] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Detach interface failed, port_id=2dcfe7b8-34fe-4860-b820-f3de7c00df62, reason: Instance 49a10075-0d3c-4cd6-bb43-10238cad3f37 could not be found. {{(pid=72351) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10866}}
Aug 09 02:35:43 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:43 user nova-compute[72351]: DEBUG nova.compute.manager [req-9b5d6107-c6ae-4850-ad77-d9488e8b7f05 req-ab6a01cb-40d7-4806-acf0-63368fd1e27d service nova] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Received event network-vif-plugged-2dcfe7b8-34fe-4860-b820-f3de7c00df62 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:35:43 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-9b5d6107-c6ae-4850-ad77-d9488e8b7f05 req-ab6a01cb-40d7-4806-acf0-63368fd1e27d service nova] Acquiring lock "49a10075-0d3c-4cd6-bb43-10238cad3f37-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:35:43 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-9b5d6107-c6ae-4850-ad77-d9488e8b7f05 req-ab6a01cb-40d7-4806-acf0-63368fd1e27d service nova] Lock "49a10075-0d3c-4cd6-bb43-10238cad3f37-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:35:43 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-9b5d6107-c6ae-4850-ad77-d9488e8b7f05 req-ab6a01cb-40d7-4806-acf0-63368fd1e27d service nova] Lock "49a10075-0d3c-4cd6-bb43-10238cad3f37-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:35:43 user nova-compute[72351]: DEBUG nova.compute.manager [req-9b5d6107-c6ae-4850-ad77-d9488e8b7f05 req-ab6a01cb-40d7-4806-acf0-63368fd1e27d service nova] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] No waiting events found dispatching network-vif-plugged-2dcfe7b8-34fe-4860-b820-f3de7c00df62 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:35:43 user nova-compute[72351]: WARNING nova.compute.manager [req-9b5d6107-c6ae-4850-ad77-d9488e8b7f05 req-ab6a01cb-40d7-4806-acf0-63368fd1e27d service nova] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Received unexpected event network-vif-plugged-2dcfe7b8-34fe-4860-b820-f3de7c00df62 for instance with vm_state active and task_state deleting.
Aug 09 02:35:44 user nova-compute[72351]: INFO nova.compute.manager [None req-8175e910-d6bf-444c-ba81-0ddb1b0cfec0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Took 0.77 seconds to detach 1 volumes for instance.
Aug 09 02:35:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-8175e910-d6bf-444c-ba81-0ddb1b0cfec0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:35:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-8175e910-d6bf-444c-ba81-0ddb1b0cfec0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:35:44 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-8175e910-d6bf-444c-ba81-0ddb1b0cfec0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:35:44 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-8175e910-d6bf-444c-ba81-0ddb1b0cfec0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:35:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-8175e910-d6bf-444c-ba81-0ddb1b0cfec0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.410s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:35:44 user nova-compute[72351]: INFO nova.scheduler.client.report [None req-8175e910-d6bf-444c-ba81-0ddb1b0cfec0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Deleted allocations for instance 49a10075-0d3c-4cd6-bb43-10238cad3f37
Aug 09 02:35:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-8175e910-d6bf-444c-ba81-0ddb1b0cfec0 tempest-TestVolumeBootPattern-350743460 tempest-TestVolumeBootPattern-350743460-project-member] Lock "49a10075-0d3c-4cd6-bb43-10238cad3f37" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 3.222s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:35:47 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:48 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:51 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Creating instance directory {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4717}}
Aug 09 02:35:51 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Creating image(s)
Aug 09 02:35:51 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Did not create local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4853}}
Aug 09 02:35:51 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Ensure instance console log exists: /opt/stack/data/nova/instances/ad9b389e-ea11-4fd7-9233-a1be43052e60/console.log {{(pid=72351) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4603}}
Aug 09 02:35:51 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:35:51 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:35:51 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:35:51 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Start _get_guest_xml network_info=[{"id": "64d8b550-334e-42c4-b57f-145ccc73517c", "address": "fa:16:3e:52:34:4d", "network": {"id": "f7042dd1-90f3-4ee1-99cd-e52f9264dc87", "bridge": "br-int", "label": "tempest-ServerActionsV293TestJSON-1595239201-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.209", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "624e34ea80f944509fde2cb5472ed203", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap64d8b550-33", "ovs_interfaceid": "64d8b550-334e-42c4-b57f-145ccc73517c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'device_type': 'disk', 'guest_format': None, 'disk_bus': 'virtio', 'mount_device': '/dev/vda', 'boot_index': 0, 'attachment_id': '204ad121-a9fb-436b-a519-df7d39df12ec', 'delete_on_termination': True, 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqns': ['iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'arRsVtvH', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': 'ad9b389e-ea11-4fd7-9233-a1be43052e60', 'attached_at': '', 'detached_at': '', 'volume_id': '75a3424d-ab51-409b-b1e1-f702f4a84d3d', 'serial': '75a3424d-ab51-409b-b1e1-f702f4a84d3d'}, 'volume_type': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:35:51 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.: NotImplementedError
Aug 09 02:35:51 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.: NotImplementedError
Aug 09 02:35:51 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:35:51 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:35:51 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:35:51 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:35:51 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:35:51 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:35:51 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:35:51 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:35:51 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:35:51 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:35:51 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:35:51 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:35:51 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:35:51 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:35:51 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:35:51 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:35:51 user nova-compute[72351]: DEBUG nova.objects.instance [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Lazy-loading 'vcpu_model' on Instance uuid ad9b389e-ea11-4fd7-9233-a1be43052e60 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:35:51 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 02:35:51 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'arRsVtvH', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:35:51 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release: NotImplementedError
Aug 09 02:35:51 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:35:51 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:35:51 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release: NotImplementedError
Aug 09 02:35:51 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:35:51 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:35:51 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 02:35:51 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:51 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d -p 172.16.0.220:3260" returned: 21 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:51 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[f8c1af8c-cc66-4fa0-bacc-fe8eb0015de8]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:51 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Aug 09 02:35:51 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:35:51 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d -p 172.16.0.220:3260 --interface default --op new {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:51 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:51 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[3634e577-38b4-48e0-81fc-bb1d8ef25443]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d] added\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:51 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d] added
Aug 09 02:35:51 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:35:51 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:51 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:51 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[865bb072-8dd9-496a-96bc-76d5c7a7c808]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:51 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:35:51 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:51 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:51 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d874aef6-bfb9-4389-8149-83fa6bf8a9bf]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:51 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:35:51 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d -p 172.16.0.220:3260 --op update -n node.session.auth.username -v arRsVtvH {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:51 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d -p 172.16.0.220:3260 --op update -n node.session.auth.username -v arRsVtvH" returned: 0 in 0.024s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:51 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d1956848-b265-4f8e-ad1d-2e24dd17be35]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:51 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'arRsVtvH'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:35:51 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:51 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:51 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e7b07782-937c-41ef-9342-e4f4e4c48e55]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:51 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:35:51 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.036s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:52 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[87b66fe7-14ea-47dd-9dc6-5fcc94180b71]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)\ntcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)\ntcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)\ntcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:52 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:35:52 user nova-compute[72351]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)
Aug 09 02:35:52 user nova-compute[72351]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:35:52 user nova-compute[72351]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)
Aug 09 02:35:52 user nova-compute[72351]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df (non-flash)
Aug 09 02:35:52 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:35:52 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:35:52 user nova-compute[72351]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)
Aug 09 02:35:52 user nova-compute[72351]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:35:52 user nova-compute[72351]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)
Aug 09 02:35:52 user nova-compute[72351]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df (non-flash)
Aug 09 02:35:52 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:35:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d -p 172.16.0.220:3260 --login {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d -p 172.16.0.220:3260 --login" returned: 0 in 0.070s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:52 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[c68c4f5a-55da-42e8-8dc6-4061c9345f1d]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:52 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d, portal: 172.16.0.220,3260]
Aug 09 02:35:52 user nova-compute[72351]: Login to [iface: default, target: iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d, portal: 172.16.0.220,3260] successful.
Aug 09 02:35:52 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:35:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:52 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[4a2f29d5-7a90-408c-976c-3247e185b42d]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:52 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:35:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.027s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:52 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[3aed383d-cb28-49fa-8323-a129a6b9ab2c]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)\ntcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)\ntcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)\ntcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df (non-flash)\ntcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:52 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:35:52 user nova-compute[72351]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)
Aug 09 02:35:52 user nova-compute[72351]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:35:52 user nova-compute[72351]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)
Aug 09 02:35:52 user nova-compute[72351]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df (non-flash)
Aug 09 02:35:52 user nova-compute[72351]: tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)
Aug 09 02:35:52 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:35:52 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:35:52 user nova-compute[72351]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)
Aug 09 02:35:52 user nova-compute[72351]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:35:52 user nova-compute[72351]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)
Aug 09 02:35:52 user nova-compute[72351]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df (non-flash)
Aug 09 02:35:52 user nova-compute[72351]: tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)
Aug 09 02:35:52 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:35:52 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.321s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:35:52 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 02:35:52 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] HCTL ('36', '-', '-', 0) found on session 50 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 02:35:52 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Scanning host 36 c: -, t: -, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 02:35:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host36/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:35:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host36/scan" returned: 0 in 0.016s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:35:52 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a5212b9a-6caa-4e1d-a133-980adbcbe3a5]: (4, ('- - 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:35:52 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Searching for a device in session 50 and hctl ['36', '*', '*', 0] yield: None {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:35:52 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:53 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Searching for a device in session 50 and hctl ['36', '*', '*', 0] yield: sde {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:35:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Connected to sde using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'arRsVtvH', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 02:35:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.350s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:35:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] <== connect_volume: return (1350ms) {'type': 'block', 'scsi_wwn': '23435343235323239', 'path': '/dev/sde'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:35:53 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23435343235323239', 'path': '/dev/sde'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 02:35:53 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,compute_id=1,config_drive='',created_at=2023-08-09T02:33:27Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-ServerActionsV293TestJSON-server-2038847230',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serveractionsv293testjson-server-657819557',id=42,image_ref='',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBEunnOgCeXfv3GBp09pSPyhwCL3D6Q1n187gHEnIiV9PA3lK8lSVc8G3tp5ZU9m6m0OxdZGYGPGwrp313iNid8wPNsg46CMpUwvg2Jv3lbvp6Ufeyxq5cu1dosjL2Qlv6A==',key_name='tempest-keypair-461212954',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:33:43Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={rebuild='server'},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='624e34ea80f944509fde2cb5472ed203',ramdisk_id='',reservation_id='r-wzsh0h7v',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerActionsV293TestJSON-475025801',owner_user_name='tempest-ServerActionsV293TestJSON-475025801-project-member'},tags=<?>,task_state='rebuild_spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:35:43Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='801c60b1fab94f38b4d40222e9946f89',uuid=ad9b389e-ea11-4fd7-9233-a1be43052e60,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "64d8b550-334e-42c4-b57f-145ccc73517c", "address": "fa:16:3e:52:34:4d", "network": {"id": "f7042dd1-90f3-4ee1-99cd-e52f9264dc87", "bridge": "br-int", "label": "tempest-ServerActionsV293TestJSON-1595239201-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.209", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "624e34ea80f944509fde2cb5472ed203", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap64d8b550-33", "ovs_interfaceid": "64d8b550-334e-42c4-b57f-145ccc73517c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:35:53 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Converting VIF {"id": "64d8b550-334e-42c4-b57f-145ccc73517c", "address": "fa:16:3e:52:34:4d", "network": {"id": "f7042dd1-90f3-4ee1-99cd-e52f9264dc87", "bridge": "br-int", "label": "tempest-ServerActionsV293TestJSON-1595239201-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.209", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "624e34ea80f944509fde2cb5472ed203", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap64d8b550-33", "ovs_interfaceid": "64d8b550-334e-42c4-b57f-145ccc73517c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:35:53 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:52:34:4d,bridge_name='br-int',has_traffic_filtering=True,id=64d8b550-334e-42c4-b57f-145ccc73517c,network=Network(f7042dd1-90f3-4ee1-99cd-e52f9264dc87),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap64d8b550-33') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:35:53 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:35:53 user nova-compute[72351]:   <uuid>ad9b389e-ea11-4fd7-9233-a1be43052e60</uuid>
Aug 09 02:35:53 user nova-compute[72351]:   <name>instance-0000002a</name>
Aug 09 02:35:53 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:35:53 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:35:53 user nova-compute[72351]:   <metadata>
Aug 09 02:35:53 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:35:53 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:35:53 user nova-compute[72351]:       <nova:name>tempest-ServerActionsV293TestJSON-server-2038847230</nova:name>
Aug 09 02:35:53 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:35:51</nova:creationTime>
Aug 09 02:35:53 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:35:53 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:35:53 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:35:53 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:35:53 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:35:53 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:35:53 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:35:53 user nova-compute[72351]:       <nova:owner>
Aug 09 02:35:53 user nova-compute[72351]:         <nova:user uuid="801c60b1fab94f38b4d40222e9946f89">tempest-ServerActionsV293TestJSON-475025801-project-member</nova:user>
Aug 09 02:35:53 user nova-compute[72351]:         <nova:project uuid="624e34ea80f944509fde2cb5472ed203">tempest-ServerActionsV293TestJSON-475025801</nova:project>
Aug 09 02:35:53 user nova-compute[72351]:       </nova:owner>
Aug 09 02:35:53 user nova-compute[72351]:       <nova:ports>
Aug 09 02:35:53 user nova-compute[72351]:         <nova:port uuid="64d8b550-334e-42c4-b57f-145ccc73517c">
Aug 09 02:35:53 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.6" ipVersion="4"/>
Aug 09 02:35:53 user nova-compute[72351]:         </nova:port>
Aug 09 02:35:53 user nova-compute[72351]:       </nova:ports>
Aug 09 02:35:53 user nova-compute[72351]:     </nova:instance>
Aug 09 02:35:53 user nova-compute[72351]:   </metadata>
Aug 09 02:35:53 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:35:53 user nova-compute[72351]:     <system>
Aug 09 02:35:53 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:35:53 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:35:53 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:35:53 user nova-compute[72351]:       <entry name="serial">ad9b389e-ea11-4fd7-9233-a1be43052e60</entry>
Aug 09 02:35:53 user nova-compute[72351]:       <entry name="uuid">ad9b389e-ea11-4fd7-9233-a1be43052e60</entry>
Aug 09 02:35:53 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:35:53 user nova-compute[72351]:     </system>
Aug 09 02:35:53 user nova-compute[72351]:   </sysinfo>
Aug 09 02:35:53 user nova-compute[72351]:   <os>
Aug 09 02:35:53 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:35:53 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:35:53 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:35:53 user nova-compute[72351]:   </os>
Aug 09 02:35:53 user nova-compute[72351]:   <features>
Aug 09 02:35:53 user nova-compute[72351]:     <acpi/>
Aug 09 02:35:53 user nova-compute[72351]:     <apic/>
Aug 09 02:35:53 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:35:53 user nova-compute[72351]:   </features>
Aug 09 02:35:53 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:35:53 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:35:53 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:35:53 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:35:53 user nova-compute[72351]:   </clock>
Aug 09 02:35:53 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:35:53 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:35:53 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:35:53 user nova-compute[72351]:   </cpu>
Aug 09 02:35:53 user nova-compute[72351]:   <devices>
Aug 09 02:35:53 user nova-compute[72351]:     <disk type="block" device="disk">
Aug 09 02:35:53 user nova-compute[72351]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:35:53 user nova-compute[72351]:       <source dev="/dev/sde"/>
Aug 09 02:35:53 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:35:53 user nova-compute[72351]:       <serial>75a3424d-ab51-409b-b1e1-f702f4a84d3d</serial>
Aug 09 02:35:53 user nova-compute[72351]:     </disk>
Aug 09 02:35:53 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:35:53 user nova-compute[72351]:       <mac address="fa:16:3e:52:34:4d"/>
Aug 09 02:35:53 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:35:53 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:35:53 user nova-compute[72351]:       <target dev="tap64d8b550-33"/>
Aug 09 02:35:53 user nova-compute[72351]:     </interface>
Aug 09 02:35:53 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:35:53 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/ad9b389e-ea11-4fd7-9233-a1be43052e60/console.log" append="off"/>
Aug 09 02:35:53 user nova-compute[72351]:     </serial>
Aug 09 02:35:53 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:35:53 user nova-compute[72351]:     <video>
Aug 09 02:35:53 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:35:53 user nova-compute[72351]:     </video>
Aug 09 02:35:53 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:35:53 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:35:53 user nova-compute[72351]:     </rng>
Aug 09 02:35:53 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:35:53 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:35:53 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:35:53 user nova-compute[72351]:     </memballoon>
Aug 09 02:35:53 user nova-compute[72351]:   </devices>
Aug 09 02:35:53 user nova-compute[72351]: </domain>
Aug 09 02:35:53 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:35:53 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,compute_id=1,config_drive='',created_at=2023-08-09T02:33:27Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-ServerActionsV293TestJSON-server-2038847230',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serveractionsv293testjson-server-657819557',id=42,image_ref='',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBEunnOgCeXfv3GBp09pSPyhwCL3D6Q1n187gHEnIiV9PA3lK8lSVc8G3tp5ZU9m6m0OxdZGYGPGwrp313iNid8wPNsg46CMpUwvg2Jv3lbvp6Ufeyxq5cu1dosjL2Qlv6A==',key_name='tempest-keypair-461212954',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:33:43Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={rebuild='server'},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='624e34ea80f944509fde2cb5472ed203',ramdisk_id='',reservation_id='r-wzsh0h7v',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerActionsV293TestJSON-475025801',owner_user_name='tempest-ServerActionsV293TestJSON-475025801-project-member'},tags=<?>,task_state='rebuild_spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:35:43Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='801c60b1fab94f38b4d40222e9946f89',uuid=ad9b389e-ea11-4fd7-9233-a1be43052e60,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "64d8b550-334e-42c4-b57f-145ccc73517c", "address": "fa:16:3e:52:34:4d", "network": {"id": "f7042dd1-90f3-4ee1-99cd-e52f9264dc87", "bridge": "br-int", "label": "tempest-ServerActionsV293TestJSON-1595239201-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.209", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "624e34ea80f944509fde2cb5472ed203", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap64d8b550-33", "ovs_interfaceid": "64d8b550-334e-42c4-b57f-145ccc73517c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:35:53 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Converting VIF {"id": "64d8b550-334e-42c4-b57f-145ccc73517c", "address": "fa:16:3e:52:34:4d", "network": {"id": "f7042dd1-90f3-4ee1-99cd-e52f9264dc87", "bridge": "br-int", "label": "tempest-ServerActionsV293TestJSON-1595239201-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.209", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "624e34ea80f944509fde2cb5472ed203", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap64d8b550-33", "ovs_interfaceid": "64d8b550-334e-42c4-b57f-145ccc73517c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:35:53 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:52:34:4d,bridge_name='br-int',has_traffic_filtering=True,id=64d8b550-334e-42c4-b57f-145ccc73517c,network=Network(f7042dd1-90f3-4ee1-99cd-e52f9264dc87),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap64d8b550-33') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:35:53 user nova-compute[72351]: DEBUG os_vif [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Plugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:52:34:4d,bridge_name='br-int',has_traffic_filtering=True,id=64d8b550-334e-42c4-b57f-145ccc73517c,network=Network(f7042dd1-90f3-4ee1-99cd-e52f9264dc87),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap64d8b550-33') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:35:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:35:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:35:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap64d8b550-33, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:35:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap64d8b550-33, col_values=(('external_ids', {'iface-id': '64d8b550-334e-42c4-b57f-145ccc73517c', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:52:34:4d', 'vm-uuid': 'ad9b389e-ea11-4fd7-9233-a1be43052e60'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:35:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:35:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:53 user nova-compute[72351]: INFO os_vif [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Successfully plugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:52:34:4d,bridge_name='br-int',has_traffic_filtering=True,id=64d8b550-334e-42c4-b57f-145ccc73517c,network=Network(f7042dd1-90f3-4ee1-99cd-e52f9264dc87),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap64d8b550-33')
Aug 09 02:35:53 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:35:53 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] No VIF found with MAC fa:16:3e:52:34:4d, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:35:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:54 user nova-compute[72351]: DEBUG nova.compute.manager [req-4823f78c-0d7c-41fc-904b-175dd78252d1 req-f2870f39-3331-4dea-b5eb-be816f2b1461 service nova] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Received event network-vif-plugged-64d8b550-334e-42c4-b57f-145ccc73517c {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:35:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-4823f78c-0d7c-41fc-904b-175dd78252d1 req-f2870f39-3331-4dea-b5eb-be816f2b1461 service nova] Acquiring lock "ad9b389e-ea11-4fd7-9233-a1be43052e60-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:35:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-4823f78c-0d7c-41fc-904b-175dd78252d1 req-f2870f39-3331-4dea-b5eb-be816f2b1461 service nova] Lock "ad9b389e-ea11-4fd7-9233-a1be43052e60-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:35:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-4823f78c-0d7c-41fc-904b-175dd78252d1 req-f2870f39-3331-4dea-b5eb-be816f2b1461 service nova] Lock "ad9b389e-ea11-4fd7-9233-a1be43052e60-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:35:54 user nova-compute[72351]: DEBUG nova.compute.manager [req-4823f78c-0d7c-41fc-904b-175dd78252d1 req-f2870f39-3331-4dea-b5eb-be816f2b1461 service nova] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] No waiting events found dispatching network-vif-plugged-64d8b550-334e-42c4-b57f-145ccc73517c {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:35:54 user nova-compute[72351]: WARNING nova.compute.manager [req-4823f78c-0d7c-41fc-904b-175dd78252d1 req-f2870f39-3331-4dea-b5eb-be816f2b1461 service nova] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Received unexpected event network-vif-plugged-64d8b550-334e-42c4-b57f-145ccc73517c for instance with vm_state active and task_state rebuild_spawning.
Aug 09 02:35:55 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:55 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:55 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:56 user nova-compute[72351]: DEBUG nova.compute.manager [req-09a6addd-d0d7-4b9c-9626-02f18b836b4c req-875baadc-1b42-492f-b8f7-4343e6f5a460 service nova] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Received event network-vif-plugged-64d8b550-334e-42c4-b57f-145ccc73517c {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:35:56 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-09a6addd-d0d7-4b9c-9626-02f18b836b4c req-875baadc-1b42-492f-b8f7-4343e6f5a460 service nova] Acquiring lock "ad9b389e-ea11-4fd7-9233-a1be43052e60-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:35:56 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-09a6addd-d0d7-4b9c-9626-02f18b836b4c req-875baadc-1b42-492f-b8f7-4343e6f5a460 service nova] Lock "ad9b389e-ea11-4fd7-9233-a1be43052e60-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:35:56 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-09a6addd-d0d7-4b9c-9626-02f18b836b4c req-875baadc-1b42-492f-b8f7-4343e6f5a460 service nova] Lock "ad9b389e-ea11-4fd7-9233-a1be43052e60-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:35:56 user nova-compute[72351]: DEBUG nova.compute.manager [req-09a6addd-d0d7-4b9c-9626-02f18b836b4c req-875baadc-1b42-492f-b8f7-4343e6f5a460 service nova] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] No waiting events found dispatching network-vif-plugged-64d8b550-334e-42c4-b57f-145ccc73517c {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:35:56 user nova-compute[72351]: WARNING nova.compute.manager [req-09a6addd-d0d7-4b9c-9626-02f18b836b4c req-875baadc-1b42-492f-b8f7-4343e6f5a460 service nova] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Received unexpected event network-vif-plugged-64d8b550-334e-42c4-b57f-145ccc73517c for instance with vm_state active and task_state rebuild_spawning.
Aug 09 02:35:57 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548557.0049462, ad9b389e-ea11-4fd7-9233-a1be43052e60 => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:35:57 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] VM Resumed (Lifecycle Event)
Aug 09 02:35:57 user nova-compute[72351]: DEBUG nova.compute.manager [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:35:57 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Guest created on hypervisor {{(pid=72351) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4411}}
Aug 09 02:35:57 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Instance spawned successfully.
Aug 09 02:35:57 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:894}}
Aug 09 02:35:57 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:35:57 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: active, current task_state: rebuild_spawning, current DB power_state: 1, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:35:57 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Found default for hw_cdrom_bus of ide {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:35:57 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Found default for hw_disk_bus of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:35:57 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Found default for hw_input_bus of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:35:57 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Found default for hw_pointer_model of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:35:57 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Found default for hw_video_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:35:57 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Found default for hw_vif_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:35:57 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] During sync_power_state the instance has a pending task (rebuild_spawning). Skip.
Aug 09 02:35:57 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548557.006683, ad9b389e-ea11-4fd7-9233-a1be43052e60 => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:35:57 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] VM Started (Lifecycle Event)
Aug 09 02:35:57 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:35:57 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Synchronizing instance power state after lifecycle event "Started"; current vm_state: active, current task_state: rebuild_spawning, current DB power_state: 1, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:35:57 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] During sync_power_state the instance has a pending task (rebuild_spawning). Skip.
Aug 09 02:35:57 user nova-compute[72351]: DEBUG nova.compute.manager [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:35:57 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691548542.193751, 49a10075-0d3c-4cd6-bb43-10238cad3f37 => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:35:57 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] VM Stopped (Lifecycle Event)
Aug 09 02:35:57 user nova-compute[72351]: DEBUG nova.compute.manager [None req-2ee60375-8589-4e21-8a34-9a0ed608692c None None] [instance: 49a10075-0d3c-4cd6-bb43-10238cad3f37] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:35:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.finish_evacuation" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:35:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.finish_evacuation" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:35:57 user nova-compute[72351]: DEBUG nova.objects.instance [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Trying to apply a migration context that does not seem to be set for this instance {{(pid=72351) apply_migration_context /opt/stack/nova/nova/objects/instance.py:1067}}
Aug 09 02:35:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-88e31a2c-0833-4b83-8d2c-33e4a21ef822 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.finish_evacuation" :: held 0.077s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:35:58 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:35:58 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:36:00 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:36:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:36:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:36:04 user nova-compute[72351]: DEBUG nova.compute.manager [req-d190bd9d-1edf-47fe-a7a7-5ce501570ec2 req-14d8fd00-16f0-4f4a-9ace-b97ff550e510 service nova] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Received event network-changed-c4a56310-1841-46ae-9a97-0421861e62c0 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:36:04 user nova-compute[72351]: DEBUG nova.compute.manager [req-d190bd9d-1edf-47fe-a7a7-5ce501570ec2 req-14d8fd00-16f0-4f4a-9ace-b97ff550e510 service nova] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Refreshing instance network info cache due to event network-changed-c4a56310-1841-46ae-9a97-0421861e62c0. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:36:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d190bd9d-1edf-47fe-a7a7-5ce501570ec2 req-14d8fd00-16f0-4f4a-9ace-b97ff550e510 service nova] Acquiring lock "refresh_cache-7b6f5d60-5f23-4ef1-b972-9ba7086f56cb" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:36:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d190bd9d-1edf-47fe-a7a7-5ce501570ec2 req-14d8fd00-16f0-4f4a-9ace-b97ff550e510 service nova] Acquired lock "refresh_cache-7b6f5d60-5f23-4ef1-b972-9ba7086f56cb" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:36:04 user nova-compute[72351]: DEBUG nova.network.neutron [req-d190bd9d-1edf-47fe-a7a7-5ce501570ec2 req-14d8fd00-16f0-4f4a-9ace-b97ff550e510 service nova] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Refreshing network info cache for port c4a56310-1841-46ae-9a97-0421861e62c0 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:36:04 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:36:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e15e16a2-6b76-43c7-97a3-6e8474229e89 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Acquiring lock "7b6f5d60-5f23-4ef1-b972-9ba7086f56cb" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:36:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e15e16a2-6b76-43c7-97a3-6e8474229e89 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "7b6f5d60-5f23-4ef1-b972-9ba7086f56cb" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:36:05 user nova-compute[72351]: INFO nova.compute.manager [None req-e15e16a2-6b76-43c7-97a3-6e8474229e89 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Detaching volume 8ad3397f-79b8-4b2e-b253-fc2c09dafe22
Aug 09 02:36:05 user nova-compute[72351]: INFO nova.virt.block_device [None req-e15e16a2-6b76-43c7-97a3-6e8474229e89 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Attempting to driver detach volume 8ad3397f-79b8-4b2e-b253-fc2c09dafe22 from mountpoint /dev/vdb
Aug 09 02:36:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-e15e16a2-6b76-43c7-97a3-6e8474229e89 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Attempting to detach device vdb from instance 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb from the persistent domain config. {{(pid=72351) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2477}}
Aug 09 02:36:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-e15e16a2-6b76-43c7-97a3-6e8474229e89 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 02:36:05 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:36:05 user nova-compute[72351]:   <source dev="/dev/sdb"/>
Aug 09 02:36:05 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:36:05 user nova-compute[72351]:   <serial>8ad3397f-79b8-4b2e-b253-fc2c09dafe22</serial>
Aug 09 02:36:05 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Aug 09 02:36:05 user nova-compute[72351]: </disk>
Aug 09 02:36:05 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 02:36:05 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-e15e16a2-6b76-43c7-97a3-6e8474229e89 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Successfully detached device vdb from instance 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb from the persistent domain config.
Aug 09 02:36:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-e15e16a2-6b76-43c7-97a3-6e8474229e89 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb from the live domain config. {{(pid=72351) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2513}}
Aug 09 02:36:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-e15e16a2-6b76-43c7-97a3-6e8474229e89 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 02:36:05 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:36:05 user nova-compute[72351]:   <source dev="/dev/sdb"/>
Aug 09 02:36:05 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:36:05 user nova-compute[72351]:   <serial>8ad3397f-79b8-4b2e-b253-fc2c09dafe22</serial>
Aug 09 02:36:05 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Aug 09 02:36:05 user nova-compute[72351]: </disk>
Aug 09 02:36:05 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 02:36:05 user nova-compute[72351]: DEBUG nova.network.neutron [req-d190bd9d-1edf-47fe-a7a7-5ce501570ec2 req-14d8fd00-16f0-4f4a-9ace-b97ff550e510 service nova] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Updated VIF entry in instance network info cache for port c4a56310-1841-46ae-9a97-0421861e62c0. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:36:05 user nova-compute[72351]: DEBUG nova.network.neutron [req-d190bd9d-1edf-47fe-a7a7-5ce501570ec2 req-14d8fd00-16f0-4f4a-9ace-b97ff550e510 service nova] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Updating instance_info_cache with network_info: [{"id": "c4a56310-1841-46ae-9a97-0421861e62c0", "address": "fa:16:3e:75:cf:ba", "network": {"id": "49e8751c-e119-430f-9110-3dc6069b1589", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-899275135-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.125", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3aea1e66805146f7a8cd1d97e64a24af", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc4a56310-18", "ovs_interfaceid": "c4a56310-1841-46ae-9a97-0421861e62c0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:36:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d190bd9d-1edf-47fe-a7a7-5ce501570ec2 req-14d8fd00-16f0-4f4a-9ace-b97ff550e510 service nova] Releasing lock "refresh_cache-7b6f5d60-5f23-4ef1-b972-9ba7086f56cb" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:36:05 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:36:05 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72351) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10461}}
Aug 09 02:36:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Received event <DeviceRemovedEvent: 1691548565.9650202, 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=72351) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2360}}
Aug 09 02:36:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-e15e16a2-6b76-43c7-97a3-6e8474229e89 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb {{(pid=72351) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2589}}
Aug 09 02:36:05 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-e15e16a2-6b76-43c7-97a3-6e8474229e89 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Successfully detached device vdb from instance 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb from the live domain config.
Aug 09 02:36:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-e15e16a2-6b76-43c7-97a3-6e8474229e89 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 02:36:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e15e16a2-6b76-43c7-97a3-6e8474229e89 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'vlwMEpQJ', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb'}, None), 'kwargs': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:36:05 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-e15e16a2-6b76-43c7-97a3-6e8474229e89 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:36:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-e15e16a2-6b76-43c7-97a3-6e8474229e89 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:36:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-e15e16a2-6b76-43c7-97a3-6e8474229e89 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:36:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e15e16a2-6b76-43c7-97a3-6e8474229e89 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 02:36:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.026s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:06 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[4d101aca-855a-4049-bf82-2f8fb9414b74]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:36:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.037s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:06 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[f571ee3c-582f-4b2b-82ff-8ac8a078b993]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)\ntcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)\ntcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)\ntcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df (non-flash)\ntcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:36:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e15e16a2-6b76-43c7-97a3-6e8474229e89 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:36:06 user nova-compute[72351]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)
Aug 09 02:36:06 user nova-compute[72351]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:36:06 user nova-compute[72351]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)
Aug 09 02:36:06 user nova-compute[72351]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df (non-flash)
Aug 09 02:36:06 user nova-compute[72351]: tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)
Aug 09 02:36:06 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:36:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e15e16a2-6b76-43c7-97a3-6e8474229e89 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:36:06 user nova-compute[72351]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 (non-flash)
Aug 09 02:36:06 user nova-compute[72351]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:36:06 user nova-compute[72351]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)
Aug 09 02:36:06 user nova-compute[72351]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df (non-flash)
Aug 09 02:36:06 user nova-compute[72351]: tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)
Aug 09 02:36:06 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:36:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e15e16a2-6b76-43c7-97a3-6e8474229e89 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fd00b3d4ca0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22'): ({'sdb'}, set())}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 02:36:06 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-e15e16a2-6b76-43c7-97a3-6e8474229e89 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Removing single pathed devices sdb {{(pid=72351) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Aug 09 02:36:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.024s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:06 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[2d3bddae-4500-41e0-a9d7-8af66cdc26fb]: (4, ('path checker states:\nup                  6\n\npaths: 0\nbusy: False\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:36:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.024s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:06 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[9874a18c-0a31-40db-a795-e14e5c153742]: (4, ('ok\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:36:06 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-e15e16a2-6b76-43c7-97a3-6e8474229e89 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Flushing IO for device /dev/sdb {{(pid=72351) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Aug 09 02:36:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.015s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:06 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[93ec6fad-39f0-4d12-a34a-9bbd3e22088e]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:36:06 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-e15e16a2-6b76-43c7-97a3-6e8474229e89 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=72351) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Aug 09 02:36:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.050s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:06 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[36a4bd5a-eef7-4efe-adbd-e250a89906b5]: (4, ('1', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:36:06 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-e15e16a2-6b76-43c7-97a3-6e8474229e89 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Aug 09 02:36:06 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-e15e16a2-6b76-43c7-97a3-6e8474229e89 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] SCSI volumes sdb have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Aug 09 02:36:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e15e16a2-6b76-43c7-97a3-6e8474229e89 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22')] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 02:36:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.028s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:06 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[f783f3a4-c828-4df2-a085-560a03a9502f]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:36:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e15e16a2-6b76-43c7-97a3-6e8474229e89 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:36:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 -p 172.16.0.220:3260 --logout {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 -p 172.16.0.220:3260 --logout" returned: 0 in 0.073s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:06 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[c4e0e839-cb29-4d4c-a716-a4b2694d43f8]: (4, ('Logging out of session [sid: 40, target: iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22, portal: 172.16.0.220,3260]\nLogout of [sid: 40, target: iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:36:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e15e16a2-6b76-43c7-97a3-6e8474229e89 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 40, target: iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22, portal: 172.16.0.220,3260]
Aug 09 02:36:06 user nova-compute[72351]: Logout of [sid: 40, target: iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22, portal: 172.16.0.220,3260] successful.
Aug 09 02:36:06 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:36:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 -p 172.16.0.220:3260 --op delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:8ad3397f-79b8-4b2e-b253-fc2c09dafe22 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.031s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:06 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[42a88fc9-477a-43c4-a0bc-2eb9e9d61f7a]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:36:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e15e16a2-6b76-43c7-97a3-6e8474229e89 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:36:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-e15e16a2-6b76-43c7-97a3-6e8474229e89 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.355s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:36:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e15e16a2-6b76-43c7-97a3-6e8474229e89 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] <== disconnect_volume: return (358ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:36:06 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-e15e16a2-6b76-43c7-97a3-6e8474229e89 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 02:36:06 user nova-compute[72351]: DEBUG nova.compute.manager [req-ceb7760b-3c0d-4d1a-ac2f-53215297a2a0 req-35cda329-8bc9-4619-9b9b-b797d9a5f377 service nova] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Received event network-changed-c4a56310-1841-46ae-9a97-0421861e62c0 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:36:06 user nova-compute[72351]: DEBUG nova.compute.manager [req-ceb7760b-3c0d-4d1a-ac2f-53215297a2a0 req-35cda329-8bc9-4619-9b9b-b797d9a5f377 service nova] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Refreshing instance network info cache due to event network-changed-c4a56310-1841-46ae-9a97-0421861e62c0. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:36:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ceb7760b-3c0d-4d1a-ac2f-53215297a2a0 req-35cda329-8bc9-4619-9b9b-b797d9a5f377 service nova] Acquiring lock "refresh_cache-7b6f5d60-5f23-4ef1-b972-9ba7086f56cb" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:36:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ceb7760b-3c0d-4d1a-ac2f-53215297a2a0 req-35cda329-8bc9-4619-9b9b-b797d9a5f377 service nova] Acquired lock "refresh_cache-7b6f5d60-5f23-4ef1-b972-9ba7086f56cb" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:36:06 user nova-compute[72351]: DEBUG nova.network.neutron [req-ceb7760b-3c0d-4d1a-ac2f-53215297a2a0 req-35cda329-8bc9-4619-9b9b-b797d9a5f377 service nova] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Refreshing network info cache for port c4a56310-1841-46ae-9a97-0421861e62c0 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:36:07 user nova-compute[72351]: DEBUG nova.objects.instance [None req-e15e16a2-6b76-43c7-97a3-6e8474229e89 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lazy-loading 'flavor' on Instance uuid 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:36:07 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e15e16a2-6b76-43c7-97a3-6e8474229e89 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "7b6f5d60-5f23-4ef1-b972-9ba7086f56cb" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.888s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:36:07 user nova-compute[72351]: DEBUG nova.network.neutron [req-ceb7760b-3c0d-4d1a-ac2f-53215297a2a0 req-35cda329-8bc9-4619-9b9b-b797d9a5f377 service nova] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Updated VIF entry in instance network info cache for port c4a56310-1841-46ae-9a97-0421861e62c0. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:36:07 user nova-compute[72351]: DEBUG nova.network.neutron [req-ceb7760b-3c0d-4d1a-ac2f-53215297a2a0 req-35cda329-8bc9-4619-9b9b-b797d9a5f377 service nova] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Updating instance_info_cache with network_info: [{"id": "c4a56310-1841-46ae-9a97-0421861e62c0", "address": "fa:16:3e:75:cf:ba", "network": {"id": "49e8751c-e119-430f-9110-3dc6069b1589", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-899275135-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.125", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3aea1e66805146f7a8cd1d97e64a24af", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc4a56310-18", "ovs_interfaceid": "c4a56310-1841-46ae-9a97-0421861e62c0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:36:07 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ceb7760b-3c0d-4d1a-ac2f-53215297a2a0 req-35cda329-8bc9-4619-9b9b-b797d9a5f377 service nova] Releasing lock "refresh_cache-7b6f5d60-5f23-4ef1-b972-9ba7086f56cb" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:36:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:36:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:36:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:36:08 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:36:09 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:36:09 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:36:09 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:36:09 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:36:09 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Auditing locally available compute resources for user (node: user) {{(pid=72351) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Aug 09 02:36:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7bef74f4-3bb3-4ad0-bff7-34156be31249/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7bef74f4-3bb3-4ad0-bff7-34156be31249/disk --force-share --output=json" returned: 0 in 0.166s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7bef74f4-3bb3-4ad0-bff7-34156be31249/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7bef74f4-3bb3-4ad0-bff7-34156be31249/disk --force-share --output=json" returned: 0 in 0.148s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/41b5e305-3973-4937-b262-d32db1091e20/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/41b5e305-3973-4937-b262-d32db1091e20/disk --force-share --output=json" returned: 0 in 0.167s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/41b5e305-3973-4937-b262-d32db1091e20/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/41b5e305-3973-4937-b262-d32db1091e20/disk --force-share --output=json" returned: 0 in 0.159s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:10 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sde (vda) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:36:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json" returned: 0 in 0.162s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json" returned: 0 in 0.162s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:11 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdh (vda) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:36:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.rescue --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.rescue --force-share --output=json" returned: 0 in 0.141s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.rescue --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.rescue --force-share --output=json" returned: 0 in 0.166s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json" returned: 0 in 0.133s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json" returned: 0 in 0.140s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:11 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdc (vda) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:36:11 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdg (vda) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:36:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7b6f5d60-5f23-4ef1-b972-9ba7086f56cb/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7b6f5d60-5f23-4ef1-b972-9ba7086f56cb/disk --force-share --output=json" returned: 0 in 0.160s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7b6f5d60-5f23-4ef1-b972-9ba7086f56cb/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7b6f5d60-5f23-4ef1-b972-9ba7086f56cb/disk --force-share --output=json" returned: 0 in 0.142s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json" returned: 0 in 0.133s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:12 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdd (vda) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:36:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-d1d2aa1a-2343-4d98-8fd3-5d2d25e17df0 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Acquiring lock "7bef74f4-3bb3-4ad0-bff7-34156be31249" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:36:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-d1d2aa1a-2343-4d98-8fd3-5d2d25e17df0 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "7bef74f4-3bb3-4ad0-bff7-34156be31249" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:36:12 user nova-compute[72351]: DEBUG nova.objects.instance [None req-d1d2aa1a-2343-4d98-8fd3-5d2d25e17df0 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lazy-loading 'flavor' on Instance uuid 7bef74f4-3bb3-4ad0-bff7-34156be31249 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:36:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-d1d2aa1a-2343-4d98-8fd3-5d2d25e17df0 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "7bef74f4-3bb3-4ad0-bff7-34156be31249" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.076s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:36:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json" returned: 0 in 0.154s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-d1d2aa1a-2343-4d98-8fd3-5d2d25e17df0 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Acquiring lock "7bef74f4-3bb3-4ad0-bff7-34156be31249" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:36:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-d1d2aa1a-2343-4d98-8fd3-5d2d25e17df0 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "7bef74f4-3bb3-4ad0-bff7-34156be31249" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:36:12 user nova-compute[72351]: INFO nova.compute.manager [None req-d1d2aa1a-2343-4d98-8fd3-5d2d25e17df0 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Attaching volume bf52660e-ce19-445c-a5dd-9c621c87e984 to /dev/vdb
Aug 09 02:36:13 user nova-compute[72351]: DEBUG os_brick.utils [None req-d1d2aa1a-2343-4d98-8fd3-5d2d25e17df0 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:36:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:36:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:13 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:36:13 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:36:13 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Hypervisor/Node resource view: name=user free_ram=7013MB free_disk=25.14130401611328GB free_vcpus=0 pci_devices=[{"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72351) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Aug 09 02:36:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:36:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:36:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.016s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:13 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ede1a578-1490-49b2-8026-c38ccf02dd1e]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:a52269602ff4\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:36:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:13 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[0749b452-b94d-4f4f-a833-7467c343830c]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:36:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:13 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[899f0c10-ebac-45c6-a3df-e953ac46a220]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:36:13 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[f0a75c56-e093-43a2-a77f-2ba408418fb4]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:36:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-d1d2aa1a-2343-4d98-8fd3-5d2d25e17df0 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Running cmd (subprocess): nvme version {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-d1d2aa1a-2343-4d98-8fd3-5d2d25e17df0 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] 'nvme version' failed. Not Retrying. {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:36:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.nvmeof [None req-d1d2aa1a-2343-4d98-8fd3-5d2d25e17df0 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] nvme not present on system {{(pid=72351) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Aug 09 02:36:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:36:13 user nova-compute[72351]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Aug 09 02:36:13 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[b36d551e-5d90-40af-b532-54626f394577]: (4, '') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:36:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-d1d2aa1a-2343-4d98-8fd3-5d2d25e17df0 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72351) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Aug 09 02:36:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-d1d2aa1a-2343-4d98-8fd3-5d2d25e17df0 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Aug 09 02:36:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-d1d2aa1a-2343-4d98-8fd3-5d2d25e17df0 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] LIGHTOS: no hostnqn found. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Aug 09 02:36:13 user nova-compute[72351]: DEBUG os_brick.utils [None req-d1d2aa1a-2343-4d98-8fd3-5d2d25e17df0 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] <== get_connector_properties: return (127ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:a52269602ff4', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:36:13 user nova-compute[72351]: DEBUG nova.virt.block_device [None req-d1d2aa1a-2343-4d98-8fd3-5d2d25e17df0 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Updating existing volume attachment record: 1eaaa409-94dd-4a77-a62d-2d9ee124eac0 {{(pid=72351) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Aug 09 02:36:13 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:36:13 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance eb86cb49-2018-42c2-bd97-c662b829775d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:36:13 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:36:13 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance a7f3d076-864b-4b68-9207-214ce2c21444 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:36:13 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 957c1510-1320-4492-9052-a58ff540319d actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:36:13 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:36:13 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 7bef74f4-3bb3-4ad0-bff7-34156be31249 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:36:13 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 41b5e305-3973-4937-b262-d32db1091e20 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:36:13 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 80e87454-90b9-4168-bf00-b7d93b245526 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 2048, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:36:13 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance ad9b389e-ea11-4fd7-9233-a1be43052e60 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:36:13 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 8e970bdd-54a1-4058-b09d-784cb6b5b1a4 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:36:13 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance bde1ef1e-163c-40eb-abb5-962b9064b7ab actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:36:13 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Total usable vcpus: 12, total allocated vcpus: 12 {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Aug 09 02:36:13 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Final resource view: name=user phys_ram=16011MB used_ram=8192MB phys_disk=40GB used_disk=7GB total_vcpus=12 used_vcpus=12 pci_stats=[] {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Aug 09 02:36:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:36:13 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:36:13 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:36:13 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Compute_service record updated for user:user {{(pid=72351) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Aug 09 02:36:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.578s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:36:15 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:36:15 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:36:15 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Starting heal instance info cache {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9842}}
Aug 09 02:36:15 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Didn't find any instances for network info cache update. {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9928}}
Aug 09 02:36:15 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:36:15 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:36:15 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:36:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:36:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:36:22 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-d1d2aa1a-2343-4d98-8fd3-5d2d25e17df0 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 02:36:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-d1d2aa1a-2343-4d98-8fd3-5d2d25e17df0 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:bf52660e-ce19-445c-a5dd-9c621c87e984'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'qifTWRdr', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:36:22 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-d1d2aa1a-2343-4d98-8fd3-5d2d25e17df0 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:36:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-d1d2aa1a-2343-4d98-8fd3-5d2d25e17df0 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:36:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-d1d2aa1a-2343-4d98-8fd3-5d2d25e17df0 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:36:22 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-d1d2aa1a-2343-4d98-8fd3-5d2d25e17df0 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:36:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-d1d2aa1a-2343-4d98-8fd3-5d2d25e17df0 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:bf52660e-ce19-445c-a5dd-9c621c87e984" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:36:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-d1d2aa1a-2343-4d98-8fd3-5d2d25e17df0 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:bf52660e-ce19-445c-a5dd-9c621c87e984" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:36:22 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-d1d2aa1a-2343-4d98-8fd3-5d2d25e17df0 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 02:36:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bf52660e-ce19-445c-a5dd-9c621c87e984 -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bf52660e-ce19-445c-a5dd-9c621c87e984 -p 172.16.0.220:3260" returned: 21 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:22 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[3ff8835b-e929-49c1-9190-d5c40ca12fba]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:36:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-d1d2aa1a-2343-4d98-8fd3-5d2d25e17df0 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Aug 09 02:36:22 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:36:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bf52660e-ce19-445c-a5dd-9c621c87e984 -p 172.16.0.220:3260 --interface default --op new {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bf52660e-ce19-445c-a5dd-9c621c87e984 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:22 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[95c76fd6-626f-48cd-8bd4-2aba16b1a8ff]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:bf52660e-ce19-445c-a5dd-9c621c87e984] added\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:36:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-d1d2aa1a-2343-4d98-8fd3-5d2d25e17df0 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:bf52660e-ce19-445c-a5dd-9c621c87e984] added
Aug 09 02:36:22 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:36:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bf52660e-ce19-445c-a5dd-9c621c87e984 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bf52660e-ce19-445c-a5dd-9c621c87e984 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:22 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[f58641e9-2dcf-4d15-8578-b6b4a5f36c80]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:36:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-d1d2aa1a-2343-4d98-8fd3-5d2d25e17df0 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:36:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bf52660e-ce19-445c-a5dd-9c621c87e984 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bf52660e-ce19-445c-a5dd-9c621c87e984 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.028s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:22 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[f0cc9d44-0637-4460-b808-328f449efae8]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:36:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-d1d2aa1a-2343-4d98-8fd3-5d2d25e17df0 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:36:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bf52660e-ce19-445c-a5dd-9c621c87e984 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v qifTWRdr {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bf52660e-ce19-445c-a5dd-9c621c87e984 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v qifTWRdr" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:22 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[9e557e57-d89d-472d-bdf5-e4b4b5ac9fdd]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:36:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-d1d2aa1a-2343-4d98-8fd3-5d2d25e17df0 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'qifTWRdr'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:36:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bf52660e-ce19-445c-a5dd-9c621c87e984 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bf52660e-ce19-445c-a5dd-9c621c87e984 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:22 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[737ad348-ac71-42b3-88be-c420a5fa5546]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:36:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-d1d2aa1a-2343-4d98-8fd3-5d2d25e17df0 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:36:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.028s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:22 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[01e713c4-7b65-4c04-9c2a-583c721b6143]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)\ntcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)\ntcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df (non-flash)\ntcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:36:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-d1d2aa1a-2343-4d98-8fd3-5d2d25e17df0 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:36:22 user nova-compute[72351]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:36:22 user nova-compute[72351]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)
Aug 09 02:36:22 user nova-compute[72351]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df (non-flash)
Aug 09 02:36:22 user nova-compute[72351]: tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)
Aug 09 02:36:22 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:36:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-d1d2aa1a-2343-4d98-8fd3-5d2d25e17df0 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:36:22 user nova-compute[72351]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:36:22 user nova-compute[72351]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)
Aug 09 02:36:22 user nova-compute[72351]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df (non-flash)
Aug 09 02:36:22 user nova-compute[72351]: tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)
Aug 09 02:36:22 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:36:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bf52660e-ce19-445c-a5dd-9c621c87e984 -p 172.16.0.220:3260 --login {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bf52660e-ce19-445c-a5dd-9c621c87e984 -p 172.16.0.220:3260 --login" returned: 0 in 0.085s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:22 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[f19f307b-e80d-4d1d-90d6-bff7587ebf9b]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:bf52660e-ce19-445c-a5dd-9c621c87e984, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:bf52660e-ce19-445c-a5dd-9c621c87e984, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:36:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-d1d2aa1a-2343-4d98-8fd3-5d2d25e17df0 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:bf52660e-ce19-445c-a5dd-9c621c87e984, portal: 172.16.0.220,3260]
Aug 09 02:36:22 user nova-compute[72351]: Login to [iface: default, target: iqn.2010-10.org.openstack:bf52660e-ce19-445c-a5dd-9c621c87e984, portal: 172.16.0.220,3260] successful.
Aug 09 02:36:22 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:36:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bf52660e-ce19-445c-a5dd-9c621c87e984 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bf52660e-ce19-445c-a5dd-9c621c87e984 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:22 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[6afbdccf-7077-4e27-8831-e1e89e570e88]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:36:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-d1d2aa1a-2343-4d98-8fd3-5d2d25e17df0 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:36:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.027s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:22 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[4810fdfa-2cde-48fb-af60-ee911d170d0b]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)\ntcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)\ntcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df (non-flash)\ntcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)\ntcp: [51] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf52660e-ce19-445c-a5dd-9c621c87e984 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:36:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-d1d2aa1a-2343-4d98-8fd3-5d2d25e17df0 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:36:22 user nova-compute[72351]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:36:22 user nova-compute[72351]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)
Aug 09 02:36:22 user nova-compute[72351]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df (non-flash)
Aug 09 02:36:22 user nova-compute[72351]: tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)
Aug 09 02:36:22 user nova-compute[72351]: tcp: [51] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf52660e-ce19-445c-a5dd-9c621c87e984 (non-flash)
Aug 09 02:36:22 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:36:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-d1d2aa1a-2343-4d98-8fd3-5d2d25e17df0 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:36:22 user nova-compute[72351]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:36:22 user nova-compute[72351]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)
Aug 09 02:36:22 user nova-compute[72351]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df (non-flash)
Aug 09 02:36:22 user nova-compute[72351]: tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)
Aug 09 02:36:22 user nova-compute[72351]: tcp: [51] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf52660e-ce19-445c-a5dd-9c621c87e984 (non-flash)
Aug 09 02:36:22 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:36:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-d1d2aa1a-2343-4d98-8fd3-5d2d25e17df0 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:bf52660e-ce19-445c-a5dd-9c621c87e984" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.334s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:36:22 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-d1d2aa1a-2343-4d98-8fd3-5d2d25e17df0 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 02:36:22 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-d1d2aa1a-2343-4d98-8fd3-5d2d25e17df0 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] HCTL ('33', '-', '-', 0) found on session 51 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 02:36:22 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-d1d2aa1a-2343-4d98-8fd3-5d2d25e17df0 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 02:36:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:22 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:22 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[b14a68c8-9d52-4e3e-8efc-7c205538e01c]: (4, ('- - 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:36:22 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-d1d2aa1a-2343-4d98-8fd3-5d2d25e17df0 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Searching for a device in session 51 and hctl ['33', '*', '*', 0] yield: None {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:36:23 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:36:23 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-d1d2aa1a-2343-4d98-8fd3-5d2d25e17df0 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Searching for a device in session 51 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:36:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-d1d2aa1a-2343-4d98-8fd3-5d2d25e17df0 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Connected to sdb using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'qifTWRdr', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 02:36:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-d1d2aa1a-2343-4d98-8fd3-5d2d25e17df0 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.370s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:36:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-d1d2aa1a-2343-4d98-8fd3-5d2d25e17df0 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] <== connect_volume: return (1372ms) {'type': 'block', 'scsi_wwn': '23633333834326535', 'path': '/dev/sdb'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:36:23 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-d1d2aa1a-2343-4d98-8fd3-5d2d25e17df0 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23633333834326535', 'path': '/dev/sdb'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 02:36:23 user nova-compute[72351]: DEBUG nova.objects.instance [None req-d1d2aa1a-2343-4d98-8fd3-5d2d25e17df0 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lazy-loading 'flavor' on Instance uuid 7bef74f4-3bb3-4ad0-bff7-34156be31249 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:36:23 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-d1d2aa1a-2343-4d98-8fd3-5d2d25e17df0 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] attach device xml: <disk type="block" device="disk">
Aug 09 02:36:23 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:36:23 user nova-compute[72351]:   <source dev="/dev/sdb"/>
Aug 09 02:36:23 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:36:23 user nova-compute[72351]:   <serial>bf52660e-ce19-445c-a5dd-9c621c87e984</serial>
Aug 09 02:36:23 user nova-compute[72351]: </disk>
Aug 09 02:36:23 user nova-compute[72351]:  {{(pid=72351) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Aug 09 02:36:23 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:36:24 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-d1d2aa1a-2343-4d98-8fd3-5d2d25e17df0 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:36:24 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-d1d2aa1a-2343-4d98-8fd3-5d2d25e17df0 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:36:24 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-d1d2aa1a-2343-4d98-8fd3-5d2d25e17df0 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] No VIF found with MAC fa:16:3e:b4:8f:2e, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:36:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-d1d2aa1a-2343-4d98-8fd3-5d2d25e17df0 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "7bef74f4-3bb3-4ad0-bff7-34156be31249" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 11.528s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:36:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-3a0b7f54-81b7-4c68-bf86-e1786b5c5426 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Acquiring lock "7bef74f4-3bb3-4ad0-bff7-34156be31249" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:36:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-3a0b7f54-81b7-4c68-bf86-e1786b5c5426 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "7bef74f4-3bb3-4ad0-bff7-34156be31249" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:36:26 user nova-compute[72351]: INFO nova.compute.manager [None req-3a0b7f54-81b7-4c68-bf86-e1786b5c5426 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Detaching volume bf52660e-ce19-445c-a5dd-9c621c87e984
Aug 09 02:36:26 user nova-compute[72351]: INFO nova.virt.block_device [None req-3a0b7f54-81b7-4c68-bf86-e1786b5c5426 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Attempting to driver detach volume bf52660e-ce19-445c-a5dd-9c621c87e984 from mountpoint /dev/vdb
Aug 09 02:36:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-3a0b7f54-81b7-4c68-bf86-e1786b5c5426 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Attempting to detach device vdb from instance 7bef74f4-3bb3-4ad0-bff7-34156be31249 from the persistent domain config. {{(pid=72351) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2477}}
Aug 09 02:36:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-3a0b7f54-81b7-4c68-bf86-e1786b5c5426 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 02:36:26 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:36:26 user nova-compute[72351]:   <source dev="/dev/sdb"/>
Aug 09 02:36:26 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:36:26 user nova-compute[72351]:   <serial>bf52660e-ce19-445c-a5dd-9c621c87e984</serial>
Aug 09 02:36:26 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Aug 09 02:36:26 user nova-compute[72351]: </disk>
Aug 09 02:36:26 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 02:36:26 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-3a0b7f54-81b7-4c68-bf86-e1786b5c5426 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Successfully detached device vdb from instance 7bef74f4-3bb3-4ad0-bff7-34156be31249 from the persistent domain config.
Aug 09 02:36:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-3a0b7f54-81b7-4c68-bf86-e1786b5c5426 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 7bef74f4-3bb3-4ad0-bff7-34156be31249 from the live domain config. {{(pid=72351) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2513}}
Aug 09 02:36:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-3a0b7f54-81b7-4c68-bf86-e1786b5c5426 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 02:36:26 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:36:26 user nova-compute[72351]:   <source dev="/dev/sdb"/>
Aug 09 02:36:26 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:36:26 user nova-compute[72351]:   <serial>bf52660e-ce19-445c-a5dd-9c621c87e984</serial>
Aug 09 02:36:26 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Aug 09 02:36:26 user nova-compute[72351]: </disk>
Aug 09 02:36:26 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 02:36:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Received event <DeviceRemovedEvent: 1691548586.8143632, 7bef74f4-3bb3-4ad0-bff7-34156be31249 => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=72351) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2360}}
Aug 09 02:36:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-3a0b7f54-81b7-4c68-bf86-e1786b5c5426 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 7bef74f4-3bb3-4ad0-bff7-34156be31249 {{(pid=72351) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2589}}
Aug 09 02:36:26 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-3a0b7f54-81b7-4c68-bf86-e1786b5c5426 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Successfully detached device vdb from instance 7bef74f4-3bb3-4ad0-bff7-34156be31249 from the live domain config.
Aug 09 02:36:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-3a0b7f54-81b7-4c68-bf86-e1786b5c5426 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 02:36:26 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-3a0b7f54-81b7-4c68-bf86-e1786b5c5426 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:bf52660e-ce19-445c-a5dd-9c621c87e984'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'qifTWRdr', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb'}, None), 'kwargs': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:36:26 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-3a0b7f54-81b7-4c68-bf86-e1786b5c5426 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:36:26 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-3a0b7f54-81b7-4c68-bf86-e1786b5c5426 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:36:26 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-3a0b7f54-81b7-4c68-bf86-e1786b5c5426 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:36:26 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-3a0b7f54-81b7-4c68-bf86-e1786b5c5426 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:bf52660e-ce19-445c-a5dd-9c621c87e984', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 02:36:26 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:26 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:26 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[74d14bbb-51dd-4d5c-aab6-505f6e5bf4e0]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:18df0ce0-8d02-4500-a04d-3c0e9e8cb5cf\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:bf52660e-ce19-445c-a5dd-9c621c87e984\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:36:26 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:26 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.028s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:26 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[9c1c9c6f-0249-4fb9-9b25-f04731b2fe01]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)\ntcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)\ntcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df (non-flash)\ntcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)\ntcp: [51] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf52660e-ce19-445c-a5dd-9c621c87e984 (non-flash)\ntcp: [52] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18df0ce0-8d02-4500-a04d-3c0e9e8cb5cf (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:36:26 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-3a0b7f54-81b7-4c68-bf86-e1786b5c5426 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:36:26 user nova-compute[72351]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:36:26 user nova-compute[72351]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)
Aug 09 02:36:26 user nova-compute[72351]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df (non-flash)
Aug 09 02:36:26 user nova-compute[72351]: tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)
Aug 09 02:36:26 user nova-compute[72351]: tcp: [51] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf52660e-ce19-445c-a5dd-9c621c87e984 (non-flash)
Aug 09 02:36:26 user nova-compute[72351]: tcp: [52] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18df0ce0-8d02-4500-a04d-3c0e9e8cb5cf (non-flash)
Aug 09 02:36:26 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:36:26 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-3a0b7f54-81b7-4c68-bf86-e1786b5c5426 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:36:26 user nova-compute[72351]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:36:26 user nova-compute[72351]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)
Aug 09 02:36:26 user nova-compute[72351]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df (non-flash)
Aug 09 02:36:26 user nova-compute[72351]: tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)
Aug 09 02:36:26 user nova-compute[72351]: tcp: [51] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf52660e-ce19-445c-a5dd-9c621c87e984 (non-flash)
Aug 09 02:36:26 user nova-compute[72351]: tcp: [52] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:18df0ce0-8d02-4500-a04d-3c0e9e8cb5cf (non-flash)
Aug 09 02:36:26 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:36:26 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-3a0b7f54-81b7-4c68-bf86-e1786b5c5426 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fd00017be20>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:bf52660e-ce19-445c-a5dd-9c621c87e984'): ({'sdb'}, set())}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 02:36:26 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-3a0b7f54-81b7-4c68-bf86-e1786b5c5426 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Removing single pathed devices sdb {{(pid=72351) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Aug 09 02:36:26 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:26 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:26 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[16097b7e-fdf9-4dce-83f2-294e6209f51b]: (4, ('path checker states:\nup                  7\n\npaths: 0\nbusy: False\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:36:26 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:26 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:26 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[313a2ded-799e-4bae-8b90-cec500a89f18]: (4, ('ok\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:36:26 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-3a0b7f54-81b7-4c68-bf86-e1786b5c5426 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Flushing IO for device /dev/sdb {{(pid=72351) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Aug 09 02:36:26 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:26 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.013s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:26 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[5dc9f238-9447-4912-ab44-618a567a1915]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:36:26 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-3a0b7f54-81b7-4c68-bf86-e1786b5c5426 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=72351) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Aug 09 02:36:26 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:26 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.033s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:27 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[5f86de88-5a79-4d63-9e26-7dd233a6af6d]: (4, ('1', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:36:27 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-3a0b7f54-81b7-4c68-bf86-e1786b5c5426 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Aug 09 02:36:27 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-3a0b7f54-81b7-4c68-bf86-e1786b5c5426 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] SCSI volumes sdb have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Aug 09 02:36:27 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-3a0b7f54-81b7-4c68-bf86-e1786b5c5426 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:bf52660e-ce19-445c-a5dd-9c621c87e984')] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 02:36:27 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bf52660e-ce19-445c-a5dd-9c621c87e984 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:27 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bf52660e-ce19-445c-a5dd-9c621c87e984 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.074s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:27 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d74fc4ca-79d7-49f9-8ca1-0298f8554178]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:36:27 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-3a0b7f54-81b7-4c68-bf86-e1786b5c5426 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:36:27 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bf52660e-ce19-445c-a5dd-9c621c87e984 -p 172.16.0.220:3260 --logout {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:27 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bf52660e-ce19-445c-a5dd-9c621c87e984 -p 172.16.0.220:3260 --logout" returned: 0 in 0.072s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:27 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d0cb82d7-2432-43b9-b06a-7017603b4180]: (4, ('Logging out of session [sid: 51, target: iqn.2010-10.org.openstack:bf52660e-ce19-445c-a5dd-9c621c87e984, portal: 172.16.0.220,3260]\nLogout of [sid: 51, target: iqn.2010-10.org.openstack:bf52660e-ce19-445c-a5dd-9c621c87e984, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:36:27 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-3a0b7f54-81b7-4c68-bf86-e1786b5c5426 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 51, target: iqn.2010-10.org.openstack:bf52660e-ce19-445c-a5dd-9c621c87e984, portal: 172.16.0.220,3260]
Aug 09 02:36:27 user nova-compute[72351]: Logout of [sid: 51, target: iqn.2010-10.org.openstack:bf52660e-ce19-445c-a5dd-9c621c87e984, portal: 172.16.0.220,3260] successful.
Aug 09 02:36:27 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:36:27 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bf52660e-ce19-445c-a5dd-9c621c87e984 -p 172.16.0.220:3260 --op delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:27 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bf52660e-ce19-445c-a5dd-9c621c87e984 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.033s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:27 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[1cb30d70-05d2-402f-b205-dfd3dadcdf09]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:36:27 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-3a0b7f54-81b7-4c68-bf86-e1786b5c5426 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:36:27 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-3a0b7f54-81b7-4c68-bf86-e1786b5c5426 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.354s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:36:27 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-3a0b7f54-81b7-4c68-bf86-e1786b5c5426 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] <== disconnect_volume: return (355ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:36:27 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-3a0b7f54-81b7-4c68-bf86-e1786b5c5426 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 02:36:28 user nova-compute[72351]: DEBUG nova.objects.instance [None req-3a0b7f54-81b7-4c68-bf86-e1786b5c5426 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lazy-loading 'flavor' on Instance uuid 7bef74f4-3bb3-4ad0-bff7-34156be31249 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:36:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-3a0b7f54-81b7-4c68-bf86-e1786b5c5426 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "7bef74f4-3bb3-4ad0-bff7-34156be31249" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.842s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:36:28 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:36:28 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:36:33 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:36:38 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:36:41 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:36:43 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:36:43 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:36:44 user nova-compute[72351]: DEBUG nova.compute.manager [req-3d4d7bf5-a28e-4ba4-8578-d17c4251d8bb req-b9e585be-c08e-4df8-b19a-b38ddca21711 service nova] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Received event network-changed-a83afb1f-511f-406c-b925-eb70b32bed90 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:36:44 user nova-compute[72351]: DEBUG nova.compute.manager [req-3d4d7bf5-a28e-4ba4-8578-d17c4251d8bb req-b9e585be-c08e-4df8-b19a-b38ddca21711 service nova] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Refreshing instance network info cache due to event network-changed-a83afb1f-511f-406c-b925-eb70b32bed90. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:36:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-3d4d7bf5-a28e-4ba4-8578-d17c4251d8bb req-b9e585be-c08e-4df8-b19a-b38ddca21711 service nova] Acquiring lock "refresh_cache-8e970bdd-54a1-4058-b09d-784cb6b5b1a4" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:36:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-3d4d7bf5-a28e-4ba4-8578-d17c4251d8bb req-b9e585be-c08e-4df8-b19a-b38ddca21711 service nova] Acquired lock "refresh_cache-8e970bdd-54a1-4058-b09d-784cb6b5b1a4" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:36:44 user nova-compute[72351]: DEBUG nova.network.neutron [req-3d4d7bf5-a28e-4ba4-8578-d17c4251d8bb req-b9e585be-c08e-4df8-b19a-b38ddca21711 service nova] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Refreshing network info cache for port a83afb1f-511f-406c-b925-eb70b32bed90 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:36:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e0a29172-d4ed-4906-bc07-1138ccc8ce3c tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Acquiring lock "8e970bdd-54a1-4058-b09d-784cb6b5b1a4" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:36:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e0a29172-d4ed-4906-bc07-1138ccc8ce3c tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Lock "8e970bdd-54a1-4058-b09d-784cb6b5b1a4" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:36:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e0a29172-d4ed-4906-bc07-1138ccc8ce3c tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Acquiring lock "8e970bdd-54a1-4058-b09d-784cb6b5b1a4-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:36:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e0a29172-d4ed-4906-bc07-1138ccc8ce3c tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Lock "8e970bdd-54a1-4058-b09d-784cb6b5b1a4-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:36:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e0a29172-d4ed-4906-bc07-1138ccc8ce3c tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Lock "8e970bdd-54a1-4058-b09d-784cb6b5b1a4-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:36:44 user nova-compute[72351]: INFO nova.compute.manager [None req-e0a29172-d4ed-4906-bc07-1138ccc8ce3c tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Terminating instance
Aug 09 02:36:44 user nova-compute[72351]: DEBUG nova.compute.manager [None req-e0a29172-d4ed-4906-bc07-1138ccc8ce3c tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Start destroying the instance on the hypervisor. {{(pid=72351) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3124}}
Aug 09 02:36:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:36:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:36:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:36:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:36:44 user nova-compute[72351]: DEBUG nova.compute.manager [req-d06b5896-edbe-4f6a-964b-be09238d7b14 req-6cf35ae4-262e-41fe-af67-0317699bb510 service nova] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Received event network-vif-unplugged-a83afb1f-511f-406c-b925-eb70b32bed90 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:36:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d06b5896-edbe-4f6a-964b-be09238d7b14 req-6cf35ae4-262e-41fe-af67-0317699bb510 service nova] Acquiring lock "8e970bdd-54a1-4058-b09d-784cb6b5b1a4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:36:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d06b5896-edbe-4f6a-964b-be09238d7b14 req-6cf35ae4-262e-41fe-af67-0317699bb510 service nova] Lock "8e970bdd-54a1-4058-b09d-784cb6b5b1a4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:36:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d06b5896-edbe-4f6a-964b-be09238d7b14 req-6cf35ae4-262e-41fe-af67-0317699bb510 service nova] Lock "8e970bdd-54a1-4058-b09d-784cb6b5b1a4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:36:44 user nova-compute[72351]: DEBUG nova.compute.manager [req-d06b5896-edbe-4f6a-964b-be09238d7b14 req-6cf35ae4-262e-41fe-af67-0317699bb510 service nova] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] No waiting events found dispatching network-vif-unplugged-a83afb1f-511f-406c-b925-eb70b32bed90 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:36:44 user nova-compute[72351]: DEBUG nova.compute.manager [req-d06b5896-edbe-4f6a-964b-be09238d7b14 req-6cf35ae4-262e-41fe-af67-0317699bb510 service nova] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Received event network-vif-unplugged-a83afb1f-511f-406c-b925-eb70b32bed90 for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:36:44 user nova-compute[72351]: DEBUG nova.network.neutron [req-3d4d7bf5-a28e-4ba4-8578-d17c4251d8bb req-b9e585be-c08e-4df8-b19a-b38ddca21711 service nova] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Updated VIF entry in instance network info cache for port a83afb1f-511f-406c-b925-eb70b32bed90. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:36:44 user nova-compute[72351]: DEBUG nova.network.neutron [req-3d4d7bf5-a28e-4ba4-8578-d17c4251d8bb req-b9e585be-c08e-4df8-b19a-b38ddca21711 service nova] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Updating instance_info_cache with network_info: [{"id": "a83afb1f-511f-406c-b925-eb70b32bed90", "address": "fa:16:3e:16:83:e9", "network": {"id": "0b3a34b6-6d32-4414-9135-22bd2a1dae20", "bridge": "br-int", "label": "tempest-TestShelveInstance-2024538322-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4d0cc057c220422a9541bcb9fa7f598b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa83afb1f-51", "ovs_interfaceid": "a83afb1f-511f-406c-b925-eb70b32bed90", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:36:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:36:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:36:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-3d4d7bf5-a28e-4ba4-8578-d17c4251d8bb req-b9e585be-c08e-4df8-b19a-b38ddca21711 service nova] Releasing lock "refresh_cache-8e970bdd-54a1-4058-b09d-784cb6b5b1a4" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:36:44 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Instance destroyed successfully.
Aug 09 02:36:44 user nova-compute[72351]: DEBUG nova.objects.instance [None req-e0a29172-d4ed-4906-bc07-1138ccc8ce3c tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Lazy-loading 'resources' on Instance uuid 8e970bdd-54a1-4058-b09d-784cb6b5b1a4 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:36:44 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-e0a29172-d4ed-4906-bc07-1138ccc8ce3c tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,compute_id=1,config_drive='',created_at=2023-08-09T02:31:07Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestShelveInstance-server-805905709',display_name='tempest-TestShelveInstance-server-805905709',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testshelveinstance-server-805905709',id=37,image_ref='',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBD9usWQkTfQS1QEHlx5U391/Ea1m4d+DjLQmZxWBZf5X7UiiUjcuxsL0cAFVTRKCtfIt43KygEG5qAmqMe/mW/kt3gzCiL7+L73rWvmAvMWlhU7Lvh6YyPMPT8wlQXad2w==',key_name='tempest-TestShelveInstance-826835257',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:35:10Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='4d0cc057c220422a9541bcb9fa7f598b',ramdisk_id='',reservation_id='r-ndibkwoj',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TestShelveInstance-1998695689',owner_user_name='tempest-TestShelveInstance-1998695689-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:35:10Z,user_data=None,user_id='0aa403f4ead84aecb78a538c2ced64e9',uuid=8e970bdd-54a1-4058-b09d-784cb6b5b1a4,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "a83afb1f-511f-406c-b925-eb70b32bed90", "address": "fa:16:3e:16:83:e9", "network": {"id": "0b3a34b6-6d32-4414-9135-22bd2a1dae20", "bridge": "br-int", "label": "tempest-TestShelveInstance-2024538322-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.128", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4d0cc057c220422a9541bcb9fa7f598b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa83afb1f-51", "ovs_interfaceid": "a83afb1f-511f-406c-b925-eb70b32bed90", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:36:44 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-e0a29172-d4ed-4906-bc07-1138ccc8ce3c tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Converting VIF {"id": "a83afb1f-511f-406c-b925-eb70b32bed90", "address": "fa:16:3e:16:83:e9", "network": {"id": "0b3a34b6-6d32-4414-9135-22bd2a1dae20", "bridge": "br-int", "label": "tempest-TestShelveInstance-2024538322-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.128", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4d0cc057c220422a9541bcb9fa7f598b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa83afb1f-51", "ovs_interfaceid": "a83afb1f-511f-406c-b925-eb70b32bed90", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:36:44 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-e0a29172-d4ed-4906-bc07-1138ccc8ce3c tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:16:83:e9,bridge_name='br-int',has_traffic_filtering=True,id=a83afb1f-511f-406c-b925-eb70b32bed90,network=Network(0b3a34b6-6d32-4414-9135-22bd2a1dae20),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa83afb1f-51') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:36:44 user nova-compute[72351]: DEBUG os_vif [None req-e0a29172-d4ed-4906-bc07-1138ccc8ce3c tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:16:83:e9,bridge_name='br-int',has_traffic_filtering=True,id=a83afb1f-511f-406c-b925-eb70b32bed90,network=Network(0b3a34b6-6d32-4414-9135-22bd2a1dae20),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa83afb1f-51') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:36:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:36:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapa83afb1f-51, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:36:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:36:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:36:44 user nova-compute[72351]: INFO os_vif [None req-e0a29172-d4ed-4906-bc07-1138ccc8ce3c tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:16:83:e9,bridge_name='br-int',has_traffic_filtering=True,id=a83afb1f-511f-406c-b925-eb70b32bed90,network=Network(0b3a34b6-6d32-4414-9135-22bd2a1dae20),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa83afb1f-51')
Aug 09 02:36:44 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-e0a29172-d4ed-4906-bc07-1138ccc8ce3c tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 02:36:44 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e0a29172-d4ed-4906-bc07-1138ccc8ce3c tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'EalQALYR', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc'}, None), 'kwargs': True}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:36:44 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-e0a29172-d4ed-4906-bc07-1138ccc8ce3c tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:36:44 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-e0a29172-d4ed-4906-bc07-1138ccc8ce3c tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:36:44 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-e0a29172-d4ed-4906-bc07-1138ccc8ce3c tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:36:44 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e0a29172-d4ed-4906-bc07-1138ccc8ce3c tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 02:36:44 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:44 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:44 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[4b2d6c40-820f-47a2-800d-42bc85c078ba]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:26683818-ed23-4877-847e-c6906eab7715\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:36:44 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.034s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:45 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[7fd17586-07f9-4f8a-a260-7a90f4d17b8b]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)\ntcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)\ntcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df (non-flash)\ntcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)\ntcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:26683818-ed23-4877-847e-c6906eab7715 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:36:45 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e0a29172-d4ed-4906-bc07-1138ccc8ce3c tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:36:45 user nova-compute[72351]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:36:45 user nova-compute[72351]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)
Aug 09 02:36:45 user nova-compute[72351]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df (non-flash)
Aug 09 02:36:45 user nova-compute[72351]: tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)
Aug 09 02:36:45 user nova-compute[72351]: tcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:26683818-ed23-4877-847e-c6906eab7715 (non-flash)
Aug 09 02:36:45 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:36:45 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e0a29172-d4ed-4906-bc07-1138ccc8ce3c tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:36:45 user nova-compute[72351]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:36:45 user nova-compute[72351]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 (non-flash)
Aug 09 02:36:45 user nova-compute[72351]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df (non-flash)
Aug 09 02:36:45 user nova-compute[72351]: tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)
Aug 09 02:36:45 user nova-compute[72351]: tcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:26683818-ed23-4877-847e-c6906eab7715 (non-flash)
Aug 09 02:36:45 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:36:45 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e0a29172-d4ed-4906-bc07-1138ccc8ce3c tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fcfdc6cde10>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3'): ({'sdc'}, set())}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 02:36:45 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-e0a29172-d4ed-4906-bc07-1138ccc8ce3c tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Removing single pathed devices sdc {{(pid=72351) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Aug 09 02:36:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:45 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[9f939ee6-8a3b-4725-b198-75c77e1069cc]: (4, ('path checker states:\nup                  6\n\npaths: 0\nbusy: False\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:36:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdc {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdc" returned: 0 in 0.025s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:45 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d3d9e4ca-94ab-4026-906e-b6d9ddcb57e5]: (4, ('ok\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:36:45 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-e0a29172-d4ed-4906-bc07-1138ccc8ce3c tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Flushing IO for device /dev/sdc {{(pid=72351) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Aug 09 02:36:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdc {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdc" returned: 0 in 0.016s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:45 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[f80cf977-6f12-4c72-83ec-739a632bc530]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:36:45 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-e0a29172-d4ed-4906-bc07-1138ccc8ce3c tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Remove SCSI device /dev/sdc with /sys/block/sdc/device/delete {{(pid=72351) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Aug 09 02:36:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdc/device/delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdc/device/delete" returned: 0 in 0.051s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:45 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d31b7991-021e-4c3c-b353-fa49a8e1896d]: (4, ('1', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:36:45 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-e0a29172-d4ed-4906-bc07-1138ccc8ce3c tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Checking to see if SCSI volumes sdc have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Aug 09 02:36:45 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-e0a29172-d4ed-4906-bc07-1138ccc8ce3c tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] SCSI volumes sdc have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Aug 09 02:36:45 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e0a29172-d4ed-4906-bc07-1138ccc8ce3c tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3')] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 02:36:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.025s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:45 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[3b0a2dd6-642b-4b03-b55a-28fb6f58b980]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:36:45 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e0a29172-d4ed-4906-bc07-1138ccc8ce3c tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:36:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 -p 172.16.0.220:3260 --logout {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 -p 172.16.0.220:3260 --logout" returned: 0 in 0.072s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:45 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[0aabe240-3fca-46b2-85e6-a3e90eabaf56]: (4, ('Logging out of session [sid: 46, target: iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3, portal: 172.16.0.220,3260]\nLogout of [sid: 46, target: iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:36:45 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e0a29172-d4ed-4906-bc07-1138ccc8ce3c tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 46, target: iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3, portal: 172.16.0.220,3260]
Aug 09 02:36:45 user nova-compute[72351]: Logout of [sid: 46, target: iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3, portal: 172.16.0.220,3260] successful.
Aug 09 02:36:45 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:36:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 -p 172.16.0.220:3260 --op delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bc173cc3-1a94-467d-8d0b-d0f50936cee3 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.032s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:45 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a76a8145-f56a-4589-ab33-5aa752369785]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:36:45 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e0a29172-d4ed-4906-bc07-1138ccc8ce3c tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:36:45 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-e0a29172-d4ed-4906-bc07-1138ccc8ce3c tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.336s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:36:45 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e0a29172-d4ed-4906-bc07-1138ccc8ce3c tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] <== disconnect_volume: return (336ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:36:45 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-e0a29172-d4ed-4906-bc07-1138ccc8ce3c tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 02:36:45 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-e0a29172-d4ed-4906-bc07-1138ccc8ce3c tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Deleting instance files /opt/stack/data/nova/instances/8e970bdd-54a1-4058-b09d-784cb6b5b1a4_del
Aug 09 02:36:45 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-e0a29172-d4ed-4906-bc07-1138ccc8ce3c tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Deletion of /opt/stack/data/nova/instances/8e970bdd-54a1-4058-b09d-784cb6b5b1a4_del complete
Aug 09 02:36:45 user nova-compute[72351]: INFO nova.compute.manager [None req-e0a29172-d4ed-4906-bc07-1138ccc8ce3c tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Took 1.26 seconds to destroy the instance on the hypervisor.
Aug 09 02:36:45 user nova-compute[72351]: DEBUG oslo.service.loopingcall [None req-e0a29172-d4ed-4906-bc07-1138ccc8ce3c tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72351) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Aug 09 02:36:45 user nova-compute[72351]: DEBUG nova.compute.manager [-] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Deallocating network for instance {{(pid=72351) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2263}}
Aug 09 02:36:45 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] deallocate_for_instance() {{(pid=72351) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1795}}
Aug 09 02:36:45 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:36:46 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:36:46 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Took 1.08 seconds to deallocate network for instance.
Aug 09 02:36:46 user nova-compute[72351]: DEBUG nova.compute.manager [req-0ad69168-bd09-41e4-81cb-f9089e135935 req-8c5947c5-de2e-4d3d-b766-bb90c0a49f15 service nova] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Received event network-vif-plugged-a83afb1f-511f-406c-b925-eb70b32bed90 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:36:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-0ad69168-bd09-41e4-81cb-f9089e135935 req-8c5947c5-de2e-4d3d-b766-bb90c0a49f15 service nova] Acquiring lock "8e970bdd-54a1-4058-b09d-784cb6b5b1a4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:36:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-0ad69168-bd09-41e4-81cb-f9089e135935 req-8c5947c5-de2e-4d3d-b766-bb90c0a49f15 service nova] Lock "8e970bdd-54a1-4058-b09d-784cb6b5b1a4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:36:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-0ad69168-bd09-41e4-81cb-f9089e135935 req-8c5947c5-de2e-4d3d-b766-bb90c0a49f15 service nova] Lock "8e970bdd-54a1-4058-b09d-784cb6b5b1a4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:36:46 user nova-compute[72351]: DEBUG nova.compute.manager [req-0ad69168-bd09-41e4-81cb-f9089e135935 req-8c5947c5-de2e-4d3d-b766-bb90c0a49f15 service nova] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] No waiting events found dispatching network-vif-plugged-a83afb1f-511f-406c-b925-eb70b32bed90 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:36:46 user nova-compute[72351]: WARNING nova.compute.manager [req-0ad69168-bd09-41e4-81cb-f9089e135935 req-8c5947c5-de2e-4d3d-b766-bb90c0a49f15 service nova] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Received unexpected event network-vif-plugged-a83afb1f-511f-406c-b925-eb70b32bed90 for instance with vm_state active and task_state deleting.
Aug 09 02:36:46 user nova-compute[72351]: DEBUG nova.compute.manager [req-0ad69168-bd09-41e4-81cb-f9089e135935 req-8c5947c5-de2e-4d3d-b766-bb90c0a49f15 service nova] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Received event network-vif-deleted-a83afb1f-511f-406c-b925-eb70b32bed90 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:36:47 user nova-compute[72351]: INFO nova.compute.manager [None req-e0a29172-d4ed-4906-bc07-1138ccc8ce3c tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Took 0.81 seconds to detach 1 volumes for instance.
Aug 09 02:36:47 user nova-compute[72351]: DEBUG nova.compute.manager [None req-e0a29172-d4ed-4906-bc07-1138ccc8ce3c tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Deleting volume: bc173cc3-1a94-467d-8d0b-d0f50936cee3 {{(pid=72351) _cleanup_volumes /opt/stack/nova/nova/compute/manager.py:3221}}
Aug 09 02:36:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e0a29172-d4ed-4906-bc07-1138ccc8ce3c tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:36:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e0a29172-d4ed-4906-bc07-1138ccc8ce3c tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:36:47 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-e0a29172-d4ed-4906-bc07-1138ccc8ce3c tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:36:47 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-e0a29172-d4ed-4906-bc07-1138ccc8ce3c tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:36:47 user nova-compute[72351]: INFO nova.compute.manager [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Rescuing
Aug 09 02:36:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Acquiring lock "refresh_cache-bde1ef1e-163c-40eb-abb5-962b9064b7ab" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:36:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Acquired lock "refresh_cache-bde1ef1e-163c-40eb-abb5-962b9064b7ab" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:36:47 user nova-compute[72351]: DEBUG nova.network.neutron [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:36:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e0a29172-d4ed-4906-bc07-1138ccc8ce3c tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.538s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:36:48 user nova-compute[72351]: INFO nova.scheduler.client.report [None req-e0a29172-d4ed-4906-bc07-1138ccc8ce3c tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Deleted allocations for instance 8e970bdd-54a1-4058-b09d-784cb6b5b1a4
Aug 09 02:36:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e0a29172-d4ed-4906-bc07-1138ccc8ce3c tempest-TestShelveInstance-1998695689 tempest-TestShelveInstance-1998695689-project-member] Lock "8e970bdd-54a1-4058-b09d-784cb6b5b1a4" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 4.039s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:36:48 user nova-compute[72351]: INFO nova.compute.manager [None req-3f9ce316-4d16-421d-8d97-0a5142a06665 tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Get console output
Aug 09 02:36:48 user nova-compute[72351]: INFO nova.privsep.libvirt [-] Ignored error while reading from instance console pty: can't concat NoneType to bytes
Aug 09 02:36:48 user nova-compute[72351]: DEBUG nova.network.neutron [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Updating instance_info_cache with network_info: [{"id": "8424518d-87b7-4fc4-8a58-1b4f094a5ca0", "address": "fa:16:3e:08:61:83", "network": {"id": "9a452414-d61f-4045-a6cc-9be93c892c4c", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "321111f935dd4e8aab4a083206ea9a7e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8424518d-87", "ovs_interfaceid": "8424518d-87b7-4fc4-8a58-1b4f094a5ca0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:36:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Releasing lock "refresh_cache-bde1ef1e-163c-40eb-abb5-962b9064b7ab" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:36:48 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:36:48 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:36:48 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:36:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:36:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:36:49 user nova-compute[72351]: DEBUG nova.compute.manager [req-cf116a16-8248-4f17-8ded-be985085bbf3 req-7f45e5eb-c18a-48f3-9ba0-d7f46128defc service nova] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Received event network-vif-unplugged-8424518d-87b7-4fc4-8a58-1b4f094a5ca0 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:36:49 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-cf116a16-8248-4f17-8ded-be985085bbf3 req-7f45e5eb-c18a-48f3-9ba0-d7f46128defc service nova] Acquiring lock "bde1ef1e-163c-40eb-abb5-962b9064b7ab-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:36:49 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-cf116a16-8248-4f17-8ded-be985085bbf3 req-7f45e5eb-c18a-48f3-9ba0-d7f46128defc service nova] Lock "bde1ef1e-163c-40eb-abb5-962b9064b7ab-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:36:49 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-cf116a16-8248-4f17-8ded-be985085bbf3 req-7f45e5eb-c18a-48f3-9ba0-d7f46128defc service nova] Lock "bde1ef1e-163c-40eb-abb5-962b9064b7ab-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:36:49 user nova-compute[72351]: DEBUG nova.compute.manager [req-cf116a16-8248-4f17-8ded-be985085bbf3 req-7f45e5eb-c18a-48f3-9ba0-d7f46128defc service nova] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] No waiting events found dispatching network-vif-unplugged-8424518d-87b7-4fc4-8a58-1b4f094a5ca0 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:36:49 user nova-compute[72351]: WARNING nova.compute.manager [req-cf116a16-8248-4f17-8ded-be985085bbf3 req-7f45e5eb-c18a-48f3-9ba0-d7f46128defc service nova] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Received unexpected event network-vif-unplugged-8424518d-87b7-4fc4-8a58-1b4f094a5ca0 for instance with vm_state active and task_state rescuing.
Aug 09 02:36:49 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Instance destroyed successfully.
Aug 09 02:36:49 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Attempting a stable device rescue
Aug 09 02:36:49 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] rescue generated disk_info: {'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk.rescue': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} {{(pid=72351) rescue /opt/stack/nova/nova/virt/libvirt/driver.py:4308}}
Aug 09 02:36:49 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Instance directory exists: not creating {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4713}}
Aug 09 02:36:49 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Creating image(s)
Aug 09 02:36:49 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Acquiring lock "/opt/stack/data/nova/instances/bde1ef1e-163c-40eb-abb5-962b9064b7ab/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:36:49 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "/opt/stack/data/nova/instances/bde1ef1e-163c-40eb-abb5-962b9064b7ab/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:36:49 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "/opt/stack/data/nova/instances/bde1ef1e-163c-40eb-abb5-962b9064b7ab/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:36:49 user nova-compute[72351]: DEBUG nova.objects.instance [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lazy-loading 'trusted_certs' on Instance uuid bde1ef1e-163c-40eb-abb5-962b9064b7ab {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:36:49 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Acquiring lock "2d18b85f2b82a300c273485804088c44d5324c8d" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:36:49 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "2d18b85f2b82a300c273485804088c44d5324c8d" acquired by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:36:50 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:36:50 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/2d18b85f2b82a300c273485804088c44d5324c8d.part --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:50 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/2d18b85f2b82a300c273485804088c44d5324c8d.part --force-share --output=json" returned: 0 in 0.142s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:50 user nova-compute[72351]: DEBUG nova.virt.images [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] 741f738a-318d-4848-8765-d2cb967a0d7a was qcow2, converting to raw {{(pid=72351) fetch_to_raw /opt/stack/nova/nova/virt/images.py:165}}
Aug 09 02:36:50 user nova-compute[72351]: DEBUG nova.privsep.utils [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=72351) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Aug 09 02:36:50 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Running cmd (subprocess): qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/2d18b85f2b82a300c273485804088c44d5324c8d.part /opt/stack/data/nova/instances/_base/2d18b85f2b82a300c273485804088c44d5324c8d.converted {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:51 user nova-compute[72351]: DEBUG nova.compute.manager [req-ed32940c-d8ec-4b3e-9924-e699476b242b req-5478ac09-76d3-4124-a8d9-53bfe2e0a92e service nova] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Received event network-vif-plugged-8424518d-87b7-4fc4-8a58-1b4f094a5ca0 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:36:51 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ed32940c-d8ec-4b3e-9924-e699476b242b req-5478ac09-76d3-4124-a8d9-53bfe2e0a92e service nova] Acquiring lock "bde1ef1e-163c-40eb-abb5-962b9064b7ab-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:36:51 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ed32940c-d8ec-4b3e-9924-e699476b242b req-5478ac09-76d3-4124-a8d9-53bfe2e0a92e service nova] Lock "bde1ef1e-163c-40eb-abb5-962b9064b7ab-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:36:51 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ed32940c-d8ec-4b3e-9924-e699476b242b req-5478ac09-76d3-4124-a8d9-53bfe2e0a92e service nova] Lock "bde1ef1e-163c-40eb-abb5-962b9064b7ab-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:36:51 user nova-compute[72351]: DEBUG nova.compute.manager [req-ed32940c-d8ec-4b3e-9924-e699476b242b req-5478ac09-76d3-4124-a8d9-53bfe2e0a92e service nova] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] No waiting events found dispatching network-vif-plugged-8424518d-87b7-4fc4-8a58-1b4f094a5ca0 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:36:51 user nova-compute[72351]: WARNING nova.compute.manager [req-ed32940c-d8ec-4b3e-9924-e699476b242b req-5478ac09-76d3-4124-a8d9-53bfe2e0a92e service nova] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Received unexpected event network-vif-plugged-8424518d-87b7-4fc4-8a58-1b4f094a5ca0 for instance with vm_state active and task_state rescuing.
Aug 09 02:36:51 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] CMD "qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/2d18b85f2b82a300c273485804088c44d5324c8d.part /opt/stack/data/nova/instances/_base/2d18b85f2b82a300c273485804088c44d5324c8d.converted" returned: 0 in 0.545s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:51 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/2d18b85f2b82a300c273485804088c44d5324c8d.converted --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:51 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/2d18b85f2b82a300c273485804088c44d5324c8d.converted --force-share --output=json" returned: 0 in 0.153s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:51 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "2d18b85f2b82a300c273485804088c44d5324c8d" "released" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: held 2.076s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:36:51 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Acquiring lock "2d18b85f2b82a300c273485804088c44d5324c8d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:36:51 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "2d18b85f2b82a300c273485804088c44d5324c8d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:36:51 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/2d18b85f2b82a300c273485804088c44d5324c8d --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:51 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/2d18b85f2b82a300c273485804088c44d5324c8d --force-share --output=json" returned: 0 in 0.139s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:51 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/2d18b85f2b82a300c273485804088c44d5324c8d,backing_fmt=raw /opt/stack/data/nova/instances/bde1ef1e-163c-40eb-abb5-962b9064b7ab/disk.rescue {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:51 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/2d18b85f2b82a300c273485804088c44d5324c8d,backing_fmt=raw /opt/stack/data/nova/instances/bde1ef1e-163c-40eb-abb5-962b9064b7ab/disk.rescue" returned: 0 in 0.057s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:51 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "2d18b85f2b82a300c273485804088c44d5324c8d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.203s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:36:51 user nova-compute[72351]: DEBUG nova.objects.instance [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lazy-loading 'migration_context' on Instance uuid bde1ef1e-163c-40eb-abb5-962b9064b7ab {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:36:51 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Created local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4851}}
Aug 09 02:36:51 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Start _get_guest_xml network_info=[{"id": "8424518d-87b7-4fc4-8a58-1b4f094a5ca0", "address": "fa:16:3e:08:61:83", "network": {"id": "9a452414-d61f-4045-a6cc-9be93c892c4c", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [], "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "vif_mac": "fa:16:3e:08:61:83"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "321111f935dd4e8aab4a083206ea9a7e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8424518d-87", "ovs_interfaceid": "8424518d-87b7-4fc4-8a58-1b4f094a5ca0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk.rescue': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>) rescue={'image_id': '741f738a-318d-4848-8765-d2cb967a0d7a', 'kernel_id': '', 'ramdisk_id': ''} block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'device_type': 'disk', 'guest_format': None, 'disk_bus': 'virtio', 'mount_device': '/dev/vda', 'boot_index': 0, 'attachment_id': '62241c55-0d08-4cc4-8b91-0e840a1e3282', 'delete_on_termination': False, 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqns': ['iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'LNfcKqMo', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdh'}, 'status': 'reserved', 'instance': 'bde1ef1e-163c-40eb-abb5-962b9064b7ab', 'attached_at': '', 'detached_at': '', 'volume_id': 'b5789e9a-6f06-4239-8523-0e8607f350df', 'serial': 'b5789e9a-6f06-4239-8523-0e8607f350df'}, 'volume_type': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:36:51 user nova-compute[72351]: DEBUG nova.objects.instance [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lazy-loading 'resources' on Instance uuid bde1ef1e-163c-40eb-abb5-962b9064b7ab {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:36:51 user nova-compute[72351]: DEBUG nova.objects.instance [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lazy-loading 'numa_topology' on Instance uuid bde1ef1e-163c-40eb-abb5-962b9064b7ab {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:36:51 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:36:51 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:36:51 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:36:51 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:36:51 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:36:51 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:36:51 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:36:51 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:36:51 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:36:51 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:36:51 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:36:51 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:36:51 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:36:51 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:36:51 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:36:51 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:36:51 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:36:51 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:36:51 user nova-compute[72351]: DEBUG nova.objects.instance [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lazy-loading 'vcpu_model' on Instance uuid bde1ef1e-163c-40eb-abb5-962b9064b7ab {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:36:51 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 02:36:51 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'LNfcKqMo', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path'}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:36:51 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:36:51 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:36:51 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:36:51 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:36:51 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:36:51 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:36:51 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 02:36:51 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:51 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df -p 172.16.0.220:3260" returned: 0 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:51 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e6db4a7f-2e92-4846-8db2-98ec92c6653a]: (4, ('# BEGIN RECORD 2.1.5\nnode.name = iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df\nnode.tpgt = -1\nnode.startup = automatic\nnode.leading_login = No\niface.iscsi_ifacename = default\niface.net_ifacename = <empty>\niface.ipaddress = <empty>\niface.prefix_len = 0\niface.hwaddress = <empty>\niface.transport_name = tcp\niface.initiatorname = <empty>\niface.state = <empty>\niface.vlan_id = 0\niface.vlan_priority = 0\niface.vlan_state = <empty>\niface.iface_num = 0\niface.mtu = 0\niface.port = 0\niface.bootproto = <empty>\niface.subnet_mask = <empty>\niface.gateway = <empty>\niface.dhcp_alt_client_id_state = <empty>\niface.dhcp_alt_client_id = <empty>\niface.dhcp_dns = <empty>\niface.dhcp_learn_iqn = <empty>\niface.dhcp_req_vendor_id_state = <empty>\niface.dhcp_vendor_id_state = <empty>\niface.dhcp_vendor_id = <empty>\niface.dhcp_slp_da = <empty>\niface.fragmentation = <empty>\niface.gratuitous_arp = <empty>\niface.incoming_forwarding = <empty>\niface.tos_state = <empty>\niface.tos = 0\niface.ttl = 0\niface.delayed_ack = <empty>\niface.tcp_nagle = <empty>\niface.tcp_wsf_state = <empty>\niface.tcp_wsf = 0\niface.tcp_timer_scale = 0\niface.tcp_timestamp = <empty>\niface.redirect = <empty>\niface.def_task_mgmt_timeout = 0\niface.header_digest = <empty>\niface.data_digest = <empty>\niface.immediate_data = <empty>\niface.initial_r2t = <empty>\niface.data_seq_inorder = <empty>\niface.data_pdu_inorder = <empty>\niface.erl = 0\niface.max_receive_data_len = 0\niface.first_burst_len = 0\niface.max_outstanding_r2t = 0\niface.max_burst_len = 0\niface.chap_auth = <empty>\niface.bidi_chap = <empty>\niface.strict_login_compliance = <empty>\niface.discovery_auth = <empty>\niface.discovery_logout = <empty>\nnode.discovery_address = <empty>\nnode.discovery_port = 0\nnode.discovery_type = static\nnode.session.initial_cmdsn = 0\nnode.session.initial_login_retry_max = 8\nnode.session.xmit_thread_priority = -20\nnode.session.cmds_max = 128\nnode.session.queue_depth = 32\nnode.session.nr_sessions = 1\nnode.session.auth.authmethod = CHAP\nnode.session.auth.username = LNfcKqMo\nnode.session.auth.password = ********\nnode.session.auth.username_in = <empty>\nnode.session.auth.password_in = <empty>\nnode.session.auth.chap_algs = MD5\nnode.session.timeo.replacement_timeout = 120\nnode.session.err_timeo.abort_timeout = 15\nnode.session.err_timeo.lu_reset_timeout = 30\nnode.session.err_timeo.tgt_reset_timeout = 30\nnode.session.err_timeo.host_reset_timeout = 60\nnode.session.iscsi.FastAbort = Yes\nnode.session.iscsi.InitialR2T = No\nnode.session.iscsi.ImmediateData = Yes\nnode.session.iscsi.FirstBurstLength = 262144\nnode.session.iscsi.MaxBurstLength = 16776192\nnode.session.iscsi.DefaultTime2Retain = 0\nnode.session.iscsi.DefaultTime2Wait = 2\nnode.session.iscsi.MaxConnections = 1\nnode.session.iscsi.MaxOutstandingR2T = 1\nnode.session.iscsi.ERL = 0\nnode.session.scan = manual\nnode.session.reopen_max = 0\nnode.conn[0].address = 172.16.0.220\nnode.conn[0].port = 3260\nnode.conn[0].startup = manual\nnode.conn[0].tcp.window_size = 524288\nnode.conn[0].tcp.type_of_service = 0\nnode.conn[0].timeo.logout_timeout = 15\nnode.conn[0].timeo.login_timeout = 15\nnode.conn[0].timeo.auth_timeout = 45\nnode.conn[0].timeo.noop_out_interval = 5\nnode.conn[0].timeo.noop_out_timeout = 5\nnode.conn[0].iscsi.MaxXmitDataSegmentLength = 0\nnode.conn[0].iscsi.MaxRecvDataSegmentLength = 262144\nnode.conn[0].iscsi.HeaderDigest = None\nnode.conn[0].iscsi.DataDigest = None\nnode.conn[0].iscsi.IFMarker = No\nnode.conn[0].iscsi.OFMarker = No\n# END RECORD\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:36:51 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] iscsiadm (): stdout=# BEGIN RECORD 2.1.5
Aug 09 02:36:51 user nova-compute[72351]: node.name = iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df
Aug 09 02:36:51 user nova-compute[72351]: node.tpgt = -1
Aug 09 02:36:51 user nova-compute[72351]: node.startup = automatic
Aug 09 02:36:51 user nova-compute[72351]: node.leading_login = No
Aug 09 02:36:51 user nova-compute[72351]: iface.iscsi_ifacename = default
Aug 09 02:36:51 user nova-compute[72351]: iface.net_ifacename = <empty>
Aug 09 02:36:51 user nova-compute[72351]: iface.ipaddress = <empty>
Aug 09 02:36:51 user nova-compute[72351]: iface.prefix_len = 0
Aug 09 02:36:51 user nova-compute[72351]: iface.hwaddress = <empty>
Aug 09 02:36:51 user nova-compute[72351]: iface.transport_name = tcp
Aug 09 02:36:51 user nova-compute[72351]: iface.initiatorname = <empty>
Aug 09 02:36:51 user nova-compute[72351]: iface.state = <empty>
Aug 09 02:36:51 user nova-compute[72351]: iface.vlan_id = 0
Aug 09 02:36:51 user nova-compute[72351]: iface.vlan_priority = 0
Aug 09 02:36:51 user nova-compute[72351]: iface.vlan_state = <empty>
Aug 09 02:36:51 user nova-compute[72351]: iface.iface_num = 0
Aug 09 02:36:51 user nova-compute[72351]: iface.mtu = 0
Aug 09 02:36:51 user nova-compute[72351]: iface.port = 0
Aug 09 02:36:51 user nova-compute[72351]: iface.bootproto = <empty>
Aug 09 02:36:51 user nova-compute[72351]: iface.subnet_mask = <empty>
Aug 09 02:36:51 user nova-compute[72351]: iface.gateway = <empty>
Aug 09 02:36:51 user nova-compute[72351]: iface.dhcp_alt_client_id_state = <empty>
Aug 09 02:36:51 user nova-compute[72351]: iface.dhcp_alt_client_id = <empty>
Aug 09 02:36:51 user nova-compute[72351]: iface.dhcp_dns = <empty>
Aug 09 02:36:51 user nova-compute[72351]: iface.dhcp_learn_iqn = <empty>
Aug 09 02:36:51 user nova-compute[72351]: iface.dhcp_req_vendor_id_state = <empty>
Aug 09 02:36:51 user nova-compute[72351]: iface.dhcp_vendor_id_state = <empty>
Aug 09 02:36:51 user nova-compute[72351]: iface.dhcp_vendor_id = <empty>
Aug 09 02:36:51 user nova-compute[72351]: iface.dhcp_slp_da = <empty>
Aug 09 02:36:51 user nova-compute[72351]: iface.fragmentation = <empty>
Aug 09 02:36:51 user nova-compute[72351]: iface.gratuitous_arp = <empty>
Aug 09 02:36:51 user nova-compute[72351]: iface.incoming_forwarding = <empty>
Aug 09 02:36:51 user nova-compute[72351]: iface.tos_state = <empty>
Aug 09 02:36:51 user nova-compute[72351]: iface.tos = 0
Aug 09 02:36:51 user nova-compute[72351]: iface.ttl = 0
Aug 09 02:36:51 user nova-compute[72351]: iface.delayed_ack = <empty>
Aug 09 02:36:51 user nova-compute[72351]: iface.tcp_nagle = <empty>
Aug 09 02:36:51 user nova-compute[72351]: iface.tcp_wsf_state = <empty>
Aug 09 02:36:51 user nova-compute[72351]: iface.tcp_wsf = 0
Aug 09 02:36:51 user nova-compute[72351]: iface.tcp_timer_scale = 0
Aug 09 02:36:51 user nova-compute[72351]: iface.tcp_timestamp = <empty>
Aug 09 02:36:51 user nova-compute[72351]: iface.redirect = <empty>
Aug 09 02:36:51 user nova-compute[72351]: iface.def_task_mgmt_timeout = 0
Aug 09 02:36:51 user nova-compute[72351]: iface.header_digest = <empty>
Aug 09 02:36:51 user nova-compute[72351]: iface.data_digest = <empty>
Aug 09 02:36:51 user nova-compute[72351]: iface.immediate_data = <empty>
Aug 09 02:36:51 user nova-compute[72351]: iface.initial_r2t = <empty>
Aug 09 02:36:51 user nova-compute[72351]: iface.data_seq_inorder = <empty>
Aug 09 02:36:51 user nova-compute[72351]: iface.data_pdu_inorder = <empty>
Aug 09 02:36:51 user nova-compute[72351]: iface.erl = 0
Aug 09 02:36:51 user nova-compute[72351]: iface.max_receive_data_len = 0
Aug 09 02:36:51 user nova-compute[72351]: iface.first_burst_len = 0
Aug 09 02:36:51 user nova-compute[72351]: iface.max_outstanding_r2t = 0
Aug 09 02:36:51 user nova-compute[72351]: iface.max_burst_len = 0
Aug 09 02:36:51 user nova-compute[72351]: iface.chap_auth = <empty>
Aug 09 02:36:51 user nova-compute[72351]: iface.bidi_chap = <empty>
Aug 09 02:36:51 user nova-compute[72351]: iface.strict_login_compliance = <empty>
Aug 09 02:36:51 user nova-compute[72351]: iface.discovery_auth = <empty>
Aug 09 02:36:51 user nova-compute[72351]: iface.discovery_logout = <empty>
Aug 09 02:36:51 user nova-compute[72351]: node.discovery_address = <empty>
Aug 09 02:36:51 user nova-compute[72351]: node.discovery_port = 0
Aug 09 02:36:51 user nova-compute[72351]: node.discovery_type = static
Aug 09 02:36:51 user nova-compute[72351]: node.session.initial_cmdsn = 0
Aug 09 02:36:51 user nova-compute[72351]: node.session.initial_login_retry_max = 8
Aug 09 02:36:51 user nova-compute[72351]: node.session.xmit_thread_priority = -20
Aug 09 02:36:51 user nova-compute[72351]: node.session.cmds_max = 128
Aug 09 02:36:51 user nova-compute[72351]: node.session.queue_depth = 32
Aug 09 02:36:51 user nova-compute[72351]: node.session.nr_sessions = 1
Aug 09 02:36:51 user nova-compute[72351]: node.session.auth.authmethod = CHAP
Aug 09 02:36:51 user nova-compute[72351]: node.session.auth.username = LNfcKqMo
Aug 09 02:36:51 user nova-compute[72351]: node.session.auth.password = ***
Aug 09 02:36:51 user nova-compute[72351]: node.session.auth.username_in = <empty>
Aug 09 02:36:51 user nova-compute[72351]: node.session.auth.password_in = <empty>
Aug 09 02:36:51 user nova-compute[72351]: node.session.auth.chap_algs = MD5
Aug 09 02:36:51 user nova-compute[72351]: node.session.timeo.replacement_timeout = 120
Aug 09 02:36:51 user nova-compute[72351]: node.session.err_timeo.abort_timeout = 15
Aug 09 02:36:51 user nova-compute[72351]: node.session.err_timeo.lu_reset_timeout = 30
Aug 09 02:36:51 user nova-compute[72351]: node.session.err_timeo.tgt_reset_timeout = 30
Aug 09 02:36:51 user nova-compute[72351]: node.session.err_timeo.host_reset_timeout = 60
Aug 09 02:36:51 user nova-compute[72351]: node.session.iscsi.FastAbort = Yes
Aug 09 02:36:51 user nova-compute[72351]: node.session.iscsi.InitialR2T = No
Aug 09 02:36:51 user nova-compute[72351]: node.session.iscsi.ImmediateData = Yes
Aug 09 02:36:51 user nova-compute[72351]: node.session.iscsi.FirstBurstLength = 262144
Aug 09 02:36:51 user nova-compute[72351]: node.session.iscsi.MaxBurstLength = 16776192
Aug 09 02:36:51 user nova-compute[72351]: node.session.iscsi.DefaultTime2Retain = 0
Aug 09 02:36:51 user nova-compute[72351]: node.session.iscsi.DefaultTime2Wait = 2
Aug 09 02:36:51 user nova-compute[72351]: node.session.iscsi.MaxConnections = 1
Aug 09 02:36:51 user nova-compute[72351]: node.session.iscsi.MaxOutstandingR2T = 1
Aug 09 02:36:51 user nova-compute[72351]: node.session.iscsi.ERL = 0
Aug 09 02:36:51 user nova-compute[72351]: node.session.scan = manual
Aug 09 02:36:51 user nova-compute[72351]: node.session.reopen_max = 0
Aug 09 02:36:51 user nova-compute[72351]: node.conn[0].address = 172.16.0.220
Aug 09 02:36:51 user nova-compute[72351]: node.conn[0].port = 3260
Aug 09 02:36:51 user nova-compute[72351]: node.conn[0].startup = manual
Aug 09 02:36:51 user nova-compute[72351]: node.conn[0].tcp.window_size = 524288
Aug 09 02:36:51 user nova-compute[72351]: node.conn[0].tcp.type_of_service = 0
Aug 09 02:36:51 user nova-compute[72351]: node.conn[0].timeo.logout_timeout = 15
Aug 09 02:36:51 user nova-compute[72351]: node.conn[0].timeo.login_timeout = 15
Aug 09 02:36:51 user nova-compute[72351]: node.conn[0].timeo.auth_timeout = 45
Aug 09 02:36:51 user nova-compute[72351]: node.conn[0].timeo.noop_out_interval = 5
Aug 09 02:36:51 user nova-compute[72351]: node.conn[0].timeo.noop_out_timeout = 5
Aug 09 02:36:51 user nova-compute[72351]: node.conn[0].iscsi.MaxXmitDataSegmentLength = 0
Aug 09 02:36:51 user nova-compute[72351]: node.conn[0].iscsi.MaxRecvDataSegmentLength = 262144
Aug 09 02:36:51 user nova-compute[72351]: node.conn[0].iscsi.HeaderDigest = None
Aug 09 02:36:51 user nova-compute[72351]: node.conn[0].iscsi.DataDigest = None
Aug 09 02:36:51 user nova-compute[72351]: node.conn[0].iscsi.IFMarker = No
Aug 09 02:36:51 user nova-compute[72351]: node.conn[0].iscsi.OFMarker = No
Aug 09 02:36:51 user nova-compute[72351]: # END RECORD
Aug 09 02:36:51 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:36:51 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:51 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:51 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[4a19cbe7-e6d4-4564-aacc-3e62852e09e9]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:36:52 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:36:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:52 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[427272af-2eee-44a9-b645-592d9c63adcf]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:36:52 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:36:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df -p 172.16.0.220:3260 --op update -n node.session.auth.username -v LNfcKqMo {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df -p 172.16.0.220:3260 --op update -n node.session.auth.username -v LNfcKqMo" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:52 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e6ca36bf-dfae-4474-9d6b-2144b53115d7]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:36:52 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'LNfcKqMo'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:36:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:52 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[fb389752-05b7-46a7-948f-c86e5ac2b426]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:36:52 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:36:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.033s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:52 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[391806a1-0087-4d21-bd63-4b3887daa73d]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)\ntcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df (non-flash)\ntcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)\ntcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:26683818-ed23-4877-847e-c6906eab7715 (non-flash)\ntcp: [54] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c32db06c-590e-48d8-883e-e2c4ec62fdf9 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:36:52 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:36:52 user nova-compute[72351]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:36:52 user nova-compute[72351]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df (non-flash)
Aug 09 02:36:52 user nova-compute[72351]: tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)
Aug 09 02:36:52 user nova-compute[72351]: tcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:26683818-ed23-4877-847e-c6906eab7715 (non-flash)
Aug 09 02:36:52 user nova-compute[72351]: tcp: [54] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c32db06c-590e-48d8-883e-e2c4ec62fdf9 (non-flash)
Aug 09 02:36:52 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:36:52 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:36:52 user nova-compute[72351]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:36:52 user nova-compute[72351]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df (non-flash)
Aug 09 02:36:52 user nova-compute[72351]: tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)
Aug 09 02:36:52 user nova-compute[72351]: tcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:26683818-ed23-4877-847e-c6906eab7715 (non-flash)
Aug 09 02:36:52 user nova-compute[72351]: tcp: [54] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c32db06c-590e-48d8-883e-e2c4ec62fdf9 (non-flash)
Aug 09 02:36:52 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:36:52 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.176s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:36:52 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 02:36:52 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] HCTL ('39', '0', '0', 0) found on session 48 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 02:36:52 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Scanning host 39 c: 0, t: 0, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 02:36:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host39/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:36:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host39/scan" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:36:52 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[31a8721f-c22d-41e7-b399-87399feae4b7]: (4, ('0 0 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:36:52 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Searching for a device in session 48 and hctl ('39', '0', '0', 0) yield: sdh {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:36:52 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Connected to sdh using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'LNfcKqMo', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdh', 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 02:36:52 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 0.207s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:36:52 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] <== connect_volume: return (208ms) {'type': 'block', 'scsi_wwn': '23333363661396463', 'path': '/dev/sdh'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:36:52 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23333363661396463', 'path': '/dev/sdh'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 02:36:52 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:34:56Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-1202876429',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-1202876429',id=43,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:35:24Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='321111f935dd4e8aab4a083206ea9a7e',ramdisk_id='',reservation_id='r-p0t08coj',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-2108329302',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member'},tags=<?>,task_state='rescuing',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:35:25Z,user_data=None,user_id='73d05c48be5640c28455e8ba8b05ef31',uuid=bde1ef1e-163c-40eb-abb5-962b9064b7ab,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "8424518d-87b7-4fc4-8a58-1b4f094a5ca0", "address": "fa:16:3e:08:61:83", "network": {"id": "9a452414-d61f-4045-a6cc-9be93c892c4c", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [], "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "vif_mac": "fa:16:3e:08:61:83"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "321111f935dd4e8aab4a083206ea9a7e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8424518d-87", "ovs_interfaceid": "8424518d-87b7-4fc4-8a58-1b4f094a5ca0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:36:52 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Converting VIF {"id": "8424518d-87b7-4fc4-8a58-1b4f094a5ca0", "address": "fa:16:3e:08:61:83", "network": {"id": "9a452414-d61f-4045-a6cc-9be93c892c4c", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [], "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "vif_mac": "fa:16:3e:08:61:83"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "321111f935dd4e8aab4a083206ea9a7e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8424518d-87", "ovs_interfaceid": "8424518d-87b7-4fc4-8a58-1b4f094a5ca0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:36:52 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:08:61:83,bridge_name='br-int',has_traffic_filtering=True,id=8424518d-87b7-4fc4-8a58-1b4f094a5ca0,network=Network(9a452414-d61f-4045-a6cc-9be93c892c4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8424518d-87') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:36:52 user nova-compute[72351]: DEBUG nova.objects.instance [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lazy-loading 'pci_devices' on Instance uuid bde1ef1e-163c-40eb-abb5-962b9064b7ab {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:36:52 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:36:52 user nova-compute[72351]:   <uuid>bde1ef1e-163c-40eb-abb5-962b9064b7ab</uuid>
Aug 09 02:36:52 user nova-compute[72351]:   <name>instance-0000002b</name>
Aug 09 02:36:52 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:36:52 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:36:52 user nova-compute[72351]:   <metadata>
Aug 09 02:36:52 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:36:52 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:36:52 user nova-compute[72351]:       <nova:name>tempest-ServerBootFromVolumeStableRescueTest-server-1202876429</nova:name>
Aug 09 02:36:52 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:36:51</nova:creationTime>
Aug 09 02:36:52 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:36:52 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:36:52 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:36:52 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:36:52 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:36:52 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:36:52 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:36:52 user nova-compute[72351]:       <nova:owner>
Aug 09 02:36:52 user nova-compute[72351]:         <nova:user uuid="73d05c48be5640c28455e8ba8b05ef31">tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member</nova:user>
Aug 09 02:36:52 user nova-compute[72351]:         <nova:project uuid="321111f935dd4e8aab4a083206ea9a7e">tempest-ServerBootFromVolumeStableRescueTest-2108329302</nova:project>
Aug 09 02:36:52 user nova-compute[72351]:       </nova:owner>
Aug 09 02:36:52 user nova-compute[72351]:       <nova:root type="image" uuid="18ccda7f-31d3-4978-ab9d-4ed2f74eff94"/>
Aug 09 02:36:52 user nova-compute[72351]:       <nova:ports>
Aug 09 02:36:52 user nova-compute[72351]:         <nova:port uuid="8424518d-87b7-4fc4-8a58-1b4f094a5ca0">
Aug 09 02:36:52 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.3" ipVersion="4"/>
Aug 09 02:36:52 user nova-compute[72351]:         </nova:port>
Aug 09 02:36:52 user nova-compute[72351]:       </nova:ports>
Aug 09 02:36:52 user nova-compute[72351]:     </nova:instance>
Aug 09 02:36:52 user nova-compute[72351]:   </metadata>
Aug 09 02:36:52 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:36:52 user nova-compute[72351]:     <system>
Aug 09 02:36:52 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:36:52 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:36:52 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:36:52 user nova-compute[72351]:       <entry name="serial">bde1ef1e-163c-40eb-abb5-962b9064b7ab</entry>
Aug 09 02:36:52 user nova-compute[72351]:       <entry name="uuid">bde1ef1e-163c-40eb-abb5-962b9064b7ab</entry>
Aug 09 02:36:52 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:36:52 user nova-compute[72351]:     </system>
Aug 09 02:36:52 user nova-compute[72351]:   </sysinfo>
Aug 09 02:36:52 user nova-compute[72351]:   <os>
Aug 09 02:36:52 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:36:52 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:36:52 user nova-compute[72351]:   </os>
Aug 09 02:36:52 user nova-compute[72351]:   <features>
Aug 09 02:36:52 user nova-compute[72351]:     <acpi/>
Aug 09 02:36:52 user nova-compute[72351]:     <apic/>
Aug 09 02:36:52 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:36:52 user nova-compute[72351]:   </features>
Aug 09 02:36:52 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:36:52 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:36:52 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:36:52 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:36:52 user nova-compute[72351]:   </clock>
Aug 09 02:36:52 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:36:52 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:36:52 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:36:52 user nova-compute[72351]:   </cpu>
Aug 09 02:36:52 user nova-compute[72351]:   <devices>
Aug 09 02:36:52 user nova-compute[72351]:     <disk type="block" device="disk">
Aug 09 02:36:52 user nova-compute[72351]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:36:52 user nova-compute[72351]:       <source dev="/dev/sdh"/>
Aug 09 02:36:52 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:36:52 user nova-compute[72351]:       <serial>b5789e9a-6f06-4239-8523-0e8607f350df</serial>
Aug 09 02:36:52 user nova-compute[72351]:     </disk>
Aug 09 02:36:52 user nova-compute[72351]:     <disk type="file" device="disk">
Aug 09 02:36:52 user nova-compute[72351]:       <driver name="qemu" type="qcow2" cache="none"/>
Aug 09 02:36:52 user nova-compute[72351]:       <source file="/opt/stack/data/nova/instances/bde1ef1e-163c-40eb-abb5-962b9064b7ab/disk.rescue"/>
Aug 09 02:36:52 user nova-compute[72351]:       <target dev="vdb" bus="virtio"/>
Aug 09 02:36:52 user nova-compute[72351]:       <boot order="1"/>
Aug 09 02:36:52 user nova-compute[72351]:     </disk>
Aug 09 02:36:52 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:36:52 user nova-compute[72351]:       <mac address="fa:16:3e:08:61:83"/>
Aug 09 02:36:52 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:36:52 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:36:52 user nova-compute[72351]:       <target dev="tap8424518d-87"/>
Aug 09 02:36:52 user nova-compute[72351]:     </interface>
Aug 09 02:36:52 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:36:52 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/bde1ef1e-163c-40eb-abb5-962b9064b7ab/console.log" append="off"/>
Aug 09 02:36:52 user nova-compute[72351]:     </serial>
Aug 09 02:36:52 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:36:52 user nova-compute[72351]:     <video>
Aug 09 02:36:52 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:36:52 user nova-compute[72351]:     </video>
Aug 09 02:36:52 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:36:52 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:36:52 user nova-compute[72351]:     </rng>
Aug 09 02:36:52 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:36:52 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:36:52 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:36:52 user nova-compute[72351]:     </memballoon>
Aug 09 02:36:52 user nova-compute[72351]:   </devices>
Aug 09 02:36:52 user nova-compute[72351]: </domain>
Aug 09 02:36:52 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:36:52 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Instance destroyed successfully.
Aug 09 02:36:52 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:36:52 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:36:52 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] No VIF found with MAC fa:16:3e:08:61:83, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:36:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:36:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:36:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:36:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:36:53 user nova-compute[72351]: DEBUG nova.compute.manager [req-a9481163-e723-4411-ba06-be19df7f18bc req-cd5a7a44-7743-4233-abfa-8ddc49eddc88 service nova] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Received event network-vif-plugged-8424518d-87b7-4fc4-8a58-1b4f094a5ca0 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:36:53 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a9481163-e723-4411-ba06-be19df7f18bc req-cd5a7a44-7743-4233-abfa-8ddc49eddc88 service nova] Acquiring lock "bde1ef1e-163c-40eb-abb5-962b9064b7ab-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:36:53 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a9481163-e723-4411-ba06-be19df7f18bc req-cd5a7a44-7743-4233-abfa-8ddc49eddc88 service nova] Lock "bde1ef1e-163c-40eb-abb5-962b9064b7ab-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:36:53 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a9481163-e723-4411-ba06-be19df7f18bc req-cd5a7a44-7743-4233-abfa-8ddc49eddc88 service nova] Lock "bde1ef1e-163c-40eb-abb5-962b9064b7ab-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:36:53 user nova-compute[72351]: DEBUG nova.compute.manager [req-a9481163-e723-4411-ba06-be19df7f18bc req-cd5a7a44-7743-4233-abfa-8ddc49eddc88 service nova] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] No waiting events found dispatching network-vif-plugged-8424518d-87b7-4fc4-8a58-1b4f094a5ca0 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:36:53 user nova-compute[72351]: WARNING nova.compute.manager [req-a9481163-e723-4411-ba06-be19df7f18bc req-cd5a7a44-7743-4233-abfa-8ddc49eddc88 service nova] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Received unexpected event network-vif-plugged-8424518d-87b7-4fc4-8a58-1b4f094a5ca0 for instance with vm_state active and task_state rescuing.
Aug 09 02:36:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:36:55 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:36:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5f2bfff1-400c-4294-a952-f6a38aa65d56 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Acquiring lock "7b6f5d60-5f23-4ef1-b972-9ba7086f56cb" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:36:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5f2bfff1-400c-4294-a952-f6a38aa65d56 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "7b6f5d60-5f23-4ef1-b972-9ba7086f56cb" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:36:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5f2bfff1-400c-4294-a952-f6a38aa65d56 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Acquiring lock "7b6f5d60-5f23-4ef1-b972-9ba7086f56cb-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:36:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5f2bfff1-400c-4294-a952-f6a38aa65d56 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "7b6f5d60-5f23-4ef1-b972-9ba7086f56cb-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:36:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5f2bfff1-400c-4294-a952-f6a38aa65d56 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "7b6f5d60-5f23-4ef1-b972-9ba7086f56cb-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:36:55 user nova-compute[72351]: INFO nova.compute.manager [None req-5f2bfff1-400c-4294-a952-f6a38aa65d56 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Terminating instance
Aug 09 02:36:55 user nova-compute[72351]: DEBUG nova.compute.manager [None req-5f2bfff1-400c-4294-a952-f6a38aa65d56 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Start destroying the instance on the hypervisor. {{(pid=72351) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3124}}
Aug 09 02:36:55 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:36:55 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:36:56 user nova-compute[72351]: DEBUG nova.compute.manager [req-7403c7e1-ad92-4a0a-a5e6-8909c5164a15 req-768bbfbe-b3a4-4961-b721-5339a3eb78c0 service nova] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Received event network-vif-plugged-8424518d-87b7-4fc4-8a58-1b4f094a5ca0 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:36:56 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-7403c7e1-ad92-4a0a-a5e6-8909c5164a15 req-768bbfbe-b3a4-4961-b721-5339a3eb78c0 service nova] Acquiring lock "bde1ef1e-163c-40eb-abb5-962b9064b7ab-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:36:56 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-7403c7e1-ad92-4a0a-a5e6-8909c5164a15 req-768bbfbe-b3a4-4961-b721-5339a3eb78c0 service nova] Lock "bde1ef1e-163c-40eb-abb5-962b9064b7ab-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:36:56 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-7403c7e1-ad92-4a0a-a5e6-8909c5164a15 req-768bbfbe-b3a4-4961-b721-5339a3eb78c0 service nova] Lock "bde1ef1e-163c-40eb-abb5-962b9064b7ab-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:36:56 user nova-compute[72351]: DEBUG nova.compute.manager [req-7403c7e1-ad92-4a0a-a5e6-8909c5164a15 req-768bbfbe-b3a4-4961-b721-5339a3eb78c0 service nova] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] No waiting events found dispatching network-vif-plugged-8424518d-87b7-4fc4-8a58-1b4f094a5ca0 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:36:56 user nova-compute[72351]: WARNING nova.compute.manager [req-7403c7e1-ad92-4a0a-a5e6-8909c5164a15 req-768bbfbe-b3a4-4961-b721-5339a3eb78c0 service nova] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Received unexpected event network-vif-plugged-8424518d-87b7-4fc4-8a58-1b4f094a5ca0 for instance with vm_state active and task_state rescuing.
Aug 09 02:36:56 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:36:56 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:36:56 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Instance destroyed successfully.
Aug 09 02:36:56 user nova-compute[72351]: DEBUG nova.objects.instance [None req-5f2bfff1-400c-4294-a952-f6a38aa65d56 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lazy-loading 'resources' on Instance uuid 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:36:56 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-5f2bfff1-400c-4294-a952-f6a38aa65d56 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:30:09Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerRescueNegativeTestJSON-server-2047297087',display_name='tempest-ServerRescueNegativeTestJSON-server-2047297087',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serverrescuenegativetestjson-server-2047297087',id=34,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBPLsNIutXj2fb507aP4WngpUN1jl7pet8vFZkz28Sl/BaBQhyC41kBOXIZAx0v3md9QJVdNaIzkTSV1yroT5SZJzTuTaLLebnfImCGnkD5mtQK+8liTnNCRPuxSfIElfWA==',key_name='tempest-keypair-694850304',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:32:26Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='3aea1e66805146f7a8cd1d97e64a24af',ramdisk_id='',reservation_id='r-02psy90j',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerRescueNegativeTestJSON-289364438',owner_user_name='tempest-ServerRescueNegativeTestJSON-289364438-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:34:17Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='419b28d5a46c481db435715a30ab2614',uuid=7b6f5d60-5f23-4ef1-b972-9ba7086f56cb,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "c4a56310-1841-46ae-9a97-0421861e62c0", "address": "fa:16:3e:75:cf:ba", "network": {"id": "49e8751c-e119-430f-9110-3dc6069b1589", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-899275135-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.125", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3aea1e66805146f7a8cd1d97e64a24af", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc4a56310-18", "ovs_interfaceid": "c4a56310-1841-46ae-9a97-0421861e62c0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:36:56 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-5f2bfff1-400c-4294-a952-f6a38aa65d56 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Converting VIF {"id": "c4a56310-1841-46ae-9a97-0421861e62c0", "address": "fa:16:3e:75:cf:ba", "network": {"id": "49e8751c-e119-430f-9110-3dc6069b1589", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-899275135-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.125", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3aea1e66805146f7a8cd1d97e64a24af", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc4a56310-18", "ovs_interfaceid": "c4a56310-1841-46ae-9a97-0421861e62c0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:36:56 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-5f2bfff1-400c-4294-a952-f6a38aa65d56 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:75:cf:ba,bridge_name='br-int',has_traffic_filtering=True,id=c4a56310-1841-46ae-9a97-0421861e62c0,network=Network(49e8751c-e119-430f-9110-3dc6069b1589),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc4a56310-18') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:36:56 user nova-compute[72351]: DEBUG os_vif [None req-5f2bfff1-400c-4294-a952-f6a38aa65d56 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:75:cf:ba,bridge_name='br-int',has_traffic_filtering=True,id=c4a56310-1841-46ae-9a97-0421861e62c0,network=Network(49e8751c-e119-430f-9110-3dc6069b1589),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc4a56310-18') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:36:56 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:36:56 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapc4a56310-18, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:36:56 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:36:56 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:36:56 user nova-compute[72351]: INFO os_vif [None req-5f2bfff1-400c-4294-a952-f6a38aa65d56 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:75:cf:ba,bridge_name='br-int',has_traffic_filtering=True,id=c4a56310-1841-46ae-9a97-0421861e62c0,network=Network(49e8751c-e119-430f-9110-3dc6069b1589),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc4a56310-18')
Aug 09 02:36:56 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-5f2bfff1-400c-4294-a952-f6a38aa65d56 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Deleting instance files /opt/stack/data/nova/instances/7b6f5d60-5f23-4ef1-b972-9ba7086f56cb_del
Aug 09 02:36:56 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-5f2bfff1-400c-4294-a952-f6a38aa65d56 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Deletion of /opt/stack/data/nova/instances/7b6f5d60-5f23-4ef1-b972-9ba7086f56cb_del complete
Aug 09 02:36:56 user nova-compute[72351]: INFO nova.compute.manager [None req-5f2bfff1-400c-4294-a952-f6a38aa65d56 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Took 0.94 seconds to destroy the instance on the hypervisor.
Aug 09 02:36:56 user nova-compute[72351]: DEBUG oslo.service.loopingcall [None req-5f2bfff1-400c-4294-a952-f6a38aa65d56 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72351) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Aug 09 02:36:56 user nova-compute[72351]: DEBUG nova.compute.manager [-] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Deallocating network for instance {{(pid=72351) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2263}}
Aug 09 02:36:56 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] deallocate_for_instance() {{(pid=72351) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1795}}
Aug 09 02:36:57 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Removed pending event for bde1ef1e-163c-40eb-abb5-962b9064b7ab due to event {{(pid=72351) _event_emit_delayed /opt/stack/nova/nova/virt/libvirt/host.py:438}}
Aug 09 02:36:57 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548617.0846786, bde1ef1e-163c-40eb-abb5-962b9064b7ab => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:36:57 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] VM Resumed (Lifecycle Event)
Aug 09 02:36:57 user nova-compute[72351]: DEBUG nova.compute.manager [None req-36c5262f-b63d-4a56-8395-3474dd215c01 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:36:57 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:36:57 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: active, current task_state: rescuing, current DB power_state: 1, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:36:57 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] During sync_power_state the instance has a pending task (rescuing). Skip.
Aug 09 02:36:57 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548617.0869162, bde1ef1e-163c-40eb-abb5-962b9064b7ab => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:36:57 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] VM Started (Lifecycle Event)
Aug 09 02:36:57 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:36:57 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Synchronizing instance power state after lifecycle event "Started"; current vm_state: active, current task_state: rescuing, current DB power_state: 1, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:36:57 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:36:57 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Took 1.19 seconds to deallocate network for instance.
Aug 09 02:36:57 user nova-compute[72351]: DEBUG nova.compute.manager [req-b56e59f0-6388-4e10-9391-51fb241af8cb req-647f5609-fbe7-4ec3-8a49-24bd8a834dfb service nova] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Received event network-vif-deleted-c4a56310-1841-46ae-9a97-0421861e62c0 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:36:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5f2bfff1-400c-4294-a952-f6a38aa65d56 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:36:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5f2bfff1-400c-4294-a952-f6a38aa65d56 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:36:58 user nova-compute[72351]: DEBUG nova.compute.manager [req-98bc9dc6-1214-43ab-ae0c-f433a39bafed req-9b9f1198-8c2d-454b-8dfd-fc9157cbf256 service nova] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Received event network-vif-unplugged-c4a56310-1841-46ae-9a97-0421861e62c0 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:36:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-98bc9dc6-1214-43ab-ae0c-f433a39bafed req-9b9f1198-8c2d-454b-8dfd-fc9157cbf256 service nova] Acquiring lock "7b6f5d60-5f23-4ef1-b972-9ba7086f56cb-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:36:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-98bc9dc6-1214-43ab-ae0c-f433a39bafed req-9b9f1198-8c2d-454b-8dfd-fc9157cbf256 service nova] Lock "7b6f5d60-5f23-4ef1-b972-9ba7086f56cb-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:36:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-98bc9dc6-1214-43ab-ae0c-f433a39bafed req-9b9f1198-8c2d-454b-8dfd-fc9157cbf256 service nova] Lock "7b6f5d60-5f23-4ef1-b972-9ba7086f56cb-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:36:58 user nova-compute[72351]: DEBUG nova.compute.manager [req-98bc9dc6-1214-43ab-ae0c-f433a39bafed req-9b9f1198-8c2d-454b-8dfd-fc9157cbf256 service nova] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] No waiting events found dispatching network-vif-unplugged-c4a56310-1841-46ae-9a97-0421861e62c0 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:36:58 user nova-compute[72351]: WARNING nova.compute.manager [req-98bc9dc6-1214-43ab-ae0c-f433a39bafed req-9b9f1198-8c2d-454b-8dfd-fc9157cbf256 service nova] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Received unexpected event network-vif-unplugged-c4a56310-1841-46ae-9a97-0421861e62c0 for instance with vm_state deleted and task_state None.
Aug 09 02:36:58 user nova-compute[72351]: DEBUG nova.compute.manager [req-98bc9dc6-1214-43ab-ae0c-f433a39bafed req-9b9f1198-8c2d-454b-8dfd-fc9157cbf256 service nova] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Received event network-vif-plugged-c4a56310-1841-46ae-9a97-0421861e62c0 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:36:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-98bc9dc6-1214-43ab-ae0c-f433a39bafed req-9b9f1198-8c2d-454b-8dfd-fc9157cbf256 service nova] Acquiring lock "7b6f5d60-5f23-4ef1-b972-9ba7086f56cb-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:36:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-98bc9dc6-1214-43ab-ae0c-f433a39bafed req-9b9f1198-8c2d-454b-8dfd-fc9157cbf256 service nova] Lock "7b6f5d60-5f23-4ef1-b972-9ba7086f56cb-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:36:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-98bc9dc6-1214-43ab-ae0c-f433a39bafed req-9b9f1198-8c2d-454b-8dfd-fc9157cbf256 service nova] Lock "7b6f5d60-5f23-4ef1-b972-9ba7086f56cb-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:36:58 user nova-compute[72351]: DEBUG nova.compute.manager [req-98bc9dc6-1214-43ab-ae0c-f433a39bafed req-9b9f1198-8c2d-454b-8dfd-fc9157cbf256 service nova] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] No waiting events found dispatching network-vif-plugged-c4a56310-1841-46ae-9a97-0421861e62c0 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:36:58 user nova-compute[72351]: WARNING nova.compute.manager [req-98bc9dc6-1214-43ab-ae0c-f433a39bafed req-9b9f1198-8c2d-454b-8dfd-fc9157cbf256 service nova] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Received unexpected event network-vif-plugged-c4a56310-1841-46ae-9a97-0421861e62c0 for instance with vm_state deleted and task_state None.
Aug 09 02:36:58 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-5f2bfff1-400c-4294-a952-f6a38aa65d56 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:36:58 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-5f2bfff1-400c-4294-a952-f6a38aa65d56 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:36:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5f2bfff1-400c-4294-a952-f6a38aa65d56 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.438s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:36:58 user nova-compute[72351]: INFO nova.scheduler.client.report [None req-5f2bfff1-400c-4294-a952-f6a38aa65d56 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Deleted allocations for instance 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb
Aug 09 02:36:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-5f2bfff1-400c-4294-a952-f6a38aa65d56 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "7b6f5d60-5f23-4ef1-b972-9ba7086f56cb" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.906s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:36:58 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:36:59 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691548604.9135888, 8e970bdd-54a1-4058-b09d-784cb6b5b1a4 => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:36:59 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] VM Stopped (Lifecycle Event)
Aug 09 02:36:59 user nova-compute[72351]: DEBUG nova.compute.manager [None req-92d631f6-7826-4e63-a09e-b8333b54b0aa None None] [instance: 8e970bdd-54a1-4058-b09d-784cb6b5b1a4] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:37:01 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:37:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:37:05 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:37:05 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72351) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10461}}
Aug 09 02:37:06 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:37:06 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:37:10 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:37:11 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691548616.519368, 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:37:11 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] VM Stopped (Lifecycle Event)
Aug 09 02:37:11 user nova-compute[72351]: DEBUG nova.compute.manager [None req-cc390040-6469-475c-b82d-47b0dba33a44 None None] [instance: 7b6f5d60-5f23-4ef1-b972-9ba7086f56cb] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:37:11 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:37:11 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:37:11 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Starting heal instance info cache {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9842}}
Aug 09 02:37:11 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Rebuilding the list of instances to heal {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9846}}
Aug 09 02:37:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "refresh_cache-e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:37:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquired lock "refresh_cache-e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:37:12 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Forcefully refreshing network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1996}}
Aug 09 02:37:12 user nova-compute[72351]: DEBUG nova.objects.instance [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lazy-loading 'info_cache' on Instance uuid e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:37:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-2419fba4-76bd-48c0-a092-047fe0241865 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Acquiring lock "ad9b389e-ea11-4fd7-9233-a1be43052e60" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:37:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-2419fba4-76bd-48c0-a092-047fe0241865 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Lock "ad9b389e-ea11-4fd7-9233-a1be43052e60" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:37:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-2419fba4-76bd-48c0-a092-047fe0241865 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Acquiring lock "ad9b389e-ea11-4fd7-9233-a1be43052e60-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:37:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-2419fba4-76bd-48c0-a092-047fe0241865 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Lock "ad9b389e-ea11-4fd7-9233-a1be43052e60-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:37:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-2419fba4-76bd-48c0-a092-047fe0241865 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Lock "ad9b389e-ea11-4fd7-9233-a1be43052e60-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:37:12 user nova-compute[72351]: INFO nova.compute.manager [None req-2419fba4-76bd-48c0-a092-047fe0241865 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Terminating instance
Aug 09 02:37:12 user nova-compute[72351]: DEBUG nova.compute.manager [None req-2419fba4-76bd-48c0-a092-047fe0241865 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Start destroying the instance on the hypervisor. {{(pid=72351) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3124}}
Aug 09 02:37:12 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:37:12 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:37:12 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:37:12 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:37:12 user nova-compute[72351]: DEBUG nova.compute.manager [req-b0705726-4565-4406-9f3e-67e24b040c56 req-075945ae-7420-487d-993a-da5ad833f196 service nova] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Received event network-vif-unplugged-64d8b550-334e-42c4-b57f-145ccc73517c {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:37:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-b0705726-4565-4406-9f3e-67e24b040c56 req-075945ae-7420-487d-993a-da5ad833f196 service nova] Acquiring lock "ad9b389e-ea11-4fd7-9233-a1be43052e60-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:37:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-b0705726-4565-4406-9f3e-67e24b040c56 req-075945ae-7420-487d-993a-da5ad833f196 service nova] Lock "ad9b389e-ea11-4fd7-9233-a1be43052e60-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:37:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-b0705726-4565-4406-9f3e-67e24b040c56 req-075945ae-7420-487d-993a-da5ad833f196 service nova] Lock "ad9b389e-ea11-4fd7-9233-a1be43052e60-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:37:12 user nova-compute[72351]: DEBUG nova.compute.manager [req-b0705726-4565-4406-9f3e-67e24b040c56 req-075945ae-7420-487d-993a-da5ad833f196 service nova] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] No waiting events found dispatching network-vif-unplugged-64d8b550-334e-42c4-b57f-145ccc73517c {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:37:12 user nova-compute[72351]: DEBUG nova.compute.manager [req-b0705726-4565-4406-9f3e-67e24b040c56 req-075945ae-7420-487d-993a-da5ad833f196 service nova] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Received event network-vif-unplugged-64d8b550-334e-42c4-b57f-145ccc73517c for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:37:12 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Updating instance_info_cache with network_info: [{"id": "9481e457-9c23-4c45-b0bd-1faf63f8b7b8", "address": "fa:16:3e:69:03:30", "network": {"id": "49e8751c-e119-430f-9110-3dc6069b1589", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-899275135-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3aea1e66805146f7a8cd1d97e64a24af", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9481e457-9c", "ovs_interfaceid": "9481e457-9c23-4c45-b0bd-1faf63f8b7b8", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:37:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Releasing lock "refresh_cache-e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:37:12 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Updated the network info_cache for instance {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9913}}
Aug 09 02:37:12 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:37:12 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:37:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:37:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:37:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:37:12 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Auditing locally available compute resources for user (node: user) {{(pid=72351) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:37:13 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Instance destroyed successfully.
Aug 09 02:37:13 user nova-compute[72351]: DEBUG nova.objects.instance [None req-2419fba4-76bd-48c0-a092-047fe0241865 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Lazy-loading 'resources' on Instance uuid ad9b389e-ea11-4fd7-9233-a1be43052e60 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-2419fba4-76bd-48c0-a092-047fe0241865 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,compute_id=1,config_drive='',created_at=2023-08-09T02:33:27Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-ServerActionsV293TestJSON-server-2038847230',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serveractionsv293testjson-server-657819557',id=42,image_ref='',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBEunnOgCeXfv3GBp09pSPyhwCL3D6Q1n187gHEnIiV9PA3lK8lSVc8G3tp5ZU9m6m0OxdZGYGPGwrp313iNid8wPNsg46CMpUwvg2Jv3lbvp6Ufeyxq5cu1dosjL2Qlv6A==',key_name='tempest-keypair-461212954',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:35:57Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={rebuild='server'},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='624e34ea80f944509fde2cb5472ed203',ramdisk_id='',reservation_id='r-wzsh0h7v',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerActionsV293TestJSON-475025801',owner_user_name='tempest-ServerActionsV293TestJSON-475025801-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:35:57Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='801c60b1fab94f38b4d40222e9946f89',uuid=ad9b389e-ea11-4fd7-9233-a1be43052e60,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "64d8b550-334e-42c4-b57f-145ccc73517c", "address": "fa:16:3e:52:34:4d", "network": {"id": "f7042dd1-90f3-4ee1-99cd-e52f9264dc87", "bridge": "br-int", "label": "tempest-ServerActionsV293TestJSON-1595239201-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.209", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "624e34ea80f944509fde2cb5472ed203", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap64d8b550-33", "ovs_interfaceid": "64d8b550-334e-42c4-b57f-145ccc73517c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-2419fba4-76bd-48c0-a092-047fe0241865 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Converting VIF {"id": "64d8b550-334e-42c4-b57f-145ccc73517c", "address": "fa:16:3e:52:34:4d", "network": {"id": "f7042dd1-90f3-4ee1-99cd-e52f9264dc87", "bridge": "br-int", "label": "tempest-ServerActionsV293TestJSON-1595239201-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.209", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "624e34ea80f944509fde2cb5472ed203", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap64d8b550-33", "ovs_interfaceid": "64d8b550-334e-42c4-b57f-145ccc73517c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-2419fba4-76bd-48c0-a092-047fe0241865 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:52:34:4d,bridge_name='br-int',has_traffic_filtering=True,id=64d8b550-334e-42c4-b57f-145ccc73517c,network=Network(f7042dd1-90f3-4ee1-99cd-e52f9264dc87),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap64d8b550-33') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG os_vif [None req-2419fba4-76bd-48c0-a092-047fe0241865 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:52:34:4d,bridge_name='br-int',has_traffic_filtering=True,id=64d8b550-334e-42c4-b57f-145ccc73517c,network=Network(f7042dd1-90f3-4ee1-99cd-e52f9264dc87),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap64d8b550-33') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap64d8b550-33, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:37:13 user nova-compute[72351]: INFO os_vif [None req-2419fba4-76bd-48c0-a092-047fe0241865 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:52:34:4d,bridge_name='br-int',has_traffic_filtering=True,id=64d8b550-334e-42c4-b57f-145ccc73517c,network=Network(f7042dd1-90f3-4ee1-99cd-e52f9264dc87),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap64d8b550-33')
Aug 09 02:37:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-2419fba4-76bd-48c0-a092-047fe0241865 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-2419fba4-76bd-48c0-a092-047fe0241865 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'arRsVtvH', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sde'}, None), 'kwargs': True}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:37:13 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-2419fba4-76bd-48c0-a092-047fe0241865 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:37:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-2419fba4-76bd-48c0-a092-047fe0241865 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-2419fba4-76bd-48c0-a092-047fe0241865 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-2419fba4-76bd-48c0-a092-047fe0241865 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.034s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[5de7d091-3591-4ebb-ab7f-cc6064259c50]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.039s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[1633de1c-fbfc-4bb6-975e-0e87381a6266]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)\ntcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df (non-flash)\ntcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-2419fba4-76bd-48c0-a092-047fe0241865 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:37:13 user nova-compute[72351]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:37:13 user nova-compute[72351]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df (non-flash)
Aug 09 02:37:13 user nova-compute[72351]: tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)
Aug 09 02:37:13 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-2419fba4-76bd-48c0-a092-047fe0241865 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:37:13 user nova-compute[72351]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:37:13 user nova-compute[72351]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df (non-flash)
Aug 09 02:37:13 user nova-compute[72351]: tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d (non-flash)
Aug 09 02:37:13 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-2419fba4-76bd-48c0-a092-047fe0241865 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fcfbc63f9a0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d'): ({'sde'}, set())}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-2419fba4-76bd-48c0-a092-047fe0241865 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Removing single pathed devices sde {{(pid=72351) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.025s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[19130d11-11be-48f1-8a22-9f4b75517a72]: (4, ('path checker states:\nup                  4\n\npaths: 0\nbusy: False\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sde {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sde" returned: 0 in 0.035s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[86e1f33e-1cdd-4ec0-8bb0-9cf77cdb9fe6]: (4, ('ok\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-2419fba4-76bd-48c0-a092-047fe0241865 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Flushing IO for device /dev/sde {{(pid=72351) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sde {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sde" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[82618e33-e35a-4afe-acbe-4b49e48aedfa]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-2419fba4-76bd-48c0-a092-047fe0241865 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Remove SCSI device /dev/sde with /sys/block/sde/device/delete {{(pid=72351) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sde/device/delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7bef74f4-3bb3-4ad0-bff7-34156be31249/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sde/device/delete" returned: 0 in 0.067s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[067cd07c-8585-4729-97d4-487d98eb012a]: (4, ('1', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-2419fba4-76bd-48c0-a092-047fe0241865 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Checking to see if SCSI volumes sde have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-2419fba4-76bd-48c0-a092-047fe0241865 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] SCSI volumes sde have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-2419fba4-76bd-48c0-a092-047fe0241865 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d')] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.026s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[b892d049-6694-4650-99de-f58699db8955]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-2419fba4-76bd-48c0-a092-047fe0241865 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d -p 172.16.0.220:3260 --logout {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7bef74f4-3bb3-4ad0-bff7-34156be31249/disk --force-share --output=json" returned: 0 in 0.146s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7bef74f4-3bb3-4ad0-bff7-34156be31249/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d -p 172.16.0.220:3260 --logout" returned: 0 in 0.071s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ec55f139-21f7-46fc-bb69-98c75064281c]: (4, ('Logging out of session [sid: 50, target: iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d, portal: 172.16.0.220,3260]\nLogout of [sid: 50, target: iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-2419fba4-76bd-48c0-a092-047fe0241865 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 50, target: iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d, portal: 172.16.0.220,3260]
Aug 09 02:37:13 user nova-compute[72351]: Logout of [sid: 50, target: iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d, portal: 172.16.0.220,3260] successful.
Aug 09 02:37:13 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d -p 172.16.0.220:3260 --op delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:75a3424d-ab51-409b-b1e1-f702f4a84d3d -p 172.16.0.220:3260 --op delete" returned: 0 in 0.026s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[fe594657-2ffd-47fe-bc14-f272acea2e8a]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-2419fba4-76bd-48c0-a092-047fe0241865 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-2419fba4-76bd-48c0-a092-047fe0241865 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.429s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-2419fba4-76bd-48c0-a092-047fe0241865 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] <== disconnect_volume: return (431ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-2419fba4-76bd-48c0-a092-047fe0241865 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 02:37:13 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-2419fba4-76bd-48c0-a092-047fe0241865 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Deleting instance files /opt/stack/data/nova/instances/ad9b389e-ea11-4fd7-9233-a1be43052e60_del
Aug 09 02:37:13 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-2419fba4-76bd-48c0-a092-047fe0241865 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Deletion of /opt/stack/data/nova/instances/ad9b389e-ea11-4fd7-9233-a1be43052e60_del complete
Aug 09 02:37:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:37:13 user nova-compute[72351]: INFO nova.compute.manager [None req-2419fba4-76bd-48c0-a092-047fe0241865 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Took 1.41 seconds to destroy the instance on the hypervisor.
Aug 09 02:37:13 user nova-compute[72351]: DEBUG oslo.service.loopingcall [None req-2419fba4-76bd-48c0-a092-047fe0241865 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72351) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG nova.compute.manager [-] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Deallocating network for instance {{(pid=72351) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2263}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] deallocate_for_instance() {{(pid=72351) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1795}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7bef74f4-3bb3-4ad0-bff7-34156be31249/disk --force-share --output=json" returned: 0 in 0.170s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/41b5e305-3973-4937-b262-d32db1091e20/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/41b5e305-3973-4937-b262-d32db1091e20/disk --force-share --output=json" returned: 0 in 0.156s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/41b5e305-3973-4937-b262-d32db1091e20/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:37:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:37:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/41b5e305-3973-4937-b262-d32db1091e20/disk --force-share --output=json" returned: 0 in 0.164s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:37:14 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Error from libvirt while getting description of instance-0000002a: [Error Code 42] Domain not found: no domain with matching uuid 'ad9b389e-ea11-4fd7-9233-a1be43052e60' (instance-0000002a): libvirt.libvirtError: Domain not found: no domain with matching uuid 'ad9b389e-ea11-4fd7-9233-a1be43052e60' (instance-0000002a)
Aug 09 02:37:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:37:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json" returned: 0 in 0.150s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:37:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:37:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json" returned: 0 in 0.140s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:37:14 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdh (vda) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:37:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bde1ef1e-163c-40eb-abb5-962b9064b7ab/disk.rescue --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:37:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bde1ef1e-163c-40eb-abb5-962b9064b7ab/disk.rescue --force-share --output=json" returned: 0 in 0.136s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:37:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bde1ef1e-163c-40eb-abb5-962b9064b7ab/disk.rescue --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:37:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bde1ef1e-163c-40eb-abb5-962b9064b7ab/disk.rescue --force-share --output=json" returned: 0 in 0.136s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:37:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.rescue --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:37:14 user nova-compute[72351]: DEBUG nova.compute.manager [req-9777afb2-06f0-4315-ad25-8cac93534bdb req-c038c61a-a237-4ec5-9966-e942fd214ac2 service nova] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Received event network-vif-plugged-64d8b550-334e-42c4-b57f-145ccc73517c {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:37:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-9777afb2-06f0-4315-ad25-8cac93534bdb req-c038c61a-a237-4ec5-9966-e942fd214ac2 service nova] Acquiring lock "ad9b389e-ea11-4fd7-9233-a1be43052e60-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:37:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-9777afb2-06f0-4315-ad25-8cac93534bdb req-c038c61a-a237-4ec5-9966-e942fd214ac2 service nova] Lock "ad9b389e-ea11-4fd7-9233-a1be43052e60-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:37:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-9777afb2-06f0-4315-ad25-8cac93534bdb req-c038c61a-a237-4ec5-9966-e942fd214ac2 service nova] Lock "ad9b389e-ea11-4fd7-9233-a1be43052e60-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:37:14 user nova-compute[72351]: DEBUG nova.compute.manager [req-9777afb2-06f0-4315-ad25-8cac93534bdb req-c038c61a-a237-4ec5-9966-e942fd214ac2 service nova] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] No waiting events found dispatching network-vif-plugged-64d8b550-334e-42c4-b57f-145ccc73517c {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:37:14 user nova-compute[72351]: WARNING nova.compute.manager [req-9777afb2-06f0-4315-ad25-8cac93534bdb req-c038c61a-a237-4ec5-9966-e942fd214ac2 service nova] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Received unexpected event network-vif-plugged-64d8b550-334e-42c4-b57f-145ccc73517c for instance with vm_state active and task_state deleting.
Aug 09 02:37:14 user nova-compute[72351]: DEBUG nova.compute.manager [req-9777afb2-06f0-4315-ad25-8cac93534bdb req-c038c61a-a237-4ec5-9966-e942fd214ac2 service nova] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Received event network-vif-plugged-64d8b550-334e-42c4-b57f-145ccc73517c {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:37:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-9777afb2-06f0-4315-ad25-8cac93534bdb req-c038c61a-a237-4ec5-9966-e942fd214ac2 service nova] Acquiring lock "ad9b389e-ea11-4fd7-9233-a1be43052e60-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:37:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-9777afb2-06f0-4315-ad25-8cac93534bdb req-c038c61a-a237-4ec5-9966-e942fd214ac2 service nova] Lock "ad9b389e-ea11-4fd7-9233-a1be43052e60-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:37:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-9777afb2-06f0-4315-ad25-8cac93534bdb req-c038c61a-a237-4ec5-9966-e942fd214ac2 service nova] Lock "ad9b389e-ea11-4fd7-9233-a1be43052e60-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:37:14 user nova-compute[72351]: DEBUG nova.compute.manager [req-9777afb2-06f0-4315-ad25-8cac93534bdb req-c038c61a-a237-4ec5-9966-e942fd214ac2 service nova] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] No waiting events found dispatching network-vif-plugged-64d8b550-334e-42c4-b57f-145ccc73517c {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:37:14 user nova-compute[72351]: WARNING nova.compute.manager [req-9777afb2-06f0-4315-ad25-8cac93534bdb req-c038c61a-a237-4ec5-9966-e942fd214ac2 service nova] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Received unexpected event network-vif-plugged-64d8b550-334e-42c4-b57f-145ccc73517c for instance with vm_state active and task_state deleting.
Aug 09 02:37:14 user nova-compute[72351]: DEBUG nova.compute.manager [req-9777afb2-06f0-4315-ad25-8cac93534bdb req-c038c61a-a237-4ec5-9966-e942fd214ac2 service nova] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Received event network-vif-plugged-64d8b550-334e-42c4-b57f-145ccc73517c {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:37:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-9777afb2-06f0-4315-ad25-8cac93534bdb req-c038c61a-a237-4ec5-9966-e942fd214ac2 service nova] Acquiring lock "ad9b389e-ea11-4fd7-9233-a1be43052e60-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:37:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-9777afb2-06f0-4315-ad25-8cac93534bdb req-c038c61a-a237-4ec5-9966-e942fd214ac2 service nova] Lock "ad9b389e-ea11-4fd7-9233-a1be43052e60-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:37:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-9777afb2-06f0-4315-ad25-8cac93534bdb req-c038c61a-a237-4ec5-9966-e942fd214ac2 service nova] Lock "ad9b389e-ea11-4fd7-9233-a1be43052e60-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:37:14 user nova-compute[72351]: DEBUG nova.compute.manager [req-9777afb2-06f0-4315-ad25-8cac93534bdb req-c038c61a-a237-4ec5-9966-e942fd214ac2 service nova] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] No waiting events found dispatching network-vif-plugged-64d8b550-334e-42c4-b57f-145ccc73517c {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:37:14 user nova-compute[72351]: WARNING nova.compute.manager [req-9777afb2-06f0-4315-ad25-8cac93534bdb req-c038c61a-a237-4ec5-9966-e942fd214ac2 service nova] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Received unexpected event network-vif-plugged-64d8b550-334e-42c4-b57f-145ccc73517c for instance with vm_state active and task_state deleting.
Aug 09 02:37:14 user nova-compute[72351]: DEBUG nova.compute.manager [req-9777afb2-06f0-4315-ad25-8cac93534bdb req-c038c61a-a237-4ec5-9966-e942fd214ac2 service nova] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Received event network-vif-unplugged-64d8b550-334e-42c4-b57f-145ccc73517c {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:37:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-9777afb2-06f0-4315-ad25-8cac93534bdb req-c038c61a-a237-4ec5-9966-e942fd214ac2 service nova] Acquiring lock "ad9b389e-ea11-4fd7-9233-a1be43052e60-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:37:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-9777afb2-06f0-4315-ad25-8cac93534bdb req-c038c61a-a237-4ec5-9966-e942fd214ac2 service nova] Lock "ad9b389e-ea11-4fd7-9233-a1be43052e60-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:37:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-9777afb2-06f0-4315-ad25-8cac93534bdb req-c038c61a-a237-4ec5-9966-e942fd214ac2 service nova] Lock "ad9b389e-ea11-4fd7-9233-a1be43052e60-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:37:14 user nova-compute[72351]: DEBUG nova.compute.manager [req-9777afb2-06f0-4315-ad25-8cac93534bdb req-c038c61a-a237-4ec5-9966-e942fd214ac2 service nova] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] No waiting events found dispatching network-vif-unplugged-64d8b550-334e-42c4-b57f-145ccc73517c {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:37:14 user nova-compute[72351]: DEBUG nova.compute.manager [req-9777afb2-06f0-4315-ad25-8cac93534bdb req-c038c61a-a237-4ec5-9966-e942fd214ac2 service nova] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Received event network-vif-unplugged-64d8b550-334e-42c4-b57f-145ccc73517c for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:37:14 user nova-compute[72351]: DEBUG nova.compute.manager [req-9777afb2-06f0-4315-ad25-8cac93534bdb req-c038c61a-a237-4ec5-9966-e942fd214ac2 service nova] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Received event network-vif-plugged-64d8b550-334e-42c4-b57f-145ccc73517c {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:37:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-9777afb2-06f0-4315-ad25-8cac93534bdb req-c038c61a-a237-4ec5-9966-e942fd214ac2 service nova] Acquiring lock "ad9b389e-ea11-4fd7-9233-a1be43052e60-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:37:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-9777afb2-06f0-4315-ad25-8cac93534bdb req-c038c61a-a237-4ec5-9966-e942fd214ac2 service nova] Lock "ad9b389e-ea11-4fd7-9233-a1be43052e60-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:37:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-9777afb2-06f0-4315-ad25-8cac93534bdb req-c038c61a-a237-4ec5-9966-e942fd214ac2 service nova] Lock "ad9b389e-ea11-4fd7-9233-a1be43052e60-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:37:14 user nova-compute[72351]: DEBUG nova.compute.manager [req-9777afb2-06f0-4315-ad25-8cac93534bdb req-c038c61a-a237-4ec5-9966-e942fd214ac2 service nova] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] No waiting events found dispatching network-vif-plugged-64d8b550-334e-42c4-b57f-145ccc73517c {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:37:14 user nova-compute[72351]: WARNING nova.compute.manager [req-9777afb2-06f0-4315-ad25-8cac93534bdb req-c038c61a-a237-4ec5-9966-e942fd214ac2 service nova] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Received unexpected event network-vif-plugged-64d8b550-334e-42c4-b57f-145ccc73517c for instance with vm_state active and task_state deleting.
Aug 09 02:37:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.rescue --force-share --output=json" returned: 0 in 0.171s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:37:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.rescue --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:37:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk.rescue --force-share --output=json" returned: 0 in 0.151s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:37:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:37:15 user nova-compute[72351]: DEBUG nova.compute.manager [req-fab87802-f7e8-4a77-8979-5551c5d6de36 req-e7cbebf1-9d4f-4581-bfa7-0c11c7023371 service nova] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Received event network-vif-deleted-64d8b550-334e-42c4-b57f-145ccc73517c {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:37:15 user nova-compute[72351]: INFO nova.compute.manager [req-fab87802-f7e8-4a77-8979-5551c5d6de36 req-e7cbebf1-9d4f-4581-bfa7-0c11c7023371 service nova] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Neutron deleted interface 64d8b550-334e-42c4-b57f-145ccc73517c; detaching it from the instance and deleting it from the info cache
Aug 09 02:37:15 user nova-compute[72351]: DEBUG nova.network.neutron [req-fab87802-f7e8-4a77-8979-5551c5d6de36 req-e7cbebf1-9d4f-4581-bfa7-0c11c7023371 service nova] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:37:15 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:37:15 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Took 1.37 seconds to deallocate network for instance.
Aug 09 02:37:15 user nova-compute[72351]: DEBUG nova.compute.manager [req-fab87802-f7e8-4a77-8979-5551c5d6de36 req-e7cbebf1-9d4f-4581-bfa7-0c11c7023371 service nova] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Detach interface failed, port_id=64d8b550-334e-42c4-b57f-145ccc73517c, reason: Instance ad9b389e-ea11-4fd7-9233-a1be43052e60 could not be found. {{(pid=72351) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10866}}
Aug 09 02:37:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:37:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:37:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d/disk --force-share --output=json" returned: 0 in 0.142s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:37:15 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdg (vda) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:37:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:37:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json" returned: 0 in 0.156s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:37:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:37:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json" returned: 0 in 0.176s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:37:15 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdd (vda) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:37:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:37:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json" returned: 0 in 0.185s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:37:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:37:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json" returned: 0 in 0.291s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:37:16 user nova-compute[72351]: INFO nova.compute.manager [None req-2419fba4-76bd-48c0-a092-047fe0241865 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Took 1.10 seconds to detach 1 volumes for instance.
Aug 09 02:37:16 user nova-compute[72351]: DEBUG nova.compute.manager [None req-2419fba4-76bd-48c0-a092-047fe0241865 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Deleting volume: 75a3424d-ab51-409b-b1e1-f702f4a84d3d {{(pid=72351) _cleanup_volumes /opt/stack/nova/nova/compute/manager.py:3221}}
Aug 09 02:37:16 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:37:16 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:37:16 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Hypervisor/Node resource view: name=user free_ram=7709MB free_disk=25.060977935791016GB free_vcpus=3 pci_devices=[{"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72351) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Aug 09 02:37:16 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:37:16 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:37:16 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-2419fba4-76bd-48c0-a092-047fe0241865 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:37:16 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:37:16 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance eb86cb49-2018-42c2-bd97-c662b829775d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:37:16 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:37:16 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance a7f3d076-864b-4b68-9207-214ce2c21444 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:37:16 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 957c1510-1320-4492-9052-a58ff540319d actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:37:16 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 7bef74f4-3bb3-4ad0-bff7-34156be31249 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:37:16 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 41b5e305-3973-4937-b262-d32db1091e20 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:37:16 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 80e87454-90b9-4168-bf00-b7d93b245526 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 2048, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:37:16 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance ad9b389e-ea11-4fd7-9233-a1be43052e60 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:37:16 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance bde1ef1e-163c-40eb-abb5-962b9064b7ab actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:37:16 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Total usable vcpus: 12, total allocated vcpus: 10 {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Aug 09 02:37:16 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Final resource view: name=user phys_ram=16011MB used_ram=7168MB phys_disk=40GB used_disk=6GB total_vcpus=12 used_vcpus=10 pci_stats=[] {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Aug 09 02:37:17 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:37:17 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:37:17 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Compute_service record updated for user:user {{(pid=72351) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Aug 09 02:37:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.494s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:37:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-2419fba4-76bd-48c0-a092-047fe0241865 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.444s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:37:17 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-2419fba4-76bd-48c0-a092-047fe0241865 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:37:17 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-2419fba4-76bd-48c0-a092-047fe0241865 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:37:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-2419fba4-76bd-48c0-a092-047fe0241865 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.415s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:37:17 user nova-compute[72351]: INFO nova.scheduler.client.report [None req-2419fba4-76bd-48c0-a092-047fe0241865 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Deleted allocations for instance ad9b389e-ea11-4fd7-9233-a1be43052e60
Aug 09 02:37:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-2419fba4-76bd-48c0-a092-047fe0241865 tempest-ServerActionsV293TestJSON-475025801 tempest-ServerActionsV293TestJSON-475025801-project-member] Lock "ad9b389e-ea11-4fd7-9233-a1be43052e60" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 5.439s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:37:18 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:37:18 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:37:18 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:37:18 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:37:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:37:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:37:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-47e2d464-379c-49f9-9463-712069cbb0fe tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Acquiring lock "7bef74f4-3bb3-4ad0-bff7-34156be31249" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:37:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-47e2d464-379c-49f9-9463-712069cbb0fe tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "7bef74f4-3bb3-4ad0-bff7-34156be31249" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:37:20 user nova-compute[72351]: DEBUG nova.objects.instance [None req-47e2d464-379c-49f9-9463-712069cbb0fe tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lazy-loading 'flavor' on Instance uuid 7bef74f4-3bb3-4ad0-bff7-34156be31249 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:37:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-47e2d464-379c-49f9-9463-712069cbb0fe tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "7bef74f4-3bb3-4ad0-bff7-34156be31249" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.058s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:37:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-47e2d464-379c-49f9-9463-712069cbb0fe tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Acquiring lock "7bef74f4-3bb3-4ad0-bff7-34156be31249" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:37:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-47e2d464-379c-49f9-9463-712069cbb0fe tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "7bef74f4-3bb3-4ad0-bff7-34156be31249" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:37:20 user nova-compute[72351]: INFO nova.compute.manager [None req-47e2d464-379c-49f9-9463-712069cbb0fe tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Attaching volume aab68f72-1237-4ed0-89b4-405eb30a174e to /dev/vdb
Aug 09 02:37:20 user nova-compute[72351]: DEBUG os_brick.utils [None req-47e2d464-379c-49f9-9463-712069cbb0fe tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:37:20 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:37:20 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:37:20 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[36553bd5-40ea-4ffa-b6ce-8ae113f9dd13]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:a52269602ff4\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:37:20 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:37:20 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.016s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:37:20 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[949e4b89-e625-43ec-ad6f-4819531d318b]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:37:20 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:37:20 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.016s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:37:20 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[98f29076-4346-4ecc-8704-eebf7583c2e6]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:37:20 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[c8bc1db5-4a48-4d5a-975f-f6f73173038a]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:37:20 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-47e2d464-379c-49f9-9463-712069cbb0fe tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Running cmd (subprocess): nvme version {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:37:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-47e2d464-379c-49f9-9463-712069cbb0fe tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] 'nvme version' failed. Not Retrying. {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:37:21 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.nvmeof [None req-47e2d464-379c-49f9-9463-712069cbb0fe tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] nvme not present on system {{(pid=72351) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Aug 09 02:37:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:37:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:37:21 user nova-compute[72351]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Aug 09 02:37:21 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[43116b86-d123-430d-ade8-c4d55cf7fdae]: (4, '') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:37:21 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-47e2d464-379c-49f9-9463-712069cbb0fe tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72351) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Aug 09 02:37:21 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-47e2d464-379c-49f9-9463-712069cbb0fe tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Aug 09 02:37:21 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-47e2d464-379c-49f9-9463-712069cbb0fe tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] LIGHTOS: no hostnqn found. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Aug 09 02:37:21 user nova-compute[72351]: DEBUG os_brick.utils [None req-47e2d464-379c-49f9-9463-712069cbb0fe tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] <== get_connector_properties: return (134ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:a52269602ff4', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:37:21 user nova-compute[72351]: DEBUG nova.virt.block_device [None req-47e2d464-379c-49f9-9463-712069cbb0fe tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Updating existing volume attachment record: db503ef0-557e-49e3-98de-c4e93d1dbeae {{(pid=72351) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Aug 09 02:37:23 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:37:23 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:37:28 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691548633.152585, ad9b389e-ea11-4fd7-9233-a1be43052e60 => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:37:28 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] VM Stopped (Lifecycle Event)
Aug 09 02:37:28 user nova-compute[72351]: DEBUG nova.compute.manager [None req-fc3c6b69-5368-4f00-affb-ce2b9b884b40 None None] [instance: ad9b389e-ea11-4fd7-9233-a1be43052e60] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:37:28 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:37:28 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:37:33 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:37:33 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:37:36 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-47e2d464-379c-49f9-9463-712069cbb0fe tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 02:37:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-47e2d464-379c-49f9-9463-712069cbb0fe tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:aab68f72-1237-4ed0-89b4-405eb30a174e'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'HkfCniPY', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:37:36 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-47e2d464-379c-49f9-9463-712069cbb0fe tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:37:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-47e2d464-379c-49f9-9463-712069cbb0fe tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:37:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-47e2d464-379c-49f9-9463-712069cbb0fe tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:37:36 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-47e2d464-379c-49f9-9463-712069cbb0fe tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:37:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-47e2d464-379c-49f9-9463-712069cbb0fe tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:aab68f72-1237-4ed0-89b4-405eb30a174e" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:37:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-47e2d464-379c-49f9-9463-712069cbb0fe tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:aab68f72-1237-4ed0-89b4-405eb30a174e" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:37:36 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-47e2d464-379c-49f9-9463-712069cbb0fe tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 02:37:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:aab68f72-1237-4ed0-89b4-405eb30a174e -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:37:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:aab68f72-1237-4ed0-89b4-405eb30a174e -p 172.16.0.220:3260" returned: 21 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:37:36 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[f218379f-f06d-4e14-b749-47d9760de900]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:37:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-47e2d464-379c-49f9-9463-712069cbb0fe tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Aug 09 02:37:36 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:37:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:aab68f72-1237-4ed0-89b4-405eb30a174e -p 172.16.0.220:3260 --interface default --op new {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:37:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:aab68f72-1237-4ed0-89b4-405eb30a174e -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:37:36 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[f7259fed-2bfa-4c4f-a92d-9bd611b1f98f]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:aab68f72-1237-4ed0-89b4-405eb30a174e] added\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:37:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-47e2d464-379c-49f9-9463-712069cbb0fe tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:aab68f72-1237-4ed0-89b4-405eb30a174e] added
Aug 09 02:37:36 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:37:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:aab68f72-1237-4ed0-89b4-405eb30a174e -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:37:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:aab68f72-1237-4ed0-89b4-405eb30a174e -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:37:36 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[49c27532-b8a7-4b50-bc98-893abae76473]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:37:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-47e2d464-379c-49f9-9463-712069cbb0fe tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:37:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:aab68f72-1237-4ed0-89b4-405eb30a174e -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:37:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:aab68f72-1237-4ed0-89b4-405eb30a174e -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:37:36 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[5083d05e-450e-40b8-8ad2-69ddba765207]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:37:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-47e2d464-379c-49f9-9463-712069cbb0fe tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:37:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:aab68f72-1237-4ed0-89b4-405eb30a174e -p 172.16.0.220:3260 --op update -n node.session.auth.username -v HkfCniPY {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:37:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:aab68f72-1237-4ed0-89b4-405eb30a174e -p 172.16.0.220:3260 --op update -n node.session.auth.username -v HkfCniPY" returned: 0 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:37:36 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[0b8c678a-2f5e-4017-ae54-0ac5c8001bc8]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:37:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-47e2d464-379c-49f9-9463-712069cbb0fe tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'HkfCniPY'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:37:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:aab68f72-1237-4ed0-89b4-405eb30a174e -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:37:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:aab68f72-1237-4ed0-89b4-405eb30a174e -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:37:36 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e0f0dc01-2e36-4999-8030-7d9813678ad8]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:37:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-47e2d464-379c-49f9-9463-712069cbb0fe tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:37:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:37:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.028s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:37:36 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[1dbee5ad-e133-41f9-97b2-8af41a676429]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)\ntcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:37:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-47e2d464-379c-49f9-9463-712069cbb0fe tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:37:36 user nova-compute[72351]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:37:36 user nova-compute[72351]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df (non-flash)
Aug 09 02:37:36 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:37:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-47e2d464-379c-49f9-9463-712069cbb0fe tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:37:36 user nova-compute[72351]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:37:36 user nova-compute[72351]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df (non-flash)
Aug 09 02:37:36 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:37:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:aab68f72-1237-4ed0-89b4-405eb30a174e -p 172.16.0.220:3260 --login {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:37:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:aab68f72-1237-4ed0-89b4-405eb30a174e -p 172.16.0.220:3260 --login" returned: 0 in 0.071s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:37:36 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[5bad69ea-c952-4442-8587-7ac3253b74af]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:aab68f72-1237-4ed0-89b4-405eb30a174e, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:aab68f72-1237-4ed0-89b4-405eb30a174e, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:37:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-47e2d464-379c-49f9-9463-712069cbb0fe tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:aab68f72-1237-4ed0-89b4-405eb30a174e, portal: 172.16.0.220,3260]
Aug 09 02:37:36 user nova-compute[72351]: Login to [iface: default, target: iqn.2010-10.org.openstack:aab68f72-1237-4ed0-89b4-405eb30a174e, portal: 172.16.0.220,3260] successful.
Aug 09 02:37:36 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:37:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:aab68f72-1237-4ed0-89b4-405eb30a174e -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:37:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:aab68f72-1237-4ed0-89b4-405eb30a174e -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:37:36 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[4698ec5c-150e-47d8-9c1a-63e6236d9597]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:37:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-47e2d464-379c-49f9-9463-712069cbb0fe tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:37:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:37:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:37:36 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[0580eff1-bf9b-430d-9a57-fb2110202e95]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)\ntcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df (non-flash)\ntcp: [55] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:aab68f72-1237-4ed0-89b4-405eb30a174e (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:37:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-47e2d464-379c-49f9-9463-712069cbb0fe tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:37:36 user nova-compute[72351]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:37:36 user nova-compute[72351]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df (non-flash)
Aug 09 02:37:36 user nova-compute[72351]: tcp: [55] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:aab68f72-1237-4ed0-89b4-405eb30a174e (non-flash)
Aug 09 02:37:36 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:37:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-47e2d464-379c-49f9-9463-712069cbb0fe tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:37:36 user nova-compute[72351]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:37:36 user nova-compute[72351]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df (non-flash)
Aug 09 02:37:36 user nova-compute[72351]: tcp: [55] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:aab68f72-1237-4ed0-89b4-405eb30a174e (non-flash)
Aug 09 02:37:36 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:37:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-47e2d464-379c-49f9-9463-712069cbb0fe tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:aab68f72-1237-4ed0-89b4-405eb30a174e" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.315s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:37:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-47e2d464-379c-49f9-9463-712069cbb0fe tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 02:37:36 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-47e2d464-379c-49f9-9463-712069cbb0fe tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] HCTL ('33', '-', '-', 0) found on session 55 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 02:37:36 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-47e2d464-379c-49f9-9463-712069cbb0fe tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 02:37:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:37:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:37:36 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[4138dc85-71ee-4b25-a7c6-ad6fd6afe81f]: (4, ('- - 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:37:36 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-47e2d464-379c-49f9-9463-712069cbb0fe tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Searching for a device in session 55 and hctl ['33', '*', '*', 0] yield: None {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:37:37 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-47e2d464-379c-49f9-9463-712069cbb0fe tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Searching for a device in session 55 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:37:37 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-47e2d464-379c-49f9-9463-712069cbb0fe tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Connected to sdb using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'HkfCniPY', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 02:37:37 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-47e2d464-379c-49f9-9463-712069cbb0fe tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.364s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:37:37 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-47e2d464-379c-49f9-9463-712069cbb0fe tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] <== connect_volume: return (1366ms) {'type': 'block', 'scsi_wwn': '26463343133366664', 'path': '/dev/sdb'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:37:37 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-47e2d464-379c-49f9-9463-712069cbb0fe tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26463343133366664', 'path': '/dev/sdb'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 02:37:37 user nova-compute[72351]: DEBUG nova.objects.instance [None req-47e2d464-379c-49f9-9463-712069cbb0fe tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lazy-loading 'flavor' on Instance uuid 7bef74f4-3bb3-4ad0-bff7-34156be31249 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:37:37 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-47e2d464-379c-49f9-9463-712069cbb0fe tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] attach device xml: <disk type="block" device="disk">
Aug 09 02:37:37 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:37:37 user nova-compute[72351]:   <source dev="/dev/sdb"/>
Aug 09 02:37:37 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:37:37 user nova-compute[72351]:   <serial>aab68f72-1237-4ed0-89b4-405eb30a174e</serial>
Aug 09 02:37:37 user nova-compute[72351]: </disk>
Aug 09 02:37:37 user nova-compute[72351]:  {{(pid=72351) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Aug 09 02:37:37 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-47e2d464-379c-49f9-9463-712069cbb0fe tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:37:37 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-47e2d464-379c-49f9-9463-712069cbb0fe tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:37:37 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-47e2d464-379c-49f9-9463-712069cbb0fe tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] No VIF found with MAC fa:16:3e:b4:8f:2e, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:37:38 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:37:38 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-47e2d464-379c-49f9-9463-712069cbb0fe tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "7bef74f4-3bb3-4ad0-bff7-34156be31249" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 17.318s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:37:38 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:37:40 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-bcc0a9c5-bbc5-4273-99bb-6d6a076a1e03 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Acquiring lock "7bef74f4-3bb3-4ad0-bff7-34156be31249" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:37:40 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-bcc0a9c5-bbc5-4273-99bb-6d6a076a1e03 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "7bef74f4-3bb3-4ad0-bff7-34156be31249" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:37:40 user nova-compute[72351]: INFO nova.compute.manager [None req-bcc0a9c5-bbc5-4273-99bb-6d6a076a1e03 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Detaching volume aab68f72-1237-4ed0-89b4-405eb30a174e
Aug 09 02:37:40 user nova-compute[72351]: INFO nova.virt.block_device [None req-bcc0a9c5-bbc5-4273-99bb-6d6a076a1e03 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Attempting to driver detach volume aab68f72-1237-4ed0-89b4-405eb30a174e from mountpoint /dev/vdb
Aug 09 02:37:40 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-bcc0a9c5-bbc5-4273-99bb-6d6a076a1e03 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Attempting to detach device vdb from instance 7bef74f4-3bb3-4ad0-bff7-34156be31249 from the persistent domain config. {{(pid=72351) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2477}}
Aug 09 02:37:40 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-bcc0a9c5-bbc5-4273-99bb-6d6a076a1e03 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 02:37:40 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:37:40 user nova-compute[72351]:   <source dev="/dev/sdb"/>
Aug 09 02:37:40 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:37:40 user nova-compute[72351]:   <serial>aab68f72-1237-4ed0-89b4-405eb30a174e</serial>
Aug 09 02:37:40 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Aug 09 02:37:40 user nova-compute[72351]: </disk>
Aug 09 02:37:40 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 02:37:40 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-bcc0a9c5-bbc5-4273-99bb-6d6a076a1e03 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Successfully detached device vdb from instance 7bef74f4-3bb3-4ad0-bff7-34156be31249 from the persistent domain config.
Aug 09 02:37:40 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-bcc0a9c5-bbc5-4273-99bb-6d6a076a1e03 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 7bef74f4-3bb3-4ad0-bff7-34156be31249 from the live domain config. {{(pid=72351) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2513}}
Aug 09 02:37:40 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-bcc0a9c5-bbc5-4273-99bb-6d6a076a1e03 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 02:37:40 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:37:40 user nova-compute[72351]:   <source dev="/dev/sdb"/>
Aug 09 02:37:40 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:37:40 user nova-compute[72351]:   <serial>aab68f72-1237-4ed0-89b4-405eb30a174e</serial>
Aug 09 02:37:40 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Aug 09 02:37:40 user nova-compute[72351]: </disk>
Aug 09 02:37:40 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 02:37:41 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Received event <DeviceRemovedEvent: 1691548661.544839, 7bef74f4-3bb3-4ad0-bff7-34156be31249 => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=72351) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2360}}
Aug 09 02:37:41 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-bcc0a9c5-bbc5-4273-99bb-6d6a076a1e03 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 7bef74f4-3bb3-4ad0-bff7-34156be31249 {{(pid=72351) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2589}}
Aug 09 02:37:41 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-bcc0a9c5-bbc5-4273-99bb-6d6a076a1e03 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Successfully detached device vdb from instance 7bef74f4-3bb3-4ad0-bff7-34156be31249 from the live domain config.
Aug 09 02:37:41 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-bcc0a9c5-bbc5-4273-99bb-6d6a076a1e03 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 02:37:41 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-bcc0a9c5-bbc5-4273-99bb-6d6a076a1e03 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:aab68f72-1237-4ed0-89b4-405eb30a174e'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'HkfCniPY', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb'}, None), 'kwargs': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:37:41 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-bcc0a9c5-bbc5-4273-99bb-6d6a076a1e03 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:37:41 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-bcc0a9c5-bbc5-4273-99bb-6d6a076a1e03 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:37:41 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-bcc0a9c5-bbc5-4273-99bb-6d6a076a1e03 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:37:41 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-bcc0a9c5-bbc5-4273-99bb-6d6a076a1e03 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:aab68f72-1237-4ed0-89b4-405eb30a174e', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 02:37:41 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:37:41 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:37:41 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[769589fb-e583-4639-baaa-c4aa874be009]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:aab68f72-1237-4ed0-89b4-405eb30a174e\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:37:41 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:37:41 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:37:41 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[57eff34e-6dff-41c6-9381-ba9f23b54d99]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)\ntcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df (non-flash)\ntcp: [55] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:aab68f72-1237-4ed0-89b4-405eb30a174e (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:37:41 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-bcc0a9c5-bbc5-4273-99bb-6d6a076a1e03 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:37:41 user nova-compute[72351]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:37:41 user nova-compute[72351]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df (non-flash)
Aug 09 02:37:41 user nova-compute[72351]: tcp: [55] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:aab68f72-1237-4ed0-89b4-405eb30a174e (non-flash)
Aug 09 02:37:41 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:37:41 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-bcc0a9c5-bbc5-4273-99bb-6d6a076a1e03 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:37:41 user nova-compute[72351]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:37:41 user nova-compute[72351]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df (non-flash)
Aug 09 02:37:41 user nova-compute[72351]: tcp: [55] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:aab68f72-1237-4ed0-89b4-405eb30a174e (non-flash)
Aug 09 02:37:41 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:37:41 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-bcc0a9c5-bbc5-4273-99bb-6d6a076a1e03 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fcffc0d9cf0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:aab68f72-1237-4ed0-89b4-405eb30a174e'): ({'sdb'}, set())}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 02:37:41 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-bcc0a9c5-bbc5-4273-99bb-6d6a076a1e03 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Removing single pathed devices sdb {{(pid=72351) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Aug 09 02:37:41 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:37:41 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:37:41 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[76da5df0-33d0-4adc-bb44-4c761384f254]: (4, ('path checker states:\nup                  4\n\npaths: 0\nbusy: False\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:37:41 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:37:41 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:37:41 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[4ac91607-95cc-4114-b821-5c848a411a33]: (4, ('ok\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:37:41 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-bcc0a9c5-bbc5-4273-99bb-6d6a076a1e03 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Flushing IO for device /dev/sdb {{(pid=72351) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Aug 09 02:37:41 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:37:41 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.015s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:37:41 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[aa7fca9a-6002-49dd-be29-b15b39bd1384]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:37:41 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-bcc0a9c5-bbc5-4273-99bb-6d6a076a1e03 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=72351) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Aug 09 02:37:41 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:37:41 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.044s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:37:41 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[c4aa8821-2d29-46fa-bd23-b9611926c9c6]: (4, ('1', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:37:41 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-bcc0a9c5-bbc5-4273-99bb-6d6a076a1e03 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Aug 09 02:37:41 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-bcc0a9c5-bbc5-4273-99bb-6d6a076a1e03 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] SCSI volumes sdb have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Aug 09 02:37:41 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-bcc0a9c5-bbc5-4273-99bb-6d6a076a1e03 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:aab68f72-1237-4ed0-89b4-405eb30a174e')] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 02:37:41 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:aab68f72-1237-4ed0-89b4-405eb30a174e -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:37:41 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:aab68f72-1237-4ed0-89b4-405eb30a174e -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.030s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:37:41 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[c43d61a1-0bfa-414b-8636-7da5358ecef5]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:37:41 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-bcc0a9c5-bbc5-4273-99bb-6d6a076a1e03 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:37:41 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:aab68f72-1237-4ed0-89b4-405eb30a174e -p 172.16.0.220:3260 --logout {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:37:41 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:aab68f72-1237-4ed0-89b4-405eb30a174e -p 172.16.0.220:3260 --logout" returned: 0 in 0.070s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:37:41 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[eefaf617-7759-41f4-9ada-d2074d996865]: (4, ('Logging out of session [sid: 55, target: iqn.2010-10.org.openstack:aab68f72-1237-4ed0-89b4-405eb30a174e, portal: 172.16.0.220,3260]\nLogout of [sid: 55, target: iqn.2010-10.org.openstack:aab68f72-1237-4ed0-89b4-405eb30a174e, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:37:41 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-bcc0a9c5-bbc5-4273-99bb-6d6a076a1e03 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 55, target: iqn.2010-10.org.openstack:aab68f72-1237-4ed0-89b4-405eb30a174e, portal: 172.16.0.220,3260]
Aug 09 02:37:41 user nova-compute[72351]: Logout of [sid: 55, target: iqn.2010-10.org.openstack:aab68f72-1237-4ed0-89b4-405eb30a174e, portal: 172.16.0.220,3260] successful.
Aug 09 02:37:41 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:37:41 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:aab68f72-1237-4ed0-89b4-405eb30a174e -p 172.16.0.220:3260 --op delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:37:41 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:aab68f72-1237-4ed0-89b4-405eb30a174e -p 172.16.0.220:3260 --op delete" returned: 0 in 0.031s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:37:41 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[48a79e4d-8a99-44c2-8fe8-4d9c1821ff12]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:37:41 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-bcc0a9c5-bbc5-4273-99bb-6d6a076a1e03 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:37:41 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-bcc0a9c5-bbc5-4273-99bb-6d6a076a1e03 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.315s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:37:41 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-bcc0a9c5-bbc5-4273-99bb-6d6a076a1e03 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] <== disconnect_volume: return (316ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:37:41 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-bcc0a9c5-bbc5-4273-99bb-6d6a076a1e03 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 02:37:41 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:37:43 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:37:43 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:37:46 user nova-compute[72351]: DEBUG nova.objects.instance [None req-bcc0a9c5-bbc5-4273-99bb-6d6a076a1e03 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lazy-loading 'flavor' on Instance uuid 7bef74f4-3bb3-4ad0-bff7-34156be31249 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:37:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-bcc0a9c5-bbc5-4273-99bb-6d6a076a1e03 tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "7bef74f4-3bb3-4ad0-bff7-34156be31249" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 5.446s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:37:48 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:37:49 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-46cb4596-989a-4c58-abf6-e62295b9c7bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Acquiring lock "80e87454-90b9-4168-bf00-b7d93b245526" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:37:49 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-46cb4596-989a-4c58-abf6-e62295b9c7bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "80e87454-90b9-4168-bf00-b7d93b245526" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:37:49 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-46cb4596-989a-4c58-abf6-e62295b9c7bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Acquiring lock "80e87454-90b9-4168-bf00-b7d93b245526-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:37:49 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-46cb4596-989a-4c58-abf6-e62295b9c7bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "80e87454-90b9-4168-bf00-b7d93b245526-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:37:49 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-46cb4596-989a-4c58-abf6-e62295b9c7bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "80e87454-90b9-4168-bf00-b7d93b245526-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:37:49 user nova-compute[72351]: INFO nova.compute.manager [None req-46cb4596-989a-4c58-abf6-e62295b9c7bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Terminating instance
Aug 09 02:37:49 user nova-compute[72351]: DEBUG nova.compute.manager [None req-46cb4596-989a-4c58-abf6-e62295b9c7bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Start destroying the instance on the hypervisor. {{(pid=72351) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3124}}
Aug 09 02:37:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:37:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:37:49 user nova-compute[72351]: DEBUG nova.compute.manager [req-f9baa5a9-5713-4031-849c-8f08f16b126d req-04013d23-3447-4136-8be8-fa3fb358477c service nova] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Received event network-vif-unplugged-fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:37:49 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-f9baa5a9-5713-4031-849c-8f08f16b126d req-04013d23-3447-4136-8be8-fa3fb358477c service nova] Acquiring lock "80e87454-90b9-4168-bf00-b7d93b245526-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:37:49 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-f9baa5a9-5713-4031-849c-8f08f16b126d req-04013d23-3447-4136-8be8-fa3fb358477c service nova] Lock "80e87454-90b9-4168-bf00-b7d93b245526-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:37:49 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-f9baa5a9-5713-4031-849c-8f08f16b126d req-04013d23-3447-4136-8be8-fa3fb358477c service nova] Lock "80e87454-90b9-4168-bf00-b7d93b245526-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:37:49 user nova-compute[72351]: DEBUG nova.compute.manager [req-f9baa5a9-5713-4031-849c-8f08f16b126d req-04013d23-3447-4136-8be8-fa3fb358477c service nova] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] No waiting events found dispatching network-vif-unplugged-fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:37:49 user nova-compute[72351]: DEBUG nova.compute.manager [req-f9baa5a9-5713-4031-849c-8f08f16b126d req-04013d23-3447-4136-8be8-fa3fb358477c service nova] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Received event network-vif-unplugged-fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4 for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:37:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:37:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:37:49 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Instance destroyed successfully.
Aug 09 02:37:49 user nova-compute[72351]: DEBUG nova.objects.instance [None req-46cb4596-989a-4c58-abf6-e62295b9c7bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lazy-loading 'resources' on Instance uuid 80e87454-90b9-4168-bf00-b7d93b245526 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:37:49 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-46cb4596-989a-4c58-abf6-e62295b9c7bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:31:13Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestOtherA-server-2126139398',display_name='tempest-ServerActionsTestOtherA-server-2126139398',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(7),hidden=False,host='user',hostname='tempest-serveractionstestothera-server-2126139398',id=38,image_ref='',info_cache=InstanceInfoCache,instance_type_id=7,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBB4HNXglactRKXsG6uozX3XR7A39DBllYpgpCz+5+1RhfLOeXdmVkNfDkIG7vQrgoVNKWK/MwqtjPDLEVURcnWQfOuv4ySYdfEPj4hs6FpDqfJbAa5qX0FAbZQuEg69FVQ==',key_name='tempest-keypair-976839412',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:33:28Z,launched_on='user',locked=False,locked_by=None,memory_mb=2048,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='69d584c230b94ff8aae722eb2f364c5c',ramdisk_id='',reservation_id='r-97d5c06t',resources=None,root_device_name='/dev/vda',root_gb=20,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerActionsTestOtherA-1303549305',owner_user_name='tempest-ServerActionsTestOtherA-1303549305-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:34:59Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='5dca34fdadbb4386a1ff1cb255265d47',uuid=80e87454-90b9-4168-bf00-b7d93b245526,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4", "address": "fa:16:3e:7a:77:69", "network": {"id": "b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-210689817-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.41", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "69d584c230b94ff8aae722eb2f364c5c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfcb0ceaf-ff", "ovs_interfaceid": "fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:37:49 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-46cb4596-989a-4c58-abf6-e62295b9c7bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Converting VIF {"id": "fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4", "address": "fa:16:3e:7a:77:69", "network": {"id": "b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-210689817-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.41", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "69d584c230b94ff8aae722eb2f364c5c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfcb0ceaf-ff", "ovs_interfaceid": "fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:37:49 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-46cb4596-989a-4c58-abf6-e62295b9c7bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:7a:77:69,bridge_name='br-int',has_traffic_filtering=True,id=fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4,network=Network(b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfcb0ceaf-ff') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:37:49 user nova-compute[72351]: DEBUG os_vif [None req-46cb4596-989a-4c58-abf6-e62295b9c7bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:7a:77:69,bridge_name='br-int',has_traffic_filtering=True,id=fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4,network=Network(b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfcb0ceaf-ff') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:37:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:37:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapfcb0ceaf-ff, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:37:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:37:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:37:49 user nova-compute[72351]: INFO os_vif [None req-46cb4596-989a-4c58-abf6-e62295b9c7bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:7a:77:69,bridge_name='br-int',has_traffic_filtering=True,id=fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4,network=Network(b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfcb0ceaf-ff')
Aug 09 02:37:49 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-46cb4596-989a-4c58-abf6-e62295b9c7bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 02:37:49 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-46cb4596-989a-4c58-abf6-e62295b9c7bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'oxcrdOpE', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdd'}, None), 'kwargs': True}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:37:49 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-46cb4596-989a-4c58-abf6-e62295b9c7bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:37:49 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-46cb4596-989a-4c58-abf6-e62295b9c7bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:37:49 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-46cb4596-989a-4c58-abf6-e62295b9c7bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:37:49 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-46cb4596-989a-4c58-abf6-e62295b9c7bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 02:37:49 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:37:49 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:37:49 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d98fdc5b-2e40-46a0-80a0-bf2690597b78]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:37:49 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:37:50 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.033s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:37:50 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[0f95f1d5-2b40-4221-a3cd-49b5f1f7fcef]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)\ntcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:37:50 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-46cb4596-989a-4c58-abf6-e62295b9c7bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:37:50 user nova-compute[72351]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:37:50 user nova-compute[72351]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df (non-flash)
Aug 09 02:37:50 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:37:50 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-46cb4596-989a-4c58-abf6-e62295b9c7bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:37:50 user nova-compute[72351]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 (non-flash)
Aug 09 02:37:50 user nova-compute[72351]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df (non-flash)
Aug 09 02:37:50 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:37:50 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-46cb4596-989a-4c58-abf6-e62295b9c7bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fcfdc6cdfc0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238'): ({'sdd'}, set())}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 02:37:50 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-46cb4596-989a-4c58-abf6-e62295b9c7bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Removing single pathed devices sdd {{(pid=72351) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Aug 09 02:37:50 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:37:50 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.028s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:37:50 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[5d2d7710-7c81-444b-9bdf-a00cb37db265]: (4, ('path checker states:\nup                  3\n\npaths: 0\nbusy: False\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:37:50 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdd {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:37:50 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdd" returned: 0 in 0.026s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:37:50 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[fca03159-1c87-4c86-9113-899273d10af0]: (4, ('ok\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:37:50 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-46cb4596-989a-4c58-abf6-e62295b9c7bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Flushing IO for device /dev/sdd {{(pid=72351) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Aug 09 02:37:50 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdd {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:37:50 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdd" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:37:50 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[0091697c-5d71-4e98-a520-9fd0640469d6]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:37:50 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-46cb4596-989a-4c58-abf6-e62295b9c7bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Remove SCSI device /dev/sdd with /sys/block/sdd/device/delete {{(pid=72351) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Aug 09 02:37:50 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdd/device/delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:37:50 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdd/device/delete" returned: 0 in 0.047s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:37:50 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[9c57a3ad-3197-4c2a-a443-b31fb311fb92]: (4, ('1', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:37:50 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-46cb4596-989a-4c58-abf6-e62295b9c7bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Checking to see if SCSI volumes sdd have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Aug 09 02:37:50 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-46cb4596-989a-4c58-abf6-e62295b9c7bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] SCSI volumes sdd have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Aug 09 02:37:50 user nova-compute[72351]: DEBUG os_brick.privileged.rootwrap [-] Unlinking ('/dev/disk/by-id/scsi-23832643035363835', '/dev/disk/by-id/scsi-1SCST_BIOd262e66a-82d056853fcbd2b5', '/dev/disk/by-id/scsi-SSCST_BIO_Storage_d262e66a') {{(pid=83366) unlink_root /usr/local/lib/python3.10/dist-packages/os_brick/privileged/rootwrap.py:219}}
Aug 09 02:37:50 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[cb95d699-615f-4b46-8470-bfcd6c91652d]: (4, None) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:37:50 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-46cb4596-989a-4c58-abf6-e62295b9c7bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238')] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 02:37:50 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:37:50 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:37:50 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ef1c8498-dbb7-4cba-bc1f-3865bf228fbb]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:37:50 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-46cb4596-989a-4c58-abf6-e62295b9c7bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:37:50 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 -p 172.16.0.220:3260 --logout {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:37:50 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 -p 172.16.0.220:3260 --logout" returned: 0 in 0.051s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:37:50 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[483b2468-c694-475f-b683-8fa3d5d2badb]: (4, ('Logging out of session [sid: 43, target: iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238, portal: 172.16.0.220,3260]\nLogout of [sid: 43, target: iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:37:50 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-46cb4596-989a-4c58-abf6-e62295b9c7bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 43, target: iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238, portal: 172.16.0.220,3260]
Aug 09 02:37:50 user nova-compute[72351]: Logout of [sid: 43, target: iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238, portal: 172.16.0.220,3260] successful.
Aug 09 02:37:50 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:37:50 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 -p 172.16.0.220:3260 --op delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:37:50 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7e32fe78-f2c4-409c-813a-36924e611238 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:37:50 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a5eed184-1f55-4b8e-8924-4715fc916cf9]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:37:50 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-46cb4596-989a-4c58-abf6-e62295b9c7bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:37:50 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-46cb4596-989a-4c58-abf6-e62295b9c7bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.312s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:37:50 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-46cb4596-989a-4c58-abf6-e62295b9c7bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] <== disconnect_volume: return (313ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:37:50 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-46cb4596-989a-4c58-abf6-e62295b9c7bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 02:37:50 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-46cb4596-989a-4c58-abf6-e62295b9c7bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Deleting instance files /opt/stack/data/nova/instances/80e87454-90b9-4168-bf00-b7d93b245526_del
Aug 09 02:37:50 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-46cb4596-989a-4c58-abf6-e62295b9c7bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Deletion of /opt/stack/data/nova/instances/80e87454-90b9-4168-bf00-b7d93b245526_del complete
Aug 09 02:37:50 user nova-compute[72351]: INFO nova.compute.manager [None req-46cb4596-989a-4c58-abf6-e62295b9c7bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Took 1.18 seconds to destroy the instance on the hypervisor.
Aug 09 02:37:50 user nova-compute[72351]: DEBUG oslo.service.loopingcall [None req-46cb4596-989a-4c58-abf6-e62295b9c7bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72351) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Aug 09 02:37:50 user nova-compute[72351]: DEBUG nova.compute.manager [-] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Deallocating network for instance {{(pid=72351) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2263}}
Aug 09 02:37:50 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] deallocate_for_instance() {{(pid=72351) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1795}}
Aug 09 02:37:51 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:37:51 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Took 0.93 seconds to deallocate network for instance.
Aug 09 02:37:51 user nova-compute[72351]: DEBUG nova.compute.manager [req-78e403e0-2a38-4cd4-9c5b-6c4dcd117159 req-14e364d5-b289-4b2c-ba76-43a20c111b14 service nova] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Received event network-vif-deleted-fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:37:51 user nova-compute[72351]: DEBUG nova.compute.manager [req-60c953e3-6735-4144-bfc9-92f079a888ed req-5e59ec29-25a0-4e3c-aedc-49ff8089855e service nova] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Received event network-vif-plugged-fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:37:51 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-60c953e3-6735-4144-bfc9-92f079a888ed req-5e59ec29-25a0-4e3c-aedc-49ff8089855e service nova] Acquiring lock "80e87454-90b9-4168-bf00-b7d93b245526-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:37:51 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-60c953e3-6735-4144-bfc9-92f079a888ed req-5e59ec29-25a0-4e3c-aedc-49ff8089855e service nova] Lock "80e87454-90b9-4168-bf00-b7d93b245526-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:37:51 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-60c953e3-6735-4144-bfc9-92f079a888ed req-5e59ec29-25a0-4e3c-aedc-49ff8089855e service nova] Lock "80e87454-90b9-4168-bf00-b7d93b245526-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:37:51 user nova-compute[72351]: DEBUG nova.compute.manager [req-60c953e3-6735-4144-bfc9-92f079a888ed req-5e59ec29-25a0-4e3c-aedc-49ff8089855e service nova] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] No waiting events found dispatching network-vif-plugged-fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:37:51 user nova-compute[72351]: WARNING nova.compute.manager [req-60c953e3-6735-4144-bfc9-92f079a888ed req-5e59ec29-25a0-4e3c-aedc-49ff8089855e service nova] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Received unexpected event network-vif-plugged-fcb0ceaf-ff8f-4a55-a064-fae14c4a6da4 for instance with vm_state active and task_state deleting.
Aug 09 02:37:52 user nova-compute[72351]: INFO nova.compute.manager [None req-46cb4596-989a-4c58-abf6-e62295b9c7bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Took 0.75 seconds to detach 1 volumes for instance.
Aug 09 02:37:52 user nova-compute[72351]: DEBUG nova.compute.manager [None req-46cb4596-989a-4c58-abf6-e62295b9c7bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Deleting volume: 7e32fe78-f2c4-409c-813a-36924e611238 {{(pid=72351) _cleanup_volumes /opt/stack/nova/nova/compute/manager.py:3221}}
Aug 09 02:37:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-46cb4596-989a-4c58-abf6-e62295b9c7bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:37:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-46cb4596-989a-4c58-abf6-e62295b9c7bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:37:52 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-46cb4596-989a-4c58-abf6-e62295b9c7bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:37:52 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-46cb4596-989a-4c58-abf6-e62295b9c7bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:37:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-46cb4596-989a-4c58-abf6-e62295b9c7bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.313s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:37:52 user nova-compute[72351]: INFO nova.scheduler.client.report [None req-46cb4596-989a-4c58-abf6-e62295b9c7bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Deleted allocations for instance 80e87454-90b9-4168-bf00-b7d93b245526
Aug 09 02:37:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-46cb4596-989a-4c58-abf6-e62295b9c7bc tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "80e87454-90b9-4168-bf00-b7d93b245526" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 3.448s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:37:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:37:59 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:38:03 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6093ad4e-27b0-4b35-8e5d-52177afc969e tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Acquiring lock "eb86cb49-2018-42c2-bd97-c662b829775d" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:38:03 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6093ad4e-27b0-4b35-8e5d-52177afc969e tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "eb86cb49-2018-42c2-bd97-c662b829775d" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:38:03 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6093ad4e-27b0-4b35-8e5d-52177afc969e tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Acquiring lock "eb86cb49-2018-42c2-bd97-c662b829775d-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:38:03 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6093ad4e-27b0-4b35-8e5d-52177afc969e tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "eb86cb49-2018-42c2-bd97-c662b829775d-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:38:03 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6093ad4e-27b0-4b35-8e5d-52177afc969e tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "eb86cb49-2018-42c2-bd97-c662b829775d-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:38:03 user nova-compute[72351]: INFO nova.compute.manager [None req-6093ad4e-27b0-4b35-8e5d-52177afc969e tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Terminating instance
Aug 09 02:38:03 user nova-compute[72351]: DEBUG nova.compute.manager [None req-6093ad4e-27b0-4b35-8e5d-52177afc969e tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Start destroying the instance on the hypervisor. {{(pid=72351) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3124}}
Aug 09 02:38:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:04 user nova-compute[72351]: DEBUG nova.compute.manager [req-cb478316-da55-40ec-94c8-f3d6aef43c17 req-d77f9b2d-0eab-40b8-97bc-9c3ed8eb9b67 service nova] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Received event network-vif-unplugged-5f97db70-9e5b-4d42-b765-b832fb522f41 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:38:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-cb478316-da55-40ec-94c8-f3d6aef43c17 req-d77f9b2d-0eab-40b8-97bc-9c3ed8eb9b67 service nova] Acquiring lock "eb86cb49-2018-42c2-bd97-c662b829775d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:38:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-cb478316-da55-40ec-94c8-f3d6aef43c17 req-d77f9b2d-0eab-40b8-97bc-9c3ed8eb9b67 service nova] Lock "eb86cb49-2018-42c2-bd97-c662b829775d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:38:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-cb478316-da55-40ec-94c8-f3d6aef43c17 req-d77f9b2d-0eab-40b8-97bc-9c3ed8eb9b67 service nova] Lock "eb86cb49-2018-42c2-bd97-c662b829775d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:38:04 user nova-compute[72351]: DEBUG nova.compute.manager [req-cb478316-da55-40ec-94c8-f3d6aef43c17 req-d77f9b2d-0eab-40b8-97bc-9c3ed8eb9b67 service nova] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] No waiting events found dispatching network-vif-unplugged-5f97db70-9e5b-4d42-b765-b832fb522f41 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:38:04 user nova-compute[72351]: DEBUG nova.compute.manager [req-cb478316-da55-40ec-94c8-f3d6aef43c17 req-d77f9b2d-0eab-40b8-97bc-9c3ed8eb9b67 service nova] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Received event network-vif-unplugged-5f97db70-9e5b-4d42-b765-b832fb522f41 for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:38:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:04 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Instance destroyed successfully.
Aug 09 02:38:04 user nova-compute[72351]: DEBUG nova.objects.instance [None req-6093ad4e-27b0-4b35-8e5d-52177afc969e tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lazy-loading 'resources' on Instance uuid eb86cb49-2018-42c2-bd97-c662b829775d {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:38:04 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-6093ad4e-27b0-4b35-8e5d-52177afc969e tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:21:35Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerRescueNegativeTestJSON-server-69135895',display_name='tempest-ServerRescueNegativeTestJSON-server-69135895',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serverrescuenegativetestjson-server-69135895',id=17,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:23:33Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='3aea1e66805146f7a8cd1d97e64a24af',ramdisk_id='',reservation_id='r-b0vsogov',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerRescueNegativeTestJSON-289364438',owner_user_name='tempest-ServerRescueNegativeTestJSON-289364438-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:23:33Z,user_data=None,user_id='419b28d5a46c481db435715a30ab2614',uuid=eb86cb49-2018-42c2-bd97-c662b829775d,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='rescued') vif={"id": "5f97db70-9e5b-4d42-b765-b832fb522f41", "address": "fa:16:3e:dc:da:c7", "network": {"id": "49e8751c-e119-430f-9110-3dc6069b1589", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-899275135-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3aea1e66805146f7a8cd1d97e64a24af", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5f97db70-9e", "ovs_interfaceid": "5f97db70-9e5b-4d42-b765-b832fb522f41", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:38:04 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-6093ad4e-27b0-4b35-8e5d-52177afc969e tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Converting VIF {"id": "5f97db70-9e5b-4d42-b765-b832fb522f41", "address": "fa:16:3e:dc:da:c7", "network": {"id": "49e8751c-e119-430f-9110-3dc6069b1589", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-899275135-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3aea1e66805146f7a8cd1d97e64a24af", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5f97db70-9e", "ovs_interfaceid": "5f97db70-9e5b-4d42-b765-b832fb522f41", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:38:04 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-6093ad4e-27b0-4b35-8e5d-52177afc969e tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:dc:da:c7,bridge_name='br-int',has_traffic_filtering=True,id=5f97db70-9e5b-4d42-b765-b832fb522f41,network=Network(49e8751c-e119-430f-9110-3dc6069b1589),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap5f97db70-9e') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:38:04 user nova-compute[72351]: DEBUG os_vif [None req-6093ad4e-27b0-4b35-8e5d-52177afc969e tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:dc:da:c7,bridge_name='br-int',has_traffic_filtering=True,id=5f97db70-9e5b-4d42-b765-b832fb522f41,network=Network(49e8751c-e119-430f-9110-3dc6069b1589),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap5f97db70-9e') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:38:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap5f97db70-9e, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:38:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:38:04 user nova-compute[72351]: INFO os_vif [None req-6093ad4e-27b0-4b35-8e5d-52177afc969e tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:dc:da:c7,bridge_name='br-int',has_traffic_filtering=True,id=5f97db70-9e5b-4d42-b765-b832fb522f41,network=Network(49e8751c-e119-430f-9110-3dc6069b1589),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap5f97db70-9e')
Aug 09 02:38:04 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-6093ad4e-27b0-4b35-8e5d-52177afc969e tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Deleting instance files /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d_del
Aug 09 02:38:04 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-6093ad4e-27b0-4b35-8e5d-52177afc969e tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Deletion of /opt/stack/data/nova/instances/eb86cb49-2018-42c2-bd97-c662b829775d_del complete
Aug 09 02:38:04 user nova-compute[72351]: INFO nova.compute.manager [None req-6093ad4e-27b0-4b35-8e5d-52177afc969e tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Took 0.88 seconds to destroy the instance on the hypervisor.
Aug 09 02:38:04 user nova-compute[72351]: DEBUG oslo.service.loopingcall [None req-6093ad4e-27b0-4b35-8e5d-52177afc969e tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72351) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Aug 09 02:38:04 user nova-compute[72351]: DEBUG nova.compute.manager [-] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Deallocating network for instance {{(pid=72351) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2263}}
Aug 09 02:38:04 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] deallocate_for_instance() {{(pid=72351) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1795}}
Aug 09 02:38:04 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691548669.8994699, 80e87454-90b9-4168-bf00-b7d93b245526 => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:38:04 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] VM Stopped (Lifecycle Event)
Aug 09 02:38:04 user nova-compute[72351]: DEBUG nova.compute.manager [None req-e1241924-8465-4b58-8c6a-1c0c15c4b08e None None] [instance: 80e87454-90b9-4168-bf00-b7d93b245526] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:38:05 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:38:05 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Took 0.79 seconds to deallocate network for instance.
Aug 09 02:38:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6093ad4e-27b0-4b35-8e5d-52177afc969e tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:38:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6093ad4e-27b0-4b35-8e5d-52177afc969e tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:38:05 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-6093ad4e-27b0-4b35-8e5d-52177afc969e tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:38:05 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-6093ad4e-27b0-4b35-8e5d-52177afc969e tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:38:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6093ad4e-27b0-4b35-8e5d-52177afc969e tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.308s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:38:05 user nova-compute[72351]: INFO nova.scheduler.client.report [None req-6093ad4e-27b0-4b35-8e5d-52177afc969e tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Deleted allocations for instance eb86cb49-2018-42c2-bd97-c662b829775d
Aug 09 02:38:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6093ad4e-27b0-4b35-8e5d-52177afc969e tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "eb86cb49-2018-42c2-bd97-c662b829775d" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.150s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:38:06 user nova-compute[72351]: DEBUG nova.compute.manager [req-74cc67b7-e60f-45c4-927c-3466b00d19b2 req-1471d925-9a97-4dfd-a338-88a29104b900 service nova] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Received event network-vif-plugged-5f97db70-9e5b-4d42-b765-b832fb522f41 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:38:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-74cc67b7-e60f-45c4-927c-3466b00d19b2 req-1471d925-9a97-4dfd-a338-88a29104b900 service nova] Acquiring lock "eb86cb49-2018-42c2-bd97-c662b829775d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:38:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-74cc67b7-e60f-45c4-927c-3466b00d19b2 req-1471d925-9a97-4dfd-a338-88a29104b900 service nova] Lock "eb86cb49-2018-42c2-bd97-c662b829775d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:38:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-74cc67b7-e60f-45c4-927c-3466b00d19b2 req-1471d925-9a97-4dfd-a338-88a29104b900 service nova] Lock "eb86cb49-2018-42c2-bd97-c662b829775d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:38:06 user nova-compute[72351]: DEBUG nova.compute.manager [req-74cc67b7-e60f-45c4-927c-3466b00d19b2 req-1471d925-9a97-4dfd-a338-88a29104b900 service nova] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] No waiting events found dispatching network-vif-plugged-5f97db70-9e5b-4d42-b765-b832fb522f41 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:38:06 user nova-compute[72351]: WARNING nova.compute.manager [req-74cc67b7-e60f-45c4-927c-3466b00d19b2 req-1471d925-9a97-4dfd-a338-88a29104b900 service nova] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Received unexpected event network-vif-plugged-5f97db70-9e5b-4d42-b765-b832fb522f41 for instance with vm_state deleted and task_state None.
Aug 09 02:38:06 user nova-compute[72351]: DEBUG nova.compute.manager [req-74cc67b7-e60f-45c4-927c-3466b00d19b2 req-1471d925-9a97-4dfd-a338-88a29104b900 service nova] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Received event network-vif-deleted-5f97db70-9e5b-4d42-b765-b832fb522f41 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:38:06 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:38:07 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:07 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:38:07 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72351) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10461}}
Aug 09 02:38:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:10 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:38:11 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:38:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:38:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:38:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:38:11 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Auditing locally available compute resources for user (node: user) {{(pid=72351) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Aug 09 02:38:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:38:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json" returned: 0 in 0.140s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:38:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:38:12 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a/disk --force-share --output=json" returned: 0 in 0.229s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:38:12 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdg (vda) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:38:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/41b5e305-3973-4937-b262-d32db1091e20/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:38:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/41b5e305-3973-4937-b262-d32db1091e20/disk --force-share --output=json" returned: 0 in 0.142s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:38:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/41b5e305-3973-4937-b262-d32db1091e20/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:38:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/41b5e305-3973-4937-b262-d32db1091e20/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:38:12 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdh (vda) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:38:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bde1ef1e-163c-40eb-abb5-962b9064b7ab/disk.rescue --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:38:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bde1ef1e-163c-40eb-abb5-962b9064b7ab/disk.rescue --force-share --output=json" returned: 0 in 0.131s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:38:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bde1ef1e-163c-40eb-abb5-962b9064b7ab/disk.rescue --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:38:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bde1ef1e-163c-40eb-abb5-962b9064b7ab/disk.rescue --force-share --output=json" returned: 0 in 0.156s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:38:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:38:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json" returned: 0 in 0.129s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:38:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:38:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335/disk --force-share --output=json" returned: 0 in 0.128s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:38:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:38:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json" returned: 0 in 0.144s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:38:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:38:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json" returned: 0 in 0.139s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:38:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7bef74f4-3bb3-4ad0-bff7-34156be31249/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:38:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7bef74f4-3bb3-4ad0-bff7-34156be31249/disk --force-share --output=json" returned: 0 in 0.140s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:38:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7bef74f4-3bb3-4ad0-bff7-34156be31249/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:38:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/7bef74f4-3bb3-4ad0-bff7-34156be31249/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:38:14 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:38:14 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:38:14 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Hypervisor/Node resource view: name=user free_ram=8080MB free_disk=25.069129943847656GB free_vcpus=5 pci_devices=[{"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72351) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Aug 09 02:38:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:38:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:38:14 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:38:14 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:38:14 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance a7f3d076-864b-4b68-9207-214ce2c21444 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:38:14 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 957c1510-1320-4492-9052-a58ff540319d actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:38:14 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 7bef74f4-3bb3-4ad0-bff7-34156be31249 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:38:14 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 41b5e305-3973-4937-b262-d32db1091e20 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:38:14 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance bde1ef1e-163c-40eb-abb5-962b9064b7ab actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:38:14 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Total usable vcpus: 12, total allocated vcpus: 7 {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Aug 09 02:38:14 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Final resource view: name=user phys_ram=16011MB used_ram=4096MB phys_disk=40GB used_disk=5GB total_vcpus=12 used_vcpus=7 pci_stats=[] {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Aug 09 02:38:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:14 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:38:14 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:38:14 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Compute_service record updated for user:user {{(pid=72351) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Aug 09 02:38:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.353s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:38:14 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._cleanup_expired_console_auth_tokens {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:38:15 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:15 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:38:15 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:38:15 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Starting heal instance info cache {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9842}}
Aug 09 02:38:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "refresh_cache-585c81a5-ac24-4d12-a8a6-1e7b12a2d53a" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:38:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquired lock "refresh_cache-585c81a5-ac24-4d12-a8a6-1e7b12a2d53a" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:38:15 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Forcefully refreshing network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1996}}
Aug 09 02:38:16 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Updating instance_info_cache with network_info: [{"id": "e521fae4-093d-4c92-a32d-684c1a5137f5", "address": "fa:16:3e:e5:b0:10", "network": {"id": "b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-210689817-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "69d584c230b94ff8aae722eb2f364c5c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape521fae4-09", "ovs_interfaceid": "e521fae4-093d-4c92-a32d-684c1a5137f5", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:38:16 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Releasing lock "refresh_cache-585c81a5-ac24-4d12-a8a6-1e7b12a2d53a" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:38:16 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Updated the network info_cache for instance {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9913}}
Aug 09 02:38:16 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:38:16 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:38:16 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:38:16 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._run_pending_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:38:16 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Cleaning up deleted instances {{(pid=72351) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11129}}
Aug 09 02:38:16 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] There are 0 instances to clean {{(pid=72351) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11138}}
Aug 09 02:38:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Acquiring lock "6de75cee-9bf1-4f9f-8fa3-977d9432b805" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:38:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "6de75cee-9bf1-4f9f-8fa3-977d9432b805" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:38:17 user nova-compute[72351]: DEBUG nova.compute.manager [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Starting instance... {{(pid=72351) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2406}}
Aug 09 02:38:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:38:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:38:17 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72351) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Aug 09 02:38:17 user nova-compute[72351]: INFO nova.compute.claims [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Claim successful on node user
Aug 09 02:38:18 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:38:18 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:38:18 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.355s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:38:18 user nova-compute[72351]: DEBUG nova.compute.manager [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Start building networks asynchronously for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2803}}
Aug 09 02:38:18 user nova-compute[72351]: DEBUG nova.compute.manager [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Allocating IP information in the background. {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1956}}
Aug 09 02:38:18 user nova-compute[72351]: DEBUG nova.network.neutron [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] allocate_for_instance() {{(pid=72351) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Aug 09 02:38:18 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Aug 09 02:38:18 user nova-compute[72351]: DEBUG nova.compute.manager [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Start building block device mappings for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2838}}
Aug 09 02:38:18 user nova-compute[72351]: DEBUG nova.policy [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '17f7007adf02474e9b019cd8af099ff8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c07a6b254414b41a673468238d3aa62', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72351) authorize /opt/stack/nova/nova/policy.py:203}}
Aug 09 02:38:18 user nova-compute[72351]: DEBUG nova.compute.manager [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Start spawning the instance on the hypervisor. {{(pid=72351) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2612}}
Aug 09 02:38:18 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Creating instance directory {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4717}}
Aug 09 02:38:18 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Creating image(s)
Aug 09 02:38:18 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Acquiring lock "/opt/stack/data/nova/instances/6de75cee-9bf1-4f9f-8fa3-977d9432b805/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:38:18 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "/opt/stack/data/nova/instances/6de75cee-9bf1-4f9f-8fa3-977d9432b805/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:38:18 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "/opt/stack/data/nova/instances/6de75cee-9bf1-4f9f-8fa3-977d9432b805/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:38:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:38:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.133s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:38:18 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Acquiring lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:38:18 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:38:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:38:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.135s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:38:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/6de75cee-9bf1-4f9f-8fa3-977d9432b805/disk 1073741824 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:38:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/6de75cee-9bf1-4f9f-8fa3-977d9432b805/disk 1073741824" returned: 0 in 0.055s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:38:18 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.196s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:38:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:38:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.145s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:38:18 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Checking if we can resize image /opt/stack/data/nova/instances/6de75cee-9bf1-4f9f-8fa3-977d9432b805/disk. size=1073741824 {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Aug 09 02:38:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6de75cee-9bf1-4f9f-8fa3-977d9432b805/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:38:19 user nova-compute[72351]: DEBUG nova.network.neutron [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Successfully created port: bacefb00-adc9-4430-b7e1-83c6200d6144 {{(pid=72351) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Aug 09 02:38:19 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6de75cee-9bf1-4f9f-8fa3-977d9432b805/disk --force-share --output=json" returned: 0 in 0.131s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:38:19 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Cannot resize image /opt/stack/data/nova/instances/6de75cee-9bf1-4f9f-8fa3-977d9432b805/disk to a smaller size. {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Aug 09 02:38:19 user nova-compute[72351]: DEBUG nova.objects.instance [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lazy-loading 'migration_context' on Instance uuid 6de75cee-9bf1-4f9f-8fa3-977d9432b805 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:38:19 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Created local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4851}}
Aug 09 02:38:19 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Ensure instance console log exists: /opt/stack/data/nova/instances/6de75cee-9bf1-4f9f-8fa3-977d9432b805/console.log {{(pid=72351) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4603}}
Aug 09 02:38:19 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:38:19 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:38:19 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:38:19 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691548684.5170226, eb86cb49-2018-42c2-bd97-c662b829775d => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:38:19 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] VM Stopped (Lifecycle Event)
Aug 09 02:38:19 user nova-compute[72351]: DEBUG nova.compute.manager [None req-6a78a371-8403-4504-a847-2c21556945ee None None] [instance: eb86cb49-2018-42c2-bd97-c662b829775d] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:38:19 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:20 user nova-compute[72351]: DEBUG nova.network.neutron [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Successfully updated port: bacefb00-adc9-4430-b7e1-83c6200d6144 {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:38:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Acquiring lock "refresh_cache-6de75cee-9bf1-4f9f-8fa3-977d9432b805" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:38:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Acquired lock "refresh_cache-6de75cee-9bf1-4f9f-8fa3-977d9432b805" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:38:20 user nova-compute[72351]: DEBUG nova.network.neutron [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:38:20 user nova-compute[72351]: DEBUG nova.compute.manager [req-212947b3-9201-404e-98d7-875c81ad9903 req-1e1cf3c3-0c7b-45a6-b6e8-f6c639278594 service nova] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Received event network-changed-bacefb00-adc9-4430-b7e1-83c6200d6144 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:38:20 user nova-compute[72351]: DEBUG nova.compute.manager [req-212947b3-9201-404e-98d7-875c81ad9903 req-1e1cf3c3-0c7b-45a6-b6e8-f6c639278594 service nova] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Refreshing instance network info cache due to event network-changed-bacefb00-adc9-4430-b7e1-83c6200d6144. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:38:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-212947b3-9201-404e-98d7-875c81ad9903 req-1e1cf3c3-0c7b-45a6-b6e8-f6c639278594 service nova] Acquiring lock "refresh_cache-6de75cee-9bf1-4f9f-8fa3-977d9432b805" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:38:20 user nova-compute[72351]: DEBUG nova.network.neutron [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:38:20 user nova-compute[72351]: DEBUG nova.network.neutron [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Updating instance_info_cache with network_info: [{"id": "bacefb00-adc9-4430-b7e1-83c6200d6144", "address": "fa:16:3e:1a:c1:d6", "network": {"id": "47c43230-a244-4198-84e4-9ed98c005cd9", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1392403106-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c07a6b254414b41a673468238d3aa62", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbacefb00-ad", "ovs_interfaceid": "bacefb00-adc9-4430-b7e1-83c6200d6144", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:38:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Releasing lock "refresh_cache-6de75cee-9bf1-4f9f-8fa3-977d9432b805" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:38:20 user nova-compute[72351]: DEBUG nova.compute.manager [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Instance network_info: |[{"id": "bacefb00-adc9-4430-b7e1-83c6200d6144", "address": "fa:16:3e:1a:c1:d6", "network": {"id": "47c43230-a244-4198-84e4-9ed98c005cd9", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1392403106-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c07a6b254414b41a673468238d3aa62", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbacefb00-ad", "ovs_interfaceid": "bacefb00-adc9-4430-b7e1-83c6200d6144", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1971}}
Aug 09 02:38:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-212947b3-9201-404e-98d7-875c81ad9903 req-1e1cf3c3-0c7b-45a6-b6e8-f6c639278594 service nova] Acquired lock "refresh_cache-6de75cee-9bf1-4f9f-8fa3-977d9432b805" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:38:20 user nova-compute[72351]: DEBUG nova.network.neutron [req-212947b3-9201-404e-98d7-875c81ad9903 req-1e1cf3c3-0c7b-45a6-b6e8-f6c639278594 service nova] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Refreshing network info cache for port bacefb00-adc9-4430-b7e1-83c6200d6144 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:38:20 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Start _get_guest_xml network_info=[{"id": "bacefb00-adc9-4430-b7e1-83c6200d6144", "address": "fa:16:3e:1a:c1:d6", "network": {"id": "47c43230-a244-4198-84e4-9ed98c005cd9", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1392403106-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c07a6b254414b41a673468238d3aa62", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbacefb00-ad", "ovs_interfaceid": "bacefb00-adc9-4430-b7e1-83c6200d6144", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'size': 0, 'device_type': 'disk', 'encryption_secret_uuid': None, 'guest_format': None, 'encryption_options': None, 'disk_bus': 'virtio', 'encrypted': False, 'encryption_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'image_id': '18ccda7f-31d3-4978-ab9d-4ed2f74eff94'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:38:20 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:38:20 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:38:20 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:38:20 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:38:20 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:38:20 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:38:20 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:38:20 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:38:20 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:38:20 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:38:20 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:38:20 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:38:20 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:38:20 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:38:20 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:38:20 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:38:20 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:38:20 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:38:20 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:38:17Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachVolumeNegativeTest-server-328142793',display_name='tempest-AttachVolumeNegativeTest-server-328142793',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumenegativetest-server-328142793',id=44,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJdkoorTUzpV1GANS341gZEFGpd4RrZ1PVNEy4XX7359woZ/ReYCscWlr2MmLYK4hBSO0BqDIVirOq9+Vo0HPFngP7LodWctuluJtAdOKpAvkiD66c/ymOryEHKAKukJuA==',key_name='tempest-keypair-1555678363',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='8c07a6b254414b41a673468238d3aa62',ramdisk_id='',reservation_id='r-ommfsdzj',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeNegativeTest-758771161',owner_user_name='tempest-AttachVolumeNegativeTest-758771161-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:38:18Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='17f7007adf02474e9b019cd8af099ff8',uuid=6de75cee-9bf1-4f9f-8fa3-977d9432b805,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "bacefb00-adc9-4430-b7e1-83c6200d6144", "address": "fa:16:3e:1a:c1:d6", "network": {"id": "47c43230-a244-4198-84e4-9ed98c005cd9", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1392403106-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c07a6b254414b41a673468238d3aa62", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbacefb00-ad", "ovs_interfaceid": "bacefb00-adc9-4430-b7e1-83c6200d6144", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:38:20 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Converting VIF {"id": "bacefb00-adc9-4430-b7e1-83c6200d6144", "address": "fa:16:3e:1a:c1:d6", "network": {"id": "47c43230-a244-4198-84e4-9ed98c005cd9", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1392403106-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c07a6b254414b41a673468238d3aa62", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbacefb00-ad", "ovs_interfaceid": "bacefb00-adc9-4430-b7e1-83c6200d6144", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:38:20 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:1a:c1:d6,bridge_name='br-int',has_traffic_filtering=True,id=bacefb00-adc9-4430-b7e1-83c6200d6144,network=Network(47c43230-a244-4198-84e4-9ed98c005cd9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapbacefb00-ad') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:38:20 user nova-compute[72351]: DEBUG nova.objects.instance [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lazy-loading 'pci_devices' on Instance uuid 6de75cee-9bf1-4f9f-8fa3-977d9432b805 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:38:20 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:38:20 user nova-compute[72351]:   <uuid>6de75cee-9bf1-4f9f-8fa3-977d9432b805</uuid>
Aug 09 02:38:20 user nova-compute[72351]:   <name>instance-0000002c</name>
Aug 09 02:38:20 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:38:20 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:38:20 user nova-compute[72351]:   <metadata>
Aug 09 02:38:20 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:38:20 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:38:20 user nova-compute[72351]:       <nova:name>tempest-AttachVolumeNegativeTest-server-328142793</nova:name>
Aug 09 02:38:20 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:38:20</nova:creationTime>
Aug 09 02:38:20 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:38:20 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:38:20 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:38:20 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:38:20 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:38:20 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:38:20 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:38:20 user nova-compute[72351]:       <nova:owner>
Aug 09 02:38:20 user nova-compute[72351]:         <nova:user uuid="17f7007adf02474e9b019cd8af099ff8">tempest-AttachVolumeNegativeTest-758771161-project-member</nova:user>
Aug 09 02:38:20 user nova-compute[72351]:         <nova:project uuid="8c07a6b254414b41a673468238d3aa62">tempest-AttachVolumeNegativeTest-758771161</nova:project>
Aug 09 02:38:20 user nova-compute[72351]:       </nova:owner>
Aug 09 02:38:20 user nova-compute[72351]:       <nova:root type="image" uuid="18ccda7f-31d3-4978-ab9d-4ed2f74eff94"/>
Aug 09 02:38:20 user nova-compute[72351]:       <nova:ports>
Aug 09 02:38:20 user nova-compute[72351]:         <nova:port uuid="bacefb00-adc9-4430-b7e1-83c6200d6144">
Aug 09 02:38:20 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.9" ipVersion="4"/>
Aug 09 02:38:20 user nova-compute[72351]:         </nova:port>
Aug 09 02:38:20 user nova-compute[72351]:       </nova:ports>
Aug 09 02:38:20 user nova-compute[72351]:     </nova:instance>
Aug 09 02:38:20 user nova-compute[72351]:   </metadata>
Aug 09 02:38:20 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:38:20 user nova-compute[72351]:     <system>
Aug 09 02:38:20 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:38:20 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:38:20 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:38:20 user nova-compute[72351]:       <entry name="serial">6de75cee-9bf1-4f9f-8fa3-977d9432b805</entry>
Aug 09 02:38:20 user nova-compute[72351]:       <entry name="uuid">6de75cee-9bf1-4f9f-8fa3-977d9432b805</entry>
Aug 09 02:38:20 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:38:20 user nova-compute[72351]:     </system>
Aug 09 02:38:20 user nova-compute[72351]:   </sysinfo>
Aug 09 02:38:20 user nova-compute[72351]:   <os>
Aug 09 02:38:20 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:38:20 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:38:20 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:38:20 user nova-compute[72351]:   </os>
Aug 09 02:38:20 user nova-compute[72351]:   <features>
Aug 09 02:38:20 user nova-compute[72351]:     <acpi/>
Aug 09 02:38:20 user nova-compute[72351]:     <apic/>
Aug 09 02:38:20 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:38:20 user nova-compute[72351]:   </features>
Aug 09 02:38:20 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:38:20 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:38:20 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:38:20 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:38:20 user nova-compute[72351]:   </clock>
Aug 09 02:38:20 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:38:20 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:38:20 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:38:20 user nova-compute[72351]:   </cpu>
Aug 09 02:38:20 user nova-compute[72351]:   <devices>
Aug 09 02:38:20 user nova-compute[72351]:     <disk type="file" device="disk">
Aug 09 02:38:20 user nova-compute[72351]:       <driver name="qemu" type="qcow2" cache="none"/>
Aug 09 02:38:20 user nova-compute[72351]:       <source file="/opt/stack/data/nova/instances/6de75cee-9bf1-4f9f-8fa3-977d9432b805/disk"/>
Aug 09 02:38:20 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:38:20 user nova-compute[72351]:     </disk>
Aug 09 02:38:20 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:38:20 user nova-compute[72351]:       <mac address="fa:16:3e:1a:c1:d6"/>
Aug 09 02:38:20 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:38:20 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:38:20 user nova-compute[72351]:       <target dev="tapbacefb00-ad"/>
Aug 09 02:38:20 user nova-compute[72351]:     </interface>
Aug 09 02:38:20 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:38:20 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/6de75cee-9bf1-4f9f-8fa3-977d9432b805/console.log" append="off"/>
Aug 09 02:38:20 user nova-compute[72351]:     </serial>
Aug 09 02:38:20 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:38:20 user nova-compute[72351]:     <video>
Aug 09 02:38:20 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:38:20 user nova-compute[72351]:     </video>
Aug 09 02:38:20 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:38:20 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:38:20 user nova-compute[72351]:     </rng>
Aug 09 02:38:20 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:38:20 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:38:20 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:38:20 user nova-compute[72351]:     </memballoon>
Aug 09 02:38:20 user nova-compute[72351]:   </devices>
Aug 09 02:38:20 user nova-compute[72351]: </domain>
Aug 09 02:38:20 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:38:20 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:38:17Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachVolumeNegativeTest-server-328142793',display_name='tempest-AttachVolumeNegativeTest-server-328142793',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumenegativetest-server-328142793',id=44,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJdkoorTUzpV1GANS341gZEFGpd4RrZ1PVNEy4XX7359woZ/ReYCscWlr2MmLYK4hBSO0BqDIVirOq9+Vo0HPFngP7LodWctuluJtAdOKpAvkiD66c/ymOryEHKAKukJuA==',key_name='tempest-keypair-1555678363',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='8c07a6b254414b41a673468238d3aa62',ramdisk_id='',reservation_id='r-ommfsdzj',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeNegativeTest-758771161',owner_user_name='tempest-AttachVolumeNegativeTest-758771161-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:38:18Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='17f7007adf02474e9b019cd8af099ff8',uuid=6de75cee-9bf1-4f9f-8fa3-977d9432b805,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "bacefb00-adc9-4430-b7e1-83c6200d6144", "address": "fa:16:3e:1a:c1:d6", "network": {"id": "47c43230-a244-4198-84e4-9ed98c005cd9", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1392403106-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c07a6b254414b41a673468238d3aa62", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbacefb00-ad", "ovs_interfaceid": "bacefb00-adc9-4430-b7e1-83c6200d6144", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:38:20 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Converting VIF {"id": "bacefb00-adc9-4430-b7e1-83c6200d6144", "address": "fa:16:3e:1a:c1:d6", "network": {"id": "47c43230-a244-4198-84e4-9ed98c005cd9", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1392403106-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c07a6b254414b41a673468238d3aa62", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbacefb00-ad", "ovs_interfaceid": "bacefb00-adc9-4430-b7e1-83c6200d6144", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:38:20 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:1a:c1:d6,bridge_name='br-int',has_traffic_filtering=True,id=bacefb00-adc9-4430-b7e1-83c6200d6144,network=Network(47c43230-a244-4198-84e4-9ed98c005cd9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapbacefb00-ad') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:38:20 user nova-compute[72351]: DEBUG os_vif [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:1a:c1:d6,bridge_name='br-int',has_traffic_filtering=True,id=bacefb00-adc9-4430-b7e1-83c6200d6144,network=Network(47c43230-a244-4198-84e4-9ed98c005cd9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapbacefb00-ad') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:38:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:38:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:38:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapbacefb00-ad, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:38:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapbacefb00-ad, col_values=(('external_ids', {'iface-id': 'bacefb00-adc9-4430-b7e1-83c6200d6144', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:1a:c1:d6', 'vm-uuid': '6de75cee-9bf1-4f9f-8fa3-977d9432b805'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:38:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:38:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:20 user nova-compute[72351]: INFO os_vif [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:1a:c1:d6,bridge_name='br-int',has_traffic_filtering=True,id=bacefb00-adc9-4430-b7e1-83c6200d6144,network=Network(47c43230-a244-4198-84e4-9ed98c005cd9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapbacefb00-ad')
Aug 09 02:38:20 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:38:20 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] No VIF found with MAC fa:16:3e:1a:c1:d6, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:38:20 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._cleanup_incomplete_migrations {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:38:20 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Cleaning up deleted instances with incomplete migration  {{(pid=72351) _cleanup_incomplete_migrations /opt/stack/nova/nova/compute/manager.py:11167}}
Aug 09 02:38:20 user nova-compute[72351]: DEBUG nova.network.neutron [req-212947b3-9201-404e-98d7-875c81ad9903 req-1e1cf3c3-0c7b-45a6-b6e8-f6c639278594 service nova] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Updated VIF entry in instance network info cache for port bacefb00-adc9-4430-b7e1-83c6200d6144. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:38:20 user nova-compute[72351]: DEBUG nova.network.neutron [req-212947b3-9201-404e-98d7-875c81ad9903 req-1e1cf3c3-0c7b-45a6-b6e8-f6c639278594 service nova] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Updating instance_info_cache with network_info: [{"id": "bacefb00-adc9-4430-b7e1-83c6200d6144", "address": "fa:16:3e:1a:c1:d6", "network": {"id": "47c43230-a244-4198-84e4-9ed98c005cd9", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1392403106-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c07a6b254414b41a673468238d3aa62", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbacefb00-ad", "ovs_interfaceid": "bacefb00-adc9-4430-b7e1-83c6200d6144", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:38:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-212947b3-9201-404e-98d7-875c81ad9903 req-1e1cf3c3-0c7b-45a6-b6e8-f6c639278594 service nova] Releasing lock "refresh_cache-6de75cee-9bf1-4f9f-8fa3-977d9432b805" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:38:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:22 user nova-compute[72351]: DEBUG nova.compute.manager [req-eb687079-2176-4f1a-ae5d-3c271390e219 req-815ade22-38c1-4e11-9cc3-31f3bec7f808 service nova] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Received event network-vif-plugged-bacefb00-adc9-4430-b7e1-83c6200d6144 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:38:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-eb687079-2176-4f1a-ae5d-3c271390e219 req-815ade22-38c1-4e11-9cc3-31f3bec7f808 service nova] Acquiring lock "6de75cee-9bf1-4f9f-8fa3-977d9432b805-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:38:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-eb687079-2176-4f1a-ae5d-3c271390e219 req-815ade22-38c1-4e11-9cc3-31f3bec7f808 service nova] Lock "6de75cee-9bf1-4f9f-8fa3-977d9432b805-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:38:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-eb687079-2176-4f1a-ae5d-3c271390e219 req-815ade22-38c1-4e11-9cc3-31f3bec7f808 service nova] Lock "6de75cee-9bf1-4f9f-8fa3-977d9432b805-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:38:22 user nova-compute[72351]: DEBUG nova.compute.manager [req-eb687079-2176-4f1a-ae5d-3c271390e219 req-815ade22-38c1-4e11-9cc3-31f3bec7f808 service nova] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] No waiting events found dispatching network-vif-plugged-bacefb00-adc9-4430-b7e1-83c6200d6144 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:38:22 user nova-compute[72351]: WARNING nova.compute.manager [req-eb687079-2176-4f1a-ae5d-3c271390e219 req-815ade22-38c1-4e11-9cc3-31f3bec7f808 service nova] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Received unexpected event network-vif-plugged-bacefb00-adc9-4430-b7e1-83c6200d6144 for instance with vm_state building and task_state spawning.
Aug 09 02:38:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:23 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:24 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548704.0604482, 6de75cee-9bf1-4f9f-8fa3-977d9432b805 => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:38:24 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] VM Resumed (Lifecycle Event)
Aug 09 02:38:24 user nova-compute[72351]: DEBUG nova.compute.manager [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:38:24 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Guest created on hypervisor {{(pid=72351) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4411}}
Aug 09 02:38:24 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Instance spawned successfully.
Aug 09 02:38:24 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:894}}
Aug 09 02:38:24 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:38:24 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:38:24 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Found default for hw_cdrom_bus of ide {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:38:24 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Found default for hw_disk_bus of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:38:24 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Found default for hw_input_bus of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:38:24 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Found default for hw_pointer_model of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:38:24 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Found default for hw_video_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:38:24 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Found default for hw_vif_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:38:24 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:38:24 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548704.0638354, 6de75cee-9bf1-4f9f-8fa3-977d9432b805 => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:38:24 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] VM Started (Lifecycle Event)
Aug 09 02:38:24 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:38:24 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:38:24 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:38:24 user nova-compute[72351]: INFO nova.compute.manager [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Took 5.69 seconds to spawn the instance on the hypervisor.
Aug 09 02:38:24 user nova-compute[72351]: DEBUG nova.compute.manager [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:38:24 user nova-compute[72351]: DEBUG nova.compute.manager [req-d10c2484-f1ea-4ebc-ab69-60e405f0fd10 req-1b917f06-6b73-4152-89d4-33cde5d06f10 service nova] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Received event network-vif-plugged-bacefb00-adc9-4430-b7e1-83c6200d6144 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:38:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d10c2484-f1ea-4ebc-ab69-60e405f0fd10 req-1b917f06-6b73-4152-89d4-33cde5d06f10 service nova] Acquiring lock "6de75cee-9bf1-4f9f-8fa3-977d9432b805-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:38:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d10c2484-f1ea-4ebc-ab69-60e405f0fd10 req-1b917f06-6b73-4152-89d4-33cde5d06f10 service nova] Lock "6de75cee-9bf1-4f9f-8fa3-977d9432b805-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:38:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d10c2484-f1ea-4ebc-ab69-60e405f0fd10 req-1b917f06-6b73-4152-89d4-33cde5d06f10 service nova] Lock "6de75cee-9bf1-4f9f-8fa3-977d9432b805-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:38:24 user nova-compute[72351]: DEBUG nova.compute.manager [req-d10c2484-f1ea-4ebc-ab69-60e405f0fd10 req-1b917f06-6b73-4152-89d4-33cde5d06f10 service nova] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] No waiting events found dispatching network-vif-plugged-bacefb00-adc9-4430-b7e1-83c6200d6144 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:38:24 user nova-compute[72351]: WARNING nova.compute.manager [req-d10c2484-f1ea-4ebc-ab69-60e405f0fd10 req-1b917f06-6b73-4152-89d4-33cde5d06f10 service nova] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Received unexpected event network-vif-plugged-bacefb00-adc9-4430-b7e1-83c6200d6144 for instance with vm_state active and task_state None.
Aug 09 02:38:24 user nova-compute[72351]: INFO nova.compute.manager [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Took 6.37 seconds to build instance.
Aug 09 02:38:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-fe96adad-355b-4f42-b441-abe24cb32273 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "6de75cee-9bf1-4f9f-8fa3-977d9432b805" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 6.500s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:38:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:28 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:30 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:35 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:38 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:39 user nova-compute[72351]: INFO nova.compute.manager [None req-443fd43d-3a10-4134-8fe2-109903a56ded tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Unrescuing
Aug 09 02:38:39 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-443fd43d-3a10-4134-8fe2-109903a56ded tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Acquiring lock "refresh_cache-bde1ef1e-163c-40eb-abb5-962b9064b7ab" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:38:39 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-443fd43d-3a10-4134-8fe2-109903a56ded tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Acquired lock "refresh_cache-bde1ef1e-163c-40eb-abb5-962b9064b7ab" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:38:39 user nova-compute[72351]: DEBUG nova.network.neutron [None req-443fd43d-3a10-4134-8fe2-109903a56ded tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:38:39 user nova-compute[72351]: DEBUG nova.network.neutron [None req-443fd43d-3a10-4134-8fe2-109903a56ded tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Updating instance_info_cache with network_info: [{"id": "8424518d-87b7-4fc4-8a58-1b4f094a5ca0", "address": "fa:16:3e:08:61:83", "network": {"id": "9a452414-d61f-4045-a6cc-9be93c892c4c", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "321111f935dd4e8aab4a083206ea9a7e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8424518d-87", "ovs_interfaceid": "8424518d-87b7-4fc4-8a58-1b4f094a5ca0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:38:39 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-443fd43d-3a10-4134-8fe2-109903a56ded tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Releasing lock "refresh_cache-bde1ef1e-163c-40eb-abb5-962b9064b7ab" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:38:39 user nova-compute[72351]: DEBUG nova.objects.instance [None req-443fd43d-3a10-4134-8fe2-109903a56ded tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lazy-loading 'flavor' on Instance uuid bde1ef1e-163c-40eb-abb5-962b9064b7ab {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:38:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:40 user nova-compute[72351]: DEBUG nova.compute.manager [req-dd747e35-5a7b-4079-85da-c0306d828d80 req-9c052281-e78d-4870-8362-4752237d7db1 service nova] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Received event network-vif-unplugged-8424518d-87b7-4fc4-8a58-1b4f094a5ca0 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:38:40 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-dd747e35-5a7b-4079-85da-c0306d828d80 req-9c052281-e78d-4870-8362-4752237d7db1 service nova] Acquiring lock "bde1ef1e-163c-40eb-abb5-962b9064b7ab-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:38:40 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-dd747e35-5a7b-4079-85da-c0306d828d80 req-9c052281-e78d-4870-8362-4752237d7db1 service nova] Lock "bde1ef1e-163c-40eb-abb5-962b9064b7ab-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:38:40 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-dd747e35-5a7b-4079-85da-c0306d828d80 req-9c052281-e78d-4870-8362-4752237d7db1 service nova] Lock "bde1ef1e-163c-40eb-abb5-962b9064b7ab-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:38:40 user nova-compute[72351]: DEBUG nova.compute.manager [req-dd747e35-5a7b-4079-85da-c0306d828d80 req-9c052281-e78d-4870-8362-4752237d7db1 service nova] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] No waiting events found dispatching network-vif-unplugged-8424518d-87b7-4fc4-8a58-1b4f094a5ca0 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:38:40 user nova-compute[72351]: WARNING nova.compute.manager [req-dd747e35-5a7b-4079-85da-c0306d828d80 req-9c052281-e78d-4870-8362-4752237d7db1 service nova] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Received unexpected event network-vif-unplugged-8424518d-87b7-4fc4-8a58-1b4f094a5ca0 for instance with vm_state rescued and task_state unrescuing.
Aug 09 02:38:40 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:40 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:40 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Instance destroyed successfully.
Aug 09 02:38:40 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:41 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:41 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:41 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:41 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:42 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:42 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:42 user nova-compute[72351]: DEBUG nova.compute.manager [req-3b44a049-245c-48eb-b314-767826772ceb req-bbcbfd76-0173-489e-89a5-1138ce42e2ff service nova] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Received event network-vif-plugged-8424518d-87b7-4fc4-8a58-1b4f094a5ca0 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:38:42 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-3b44a049-245c-48eb-b314-767826772ceb req-bbcbfd76-0173-489e-89a5-1138ce42e2ff service nova] Acquiring lock "bde1ef1e-163c-40eb-abb5-962b9064b7ab-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:38:42 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-3b44a049-245c-48eb-b314-767826772ceb req-bbcbfd76-0173-489e-89a5-1138ce42e2ff service nova] Lock "bde1ef1e-163c-40eb-abb5-962b9064b7ab-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:38:42 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-3b44a049-245c-48eb-b314-767826772ceb req-bbcbfd76-0173-489e-89a5-1138ce42e2ff service nova] Lock "bde1ef1e-163c-40eb-abb5-962b9064b7ab-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:38:42 user nova-compute[72351]: DEBUG nova.compute.manager [req-3b44a049-245c-48eb-b314-767826772ceb req-bbcbfd76-0173-489e-89a5-1138ce42e2ff service nova] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] No waiting events found dispatching network-vif-plugged-8424518d-87b7-4fc4-8a58-1b4f094a5ca0 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:38:42 user nova-compute[72351]: WARNING nova.compute.manager [req-3b44a049-245c-48eb-b314-767826772ceb req-bbcbfd76-0173-489e-89a5-1138ce42e2ff service nova] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Received unexpected event network-vif-plugged-8424518d-87b7-4fc4-8a58-1b4f094a5ca0 for instance with vm_state rescued and task_state unrescuing.
Aug 09 02:38:42 user nova-compute[72351]: DEBUG nova.compute.manager [req-3b44a049-245c-48eb-b314-767826772ceb req-bbcbfd76-0173-489e-89a5-1138ce42e2ff service nova] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Received event network-vif-plugged-8424518d-87b7-4fc4-8a58-1b4f094a5ca0 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:38:42 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-3b44a049-245c-48eb-b314-767826772ceb req-bbcbfd76-0173-489e-89a5-1138ce42e2ff service nova] Acquiring lock "bde1ef1e-163c-40eb-abb5-962b9064b7ab-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:38:42 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-3b44a049-245c-48eb-b314-767826772ceb req-bbcbfd76-0173-489e-89a5-1138ce42e2ff service nova] Lock "bde1ef1e-163c-40eb-abb5-962b9064b7ab-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:38:42 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-3b44a049-245c-48eb-b314-767826772ceb req-bbcbfd76-0173-489e-89a5-1138ce42e2ff service nova] Lock "bde1ef1e-163c-40eb-abb5-962b9064b7ab-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:38:42 user nova-compute[72351]: DEBUG nova.compute.manager [req-3b44a049-245c-48eb-b314-767826772ceb req-bbcbfd76-0173-489e-89a5-1138ce42e2ff service nova] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] No waiting events found dispatching network-vif-plugged-8424518d-87b7-4fc4-8a58-1b4f094a5ca0 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:38:42 user nova-compute[72351]: WARNING nova.compute.manager [req-3b44a049-245c-48eb-b314-767826772ceb req-bbcbfd76-0173-489e-89a5-1138ce42e2ff service nova] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Received unexpected event network-vif-plugged-8424518d-87b7-4fc4-8a58-1b4f094a5ca0 for instance with vm_state rescued and task_state unrescuing.
Aug 09 02:38:43 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:44 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Removed pending event for bde1ef1e-163c-40eb-abb5-962b9064b7ab due to event {{(pid=72351) _event_emit_delayed /opt/stack/nova/nova/virt/libvirt/host.py:438}}
Aug 09 02:38:44 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548724.0023053, bde1ef1e-163c-40eb-abb5-962b9064b7ab => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:38:44 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] VM Resumed (Lifecycle Event)
Aug 09 02:38:44 user nova-compute[72351]: DEBUG nova.compute.manager [None req-443fd43d-3a10-4134-8fe2-109903a56ded tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:38:44 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:38:44 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: rescued, current task_state: unrescuing, current DB power_state: 1, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:38:44 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] During sync_power_state the instance has a pending task (unrescuing). Skip.
Aug 09 02:38:44 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548724.004924, bde1ef1e-163c-40eb-abb5-962b9064b7ab => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:38:44 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] VM Started (Lifecycle Event)
Aug 09 02:38:44 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:38:44 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Synchronizing instance power state after lifecycle event "Started"; current vm_state: active, current task_state: None, current DB power_state: 1, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:38:44 user nova-compute[72351]: DEBUG nova.compute.manager [req-379b1434-78b3-4982-b3a8-bfb194599e41 req-ea978a37-288a-4b21-a344-2384e32d127f service nova] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Received event network-vif-plugged-8424518d-87b7-4fc4-8a58-1b4f094a5ca0 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:38:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-379b1434-78b3-4982-b3a8-bfb194599e41 req-ea978a37-288a-4b21-a344-2384e32d127f service nova] Acquiring lock "bde1ef1e-163c-40eb-abb5-962b9064b7ab-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:38:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-379b1434-78b3-4982-b3a8-bfb194599e41 req-ea978a37-288a-4b21-a344-2384e32d127f service nova] Lock "bde1ef1e-163c-40eb-abb5-962b9064b7ab-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:38:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-379b1434-78b3-4982-b3a8-bfb194599e41 req-ea978a37-288a-4b21-a344-2384e32d127f service nova] Lock "bde1ef1e-163c-40eb-abb5-962b9064b7ab-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:38:44 user nova-compute[72351]: DEBUG nova.compute.manager [req-379b1434-78b3-4982-b3a8-bfb194599e41 req-ea978a37-288a-4b21-a344-2384e32d127f service nova] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] No waiting events found dispatching network-vif-plugged-8424518d-87b7-4fc4-8a58-1b4f094a5ca0 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:38:44 user nova-compute[72351]: WARNING nova.compute.manager [req-379b1434-78b3-4982-b3a8-bfb194599e41 req-ea978a37-288a-4b21-a344-2384e32d127f service nova] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Received unexpected event network-vif-plugged-8424518d-87b7-4fc4-8a58-1b4f094a5ca0 for instance with vm_state active and task_state None.
Aug 09 02:38:45 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c4af19bd-25fd-47e4-b2da-ec687121ff8a tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Acquiring lock "585c81a5-ac24-4d12-a8a6-1e7b12a2d53a" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:38:45 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c4af19bd-25fd-47e4-b2da-ec687121ff8a tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "585c81a5-ac24-4d12-a8a6-1e7b12a2d53a" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:38:45 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c4af19bd-25fd-47e4-b2da-ec687121ff8a tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Acquiring lock "585c81a5-ac24-4d12-a8a6-1e7b12a2d53a-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:38:45 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c4af19bd-25fd-47e4-b2da-ec687121ff8a tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "585c81a5-ac24-4d12-a8a6-1e7b12a2d53a-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:38:45 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c4af19bd-25fd-47e4-b2da-ec687121ff8a tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "585c81a5-ac24-4d12-a8a6-1e7b12a2d53a-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:38:45 user nova-compute[72351]: INFO nova.compute.manager [None req-c4af19bd-25fd-47e4-b2da-ec687121ff8a tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Terminating instance
Aug 09 02:38:45 user nova-compute[72351]: DEBUG nova.compute.manager [None req-c4af19bd-25fd-47e4-b2da-ec687121ff8a tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Start destroying the instance on the hypervisor. {{(pid=72351) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3124}}
Aug 09 02:38:45 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:45 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:45 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:45 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:45 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:45 user nova-compute[72351]: DEBUG nova.compute.manager [req-de86f555-d6a3-4aa8-8652-fc3f72e0ca2f req-82b38832-d23d-4b98-a504-6d4cde881333 service nova] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Received event network-vif-unplugged-e521fae4-093d-4c92-a32d-684c1a5137f5 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:38:45 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-de86f555-d6a3-4aa8-8652-fc3f72e0ca2f req-82b38832-d23d-4b98-a504-6d4cde881333 service nova] Acquiring lock "585c81a5-ac24-4d12-a8a6-1e7b12a2d53a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:38:45 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-de86f555-d6a3-4aa8-8652-fc3f72e0ca2f req-82b38832-d23d-4b98-a504-6d4cde881333 service nova] Lock "585c81a5-ac24-4d12-a8a6-1e7b12a2d53a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:38:45 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-de86f555-d6a3-4aa8-8652-fc3f72e0ca2f req-82b38832-d23d-4b98-a504-6d4cde881333 service nova] Lock "585c81a5-ac24-4d12-a8a6-1e7b12a2d53a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:38:45 user nova-compute[72351]: DEBUG nova.compute.manager [req-de86f555-d6a3-4aa8-8652-fc3f72e0ca2f req-82b38832-d23d-4b98-a504-6d4cde881333 service nova] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] No waiting events found dispatching network-vif-unplugged-e521fae4-093d-4c92-a32d-684c1a5137f5 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:38:45 user nova-compute[72351]: DEBUG nova.compute.manager [req-de86f555-d6a3-4aa8-8652-fc3f72e0ca2f req-82b38832-d23d-4b98-a504-6d4cde881333 service nova] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Received event network-vif-unplugged-e521fae4-093d-4c92-a32d-684c1a5137f5 for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:38:46 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Instance destroyed successfully.
Aug 09 02:38:46 user nova-compute[72351]: DEBUG nova.objects.instance [None req-c4af19bd-25fd-47e4-b2da-ec687121ff8a tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lazy-loading 'resources' on Instance uuid 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:38:46 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-c4af19bd-25fd-47e4-b2da-ec687121ff8a tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,compute_id=1,config_drive='',created_at=2023-08-09T02:22:20Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestOtherA-server-446242587',display_name='tempest-ServerActionsTestOtherA-server-1834248220',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serveractionstestothera-server-446242587',id=19,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBB4HNXglactRKXsG6uozX3XR7A39DBllYpgpCz+5+1RhfLOeXdmVkNfDkIG7vQrgoVNKWK/MwqtjPDLEVURcnWQfOuv4ySYdfEPj4hs6FpDqfJbAa5qX0FAbZQuEg69FVQ==',key_name='tempest-keypair-976839412',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:26:38Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={rebuild='server'},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='69d584c230b94ff8aae722eb2f364c5c',ramdisk_id='',reservation_id='r-x0v0gmsw',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerActionsTestOtherA-1303549305',owner_user_name='tempest-ServerActionsTestOtherA-1303549305-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:26:39Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='5dca34fdadbb4386a1ff1cb255265d47',uuid=585c81a5-ac24-4d12-a8a6-1e7b12a2d53a,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "e521fae4-093d-4c92-a32d-684c1a5137f5", "address": "fa:16:3e:e5:b0:10", "network": {"id": "b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-210689817-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "69d584c230b94ff8aae722eb2f364c5c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape521fae4-09", "ovs_interfaceid": "e521fae4-093d-4c92-a32d-684c1a5137f5", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:38:46 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-c4af19bd-25fd-47e4-b2da-ec687121ff8a tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Converting VIF {"id": "e521fae4-093d-4c92-a32d-684c1a5137f5", "address": "fa:16:3e:e5:b0:10", "network": {"id": "b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-210689817-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "69d584c230b94ff8aae722eb2f364c5c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape521fae4-09", "ovs_interfaceid": "e521fae4-093d-4c92-a32d-684c1a5137f5", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:38:46 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-c4af19bd-25fd-47e4-b2da-ec687121ff8a tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:e5:b0:10,bridge_name='br-int',has_traffic_filtering=True,id=e521fae4-093d-4c92-a32d-684c1a5137f5,network=Network(b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape521fae4-09') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:38:46 user nova-compute[72351]: DEBUG os_vif [None req-c4af19bd-25fd-47e4-b2da-ec687121ff8a tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:e5:b0:10,bridge_name='br-int',has_traffic_filtering=True,id=e521fae4-093d-4c92-a32d-684c1a5137f5,network=Network(b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape521fae4-09') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:38:46 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:46 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tape521fae4-09, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:38:46 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:46 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:38:46 user nova-compute[72351]: INFO os_vif [None req-c4af19bd-25fd-47e4-b2da-ec687121ff8a tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:e5:b0:10,bridge_name='br-int',has_traffic_filtering=True,id=e521fae4-093d-4c92-a32d-684c1a5137f5,network=Network(b9aa5199-1e1f-4ae9-89d8-fd174c3ff67e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape521fae4-09')
Aug 09 02:38:46 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-c4af19bd-25fd-47e4-b2da-ec687121ff8a tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Deleting instance files /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a_del
Aug 09 02:38:46 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-c4af19bd-25fd-47e4-b2da-ec687121ff8a tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Deletion of /opt/stack/data/nova/instances/585c81a5-ac24-4d12-a8a6-1e7b12a2d53a_del complete
Aug 09 02:38:46 user nova-compute[72351]: INFO nova.compute.manager [None req-c4af19bd-25fd-47e4-b2da-ec687121ff8a tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Took 0.94 seconds to destroy the instance on the hypervisor.
Aug 09 02:38:46 user nova-compute[72351]: DEBUG oslo.service.loopingcall [None req-c4af19bd-25fd-47e4-b2da-ec687121ff8a tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72351) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Aug 09 02:38:46 user nova-compute[72351]: DEBUG nova.compute.manager [-] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Deallocating network for instance {{(pid=72351) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2263}}
Aug 09 02:38:46 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] deallocate_for_instance() {{(pid=72351) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1795}}
Aug 09 02:38:46 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:38:46 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Took 0.71 seconds to deallocate network for instance.
Aug 09 02:38:46 user nova-compute[72351]: DEBUG nova.compute.manager [req-1d7be7c8-b6c4-462c-adb7-eef2b2265183 req-d500a23c-b17b-4313-bcb5-03392c9d65ec service nova] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Received event network-vif-deleted-e521fae4-093d-4c92-a32d-684c1a5137f5 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:38:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c4af19bd-25fd-47e4-b2da-ec687121ff8a tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:38:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c4af19bd-25fd-47e4-b2da-ec687121ff8a tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:38:47 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-c4af19bd-25fd-47e4-b2da-ec687121ff8a tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:38:47 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-c4af19bd-25fd-47e4-b2da-ec687121ff8a tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:38:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c4af19bd-25fd-47e4-b2da-ec687121ff8a tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.308s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:38:47 user nova-compute[72351]: INFO nova.scheduler.client.report [None req-c4af19bd-25fd-47e4-b2da-ec687121ff8a tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Deleted allocations for instance 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a
Aug 09 02:38:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c4af19bd-25fd-47e4-b2da-ec687121ff8a tempest-ServerActionsTestOtherA-1303549305 tempest-ServerActionsTestOtherA-1303549305-project-member] Lock "585c81a5-ac24-4d12-a8a6-1e7b12a2d53a" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.216s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:38:47 user nova-compute[72351]: DEBUG nova.compute.manager [req-eac1d5e2-f013-4970-875a-e5f36f5046be req-5b6c812d-0ca7-43c7-8df3-4ea94f3f4b73 service nova] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Received event network-vif-plugged-e521fae4-093d-4c92-a32d-684c1a5137f5 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:38:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-eac1d5e2-f013-4970-875a-e5f36f5046be req-5b6c812d-0ca7-43c7-8df3-4ea94f3f4b73 service nova] Acquiring lock "585c81a5-ac24-4d12-a8a6-1e7b12a2d53a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:38:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-eac1d5e2-f013-4970-875a-e5f36f5046be req-5b6c812d-0ca7-43c7-8df3-4ea94f3f4b73 service nova] Lock "585c81a5-ac24-4d12-a8a6-1e7b12a2d53a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:38:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-eac1d5e2-f013-4970-875a-e5f36f5046be req-5b6c812d-0ca7-43c7-8df3-4ea94f3f4b73 service nova] Lock "585c81a5-ac24-4d12-a8a6-1e7b12a2d53a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:38:47 user nova-compute[72351]: DEBUG nova.compute.manager [req-eac1d5e2-f013-4970-875a-e5f36f5046be req-5b6c812d-0ca7-43c7-8df3-4ea94f3f4b73 service nova] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] No waiting events found dispatching network-vif-plugged-e521fae4-093d-4c92-a32d-684c1a5137f5 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:38:47 user nova-compute[72351]: WARNING nova.compute.manager [req-eac1d5e2-f013-4970-875a-e5f36f5046be req-5b6c812d-0ca7-43c7-8df3-4ea94f3f4b73 service nova] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Received unexpected event network-vif-plugged-e521fae4-093d-4c92-a32d-684c1a5137f5 for instance with vm_state deleted and task_state None.
Aug 09 02:38:48 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:51 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-4d2c4004-60fb-4551-96f1-c401e151e7d9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Acquiring lock "e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:38:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-4d2c4004-60fb-4551-96f1-c401e151e7d9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:38:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-4d2c4004-60fb-4551-96f1-c401e151e7d9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Acquiring lock "e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:38:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-4d2c4004-60fb-4551-96f1-c401e151e7d9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:38:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-4d2c4004-60fb-4551-96f1-c401e151e7d9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:38:54 user nova-compute[72351]: INFO nova.compute.manager [None req-4d2c4004-60fb-4551-96f1-c401e151e7d9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Terminating instance
Aug 09 02:38:54 user nova-compute[72351]: DEBUG nova.compute.manager [None req-4d2c4004-60fb-4551-96f1-c401e151e7d9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Start destroying the instance on the hypervisor. {{(pid=72351) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3124}}
Aug 09 02:38:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:54 user nova-compute[72351]: DEBUG nova.compute.manager [req-542f1211-87ed-40a0-be0b-86a2f184d0f6 req-0468b22c-75bd-4d7f-a145-be9a3d4dfe69 service nova] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Received event network-vif-unplugged-9481e457-9c23-4c45-b0bd-1faf63f8b7b8 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:38:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-542f1211-87ed-40a0-be0b-86a2f184d0f6 req-0468b22c-75bd-4d7f-a145-be9a3d4dfe69 service nova] Acquiring lock "e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:38:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-542f1211-87ed-40a0-be0b-86a2f184d0f6 req-0468b22c-75bd-4d7f-a145-be9a3d4dfe69 service nova] Lock "e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:38:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-542f1211-87ed-40a0-be0b-86a2f184d0f6 req-0468b22c-75bd-4d7f-a145-be9a3d4dfe69 service nova] Lock "e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:38:54 user nova-compute[72351]: DEBUG nova.compute.manager [req-542f1211-87ed-40a0-be0b-86a2f184d0f6 req-0468b22c-75bd-4d7f-a145-be9a3d4dfe69 service nova] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] No waiting events found dispatching network-vif-unplugged-9481e457-9c23-4c45-b0bd-1faf63f8b7b8 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:38:54 user nova-compute[72351]: DEBUG nova.compute.manager [req-542f1211-87ed-40a0-be0b-86a2f184d0f6 req-0468b22c-75bd-4d7f-a145-be9a3d4dfe69 service nova] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Received event network-vif-unplugged-9481e457-9c23-4c45-b0bd-1faf63f8b7b8 for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:38:55 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:55 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:55 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Instance destroyed successfully.
Aug 09 02:38:55 user nova-compute[72351]: DEBUG nova.objects.instance [None req-4d2c4004-60fb-4551-96f1-c401e151e7d9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lazy-loading 'resources' on Instance uuid e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:38:55 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-4d2c4004-60fb-4551-96f1-c401e151e7d9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:21:34Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerRescueNegativeTestJSON-server-938229174',display_name='tempest-ServerRescueNegativeTestJSON-server-938229174',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serverrescuenegativetestjson-server-938229174',id=16,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:26:27Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='3aea1e66805146f7a8cd1d97e64a24af',ramdisk_id='',reservation_id='r-14yazlt3',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerRescueNegativeTestJSON-289364438',owner_user_name='tempest-ServerRescueNegativeTestJSON-289364438-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:28:18Z,user_data=None,user_id='419b28d5a46c481db435715a30ab2614',uuid=e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "9481e457-9c23-4c45-b0bd-1faf63f8b7b8", "address": "fa:16:3e:69:03:30", "network": {"id": "49e8751c-e119-430f-9110-3dc6069b1589", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-899275135-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3aea1e66805146f7a8cd1d97e64a24af", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9481e457-9c", "ovs_interfaceid": "9481e457-9c23-4c45-b0bd-1faf63f8b7b8", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:38:55 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-4d2c4004-60fb-4551-96f1-c401e151e7d9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Converting VIF {"id": "9481e457-9c23-4c45-b0bd-1faf63f8b7b8", "address": "fa:16:3e:69:03:30", "network": {"id": "49e8751c-e119-430f-9110-3dc6069b1589", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-899275135-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3aea1e66805146f7a8cd1d97e64a24af", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9481e457-9c", "ovs_interfaceid": "9481e457-9c23-4c45-b0bd-1faf63f8b7b8", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:38:55 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-4d2c4004-60fb-4551-96f1-c401e151e7d9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:69:03:30,bridge_name='br-int',has_traffic_filtering=True,id=9481e457-9c23-4c45-b0bd-1faf63f8b7b8,network=Network(49e8751c-e119-430f-9110-3dc6069b1589),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap9481e457-9c') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:38:55 user nova-compute[72351]: DEBUG os_vif [None req-4d2c4004-60fb-4551-96f1-c401e151e7d9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:69:03:30,bridge_name='br-int',has_traffic_filtering=True,id=9481e457-9c23-4c45-b0bd-1faf63f8b7b8,network=Network(49e8751c-e119-430f-9110-3dc6069b1589),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap9481e457-9c') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:38:55 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:55 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap9481e457-9c, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:38:55 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:38:55 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:38:55 user nova-compute[72351]: INFO os_vif [None req-4d2c4004-60fb-4551-96f1-c401e151e7d9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:69:03:30,bridge_name='br-int',has_traffic_filtering=True,id=9481e457-9c23-4c45-b0bd-1faf63f8b7b8,network=Network(49e8751c-e119-430f-9110-3dc6069b1589),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap9481e457-9c')
Aug 09 02:38:55 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-4d2c4004-60fb-4551-96f1-c401e151e7d9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Deleting instance files /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335_del
Aug 09 02:38:55 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-4d2c4004-60fb-4551-96f1-c401e151e7d9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Deletion of /opt/stack/data/nova/instances/e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335_del complete
Aug 09 02:38:55 user nova-compute[72351]: INFO nova.compute.manager [None req-4d2c4004-60fb-4551-96f1-c401e151e7d9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Took 0.88 seconds to destroy the instance on the hypervisor.
Aug 09 02:38:55 user nova-compute[72351]: DEBUG oslo.service.loopingcall [None req-4d2c4004-60fb-4551-96f1-c401e151e7d9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72351) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Aug 09 02:38:55 user nova-compute[72351]: DEBUG nova.compute.manager [-] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Deallocating network for instance {{(pid=72351) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2263}}
Aug 09 02:38:55 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] deallocate_for_instance() {{(pid=72351) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1795}}
Aug 09 02:38:55 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:38:55 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Took 0.53 seconds to deallocate network for instance.
Aug 09 02:38:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-4d2c4004-60fb-4551-96f1-c401e151e7d9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:38:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-4d2c4004-60fb-4551-96f1-c401e151e7d9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:38:56 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-4d2c4004-60fb-4551-96f1-c401e151e7d9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:38:56 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-4d2c4004-60fb-4551-96f1-c401e151e7d9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:38:56 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-4d2c4004-60fb-4551-96f1-c401e151e7d9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.341s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:38:56 user nova-compute[72351]: INFO nova.scheduler.client.report [None req-4d2c4004-60fb-4551-96f1-c401e151e7d9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Deleted allocations for instance e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335
Aug 09 02:38:56 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-4d2c4004-60fb-4551-96f1-c401e151e7d9 tempest-ServerRescueNegativeTestJSON-289364438 tempest-ServerRescueNegativeTestJSON-289364438-project-member] Lock "e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 1.951s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:38:56 user nova-compute[72351]: DEBUG nova.compute.manager [req-b65e5708-9a5a-480e-997f-f8abfa7b0ac8 req-3632f843-ea7c-4f73-9a59-8b82a3633394 service nova] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Received event network-vif-plugged-9481e457-9c23-4c45-b0bd-1faf63f8b7b8 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:38:56 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-b65e5708-9a5a-480e-997f-f8abfa7b0ac8 req-3632f843-ea7c-4f73-9a59-8b82a3633394 service nova] Acquiring lock "e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:38:56 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-b65e5708-9a5a-480e-997f-f8abfa7b0ac8 req-3632f843-ea7c-4f73-9a59-8b82a3633394 service nova] Lock "e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:38:56 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-b65e5708-9a5a-480e-997f-f8abfa7b0ac8 req-3632f843-ea7c-4f73-9a59-8b82a3633394 service nova] Lock "e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:38:56 user nova-compute[72351]: DEBUG nova.compute.manager [req-b65e5708-9a5a-480e-997f-f8abfa7b0ac8 req-3632f843-ea7c-4f73-9a59-8b82a3633394 service nova] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] No waiting events found dispatching network-vif-plugged-9481e457-9c23-4c45-b0bd-1faf63f8b7b8 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:38:56 user nova-compute[72351]: WARNING nova.compute.manager [req-b65e5708-9a5a-480e-997f-f8abfa7b0ac8 req-3632f843-ea7c-4f73-9a59-8b82a3633394 service nova] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Received unexpected event network-vif-plugged-9481e457-9c23-4c45-b0bd-1faf63f8b7b8 for instance with vm_state deleted and task_state None.
Aug 09 02:38:56 user nova-compute[72351]: DEBUG nova.compute.manager [req-b65e5708-9a5a-480e-997f-f8abfa7b0ac8 req-3632f843-ea7c-4f73-9a59-8b82a3633394 service nova] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Received event network-vif-deleted-9481e457-9c23-4c45-b0bd-1faf63f8b7b8 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:38:58 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:39:00 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:39:01 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691548726.0660198, 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:39:01 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] VM Stopped (Lifecycle Event)
Aug 09 02:39:01 user nova-compute[72351]: DEBUG nova.compute.manager [None req-ce1502cd-1b58-48e7-b6ad-c91e246d06c0 None None] [instance: 585c81a5-ac24-4d12-a8a6-1e7b12a2d53a] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:39:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:39:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:39:07 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e96ec814-fb2f-4b15-b00b-031952d2c02d tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Acquiring lock "7bef74f4-3bb3-4ad0-bff7-34156be31249" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:39:07 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e96ec814-fb2f-4b15-b00b-031952d2c02d tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "7bef74f4-3bb3-4ad0-bff7-34156be31249" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:39:07 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e96ec814-fb2f-4b15-b00b-031952d2c02d tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Acquiring lock "7bef74f4-3bb3-4ad0-bff7-34156be31249-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:39:07 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e96ec814-fb2f-4b15-b00b-031952d2c02d tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "7bef74f4-3bb3-4ad0-bff7-34156be31249-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:39:07 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e96ec814-fb2f-4b15-b00b-031952d2c02d tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "7bef74f4-3bb3-4ad0-bff7-34156be31249-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:39:07 user nova-compute[72351]: INFO nova.compute.manager [None req-e96ec814-fb2f-4b15-b00b-031952d2c02d tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Terminating instance
Aug 09 02:39:07 user nova-compute[72351]: DEBUG nova.compute.manager [None req-e96ec814-fb2f-4b15-b00b-031952d2c02d tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Start destroying the instance on the hypervisor. {{(pid=72351) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3124}}
Aug 09 02:39:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:39:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:39:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:39:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:39:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:39:08 user nova-compute[72351]: DEBUG nova.compute.manager [req-ddf6dfa0-6b6c-4ba6-9c32-28e125dbe77a req-d981c4be-4f8c-4739-b0af-8a474c9b62cc service nova] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Received event network-vif-unplugged-d8727f3f-d838-442c-a9a8-278f38aa87d4 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:39:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ddf6dfa0-6b6c-4ba6-9c32-28e125dbe77a req-d981c4be-4f8c-4739-b0af-8a474c9b62cc service nova] Acquiring lock "7bef74f4-3bb3-4ad0-bff7-34156be31249-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:39:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ddf6dfa0-6b6c-4ba6-9c32-28e125dbe77a req-d981c4be-4f8c-4739-b0af-8a474c9b62cc service nova] Lock "7bef74f4-3bb3-4ad0-bff7-34156be31249-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:39:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ddf6dfa0-6b6c-4ba6-9c32-28e125dbe77a req-d981c4be-4f8c-4739-b0af-8a474c9b62cc service nova] Lock "7bef74f4-3bb3-4ad0-bff7-34156be31249-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:39:08 user nova-compute[72351]: DEBUG nova.compute.manager [req-ddf6dfa0-6b6c-4ba6-9c32-28e125dbe77a req-d981c4be-4f8c-4739-b0af-8a474c9b62cc service nova] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] No waiting events found dispatching network-vif-unplugged-d8727f3f-d838-442c-a9a8-278f38aa87d4 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:39:08 user nova-compute[72351]: DEBUG nova.compute.manager [req-ddf6dfa0-6b6c-4ba6-9c32-28e125dbe77a req-d981c4be-4f8c-4739-b0af-8a474c9b62cc service nova] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Received event network-vif-unplugged-d8727f3f-d838-442c-a9a8-278f38aa87d4 for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:39:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:39:08 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Instance destroyed successfully.
Aug 09 02:39:08 user nova-compute[72351]: DEBUG nova.objects.instance [None req-e96ec814-fb2f-4b15-b00b-031952d2c02d tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lazy-loading 'resources' on Instance uuid 7bef74f4-3bb3-4ad0-bff7-34156be31249 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:39:08 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-e96ec814-fb2f-4b15-b00b-031952d2c02d tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:31:38Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-SnapshotDataIntegrityTests-server-1527375297',display_name='tempest-SnapshotDataIntegrityTests-server-1527375297',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-snapshotdataintegritytests-server-1527375297',id=39,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKb0wYqi2SGirp8rEcWOo1bwnUn0HV+KTTBhhMO9lQaJ37WHi7lGFbj8YPdn6+1lmH+ZUmXq/caH8VRWPRYOy7Ei7LeEYXXQj0+wJQPa35fNJDjJMxBmuiONaPaoN4KW9w==',key_name='tempest-keypair-1816826569',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:31:47Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='0dd686dc07ca42f39aa898351ca281c2',ramdisk_id='',reservation_id='r-j1j10mu7',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-SnapshotDataIntegrityTests-1458031403',owner_user_name='tempest-SnapshotDataIntegrityTests-1458031403-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:31:47Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='1d460843653049dda604faa051159936',uuid=7bef74f4-3bb3-4ad0-bff7-34156be31249,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "d8727f3f-d838-442c-a9a8-278f38aa87d4", "address": "fa:16:3e:b4:8f:2e", "network": {"id": "626455de-9781-4ee8-b0be-c51ac124b189", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-1560904912-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.150", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0dd686dc07ca42f39aa898351ca281c2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd8727f3f-d8", "ovs_interfaceid": "d8727f3f-d838-442c-a9a8-278f38aa87d4", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:39:08 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-e96ec814-fb2f-4b15-b00b-031952d2c02d tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Converting VIF {"id": "d8727f3f-d838-442c-a9a8-278f38aa87d4", "address": "fa:16:3e:b4:8f:2e", "network": {"id": "626455de-9781-4ee8-b0be-c51ac124b189", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-1560904912-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.150", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0dd686dc07ca42f39aa898351ca281c2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd8727f3f-d8", "ovs_interfaceid": "d8727f3f-d838-442c-a9a8-278f38aa87d4", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:39:08 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-e96ec814-fb2f-4b15-b00b-031952d2c02d tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:b4:8f:2e,bridge_name='br-int',has_traffic_filtering=True,id=d8727f3f-d838-442c-a9a8-278f38aa87d4,network=Network(626455de-9781-4ee8-b0be-c51ac124b189),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd8727f3f-d8') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:39:08 user nova-compute[72351]: DEBUG os_vif [None req-e96ec814-fb2f-4b15-b00b-031952d2c02d tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:b4:8f:2e,bridge_name='br-int',has_traffic_filtering=True,id=d8727f3f-d838-442c-a9a8-278f38aa87d4,network=Network(626455de-9781-4ee8-b0be-c51ac124b189),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd8727f3f-d8') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:39:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:39:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapd8727f3f-d8, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:39:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:39:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:39:08 user nova-compute[72351]: INFO os_vif [None req-e96ec814-fb2f-4b15-b00b-031952d2c02d tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:b4:8f:2e,bridge_name='br-int',has_traffic_filtering=True,id=d8727f3f-d838-442c-a9a8-278f38aa87d4,network=Network(626455de-9781-4ee8-b0be-c51ac124b189),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd8727f3f-d8')
Aug 09 02:39:08 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-e96ec814-fb2f-4b15-b00b-031952d2c02d tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Deleting instance files /opt/stack/data/nova/instances/7bef74f4-3bb3-4ad0-bff7-34156be31249_del
Aug 09 02:39:08 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-e96ec814-fb2f-4b15-b00b-031952d2c02d tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Deletion of /opt/stack/data/nova/instances/7bef74f4-3bb3-4ad0-bff7-34156be31249_del complete
Aug 09 02:39:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:39:08 user nova-compute[72351]: INFO nova.compute.manager [None req-e96ec814-fb2f-4b15-b00b-031952d2c02d tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Took 0.84 seconds to destroy the instance on the hypervisor.
Aug 09 02:39:08 user nova-compute[72351]: DEBUG oslo.service.loopingcall [None req-e96ec814-fb2f-4b15-b00b-031952d2c02d tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72351) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Aug 09 02:39:08 user nova-compute[72351]: DEBUG nova.compute.manager [-] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Deallocating network for instance {{(pid=72351) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2263}}
Aug 09 02:39:08 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] deallocate_for_instance() {{(pid=72351) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1795}}
Aug 09 02:39:08 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:39:09 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:39:09 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72351) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10461}}
Aug 09 02:39:10 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:39:10 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Took 1.23 seconds to deallocate network for instance.
Aug 09 02:39:10 user nova-compute[72351]: DEBUG nova.compute.manager [req-2d6ffa08-338d-48dc-b803-6c095caea7f3 req-3858ffbd-830e-42ae-8cac-13b2d4f1f762 service nova] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Received event network-vif-deleted-d8727f3f-d838-442c-a9a8-278f38aa87d4 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:39:10 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691548735.069984, e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335 => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:39:10 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] VM Stopped (Lifecycle Event)
Aug 09 02:39:10 user nova-compute[72351]: DEBUG nova.compute.manager [None req-714199e7-9d1c-48ad-947c-77f9c9e07d79 None None] [instance: e05c2b7c-d2cf-45a6-9c2f-3a7a9edd7335] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:39:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e96ec814-fb2f-4b15-b00b-031952d2c02d tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:39:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e96ec814-fb2f-4b15-b00b-031952d2c02d tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:39:10 user nova-compute[72351]: DEBUG nova.compute.manager [req-c2fb789b-caea-47a9-8775-5c59d21907ae req-56be229d-c8f3-46a5-b7e5-13189f8a2dcf service nova] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Received event network-vif-plugged-d8727f3f-d838-442c-a9a8-278f38aa87d4 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:39:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c2fb789b-caea-47a9-8775-5c59d21907ae req-56be229d-c8f3-46a5-b7e5-13189f8a2dcf service nova] Acquiring lock "7bef74f4-3bb3-4ad0-bff7-34156be31249-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:39:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c2fb789b-caea-47a9-8775-5c59d21907ae req-56be229d-c8f3-46a5-b7e5-13189f8a2dcf service nova] Lock "7bef74f4-3bb3-4ad0-bff7-34156be31249-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:39:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c2fb789b-caea-47a9-8775-5c59d21907ae req-56be229d-c8f3-46a5-b7e5-13189f8a2dcf service nova] Lock "7bef74f4-3bb3-4ad0-bff7-34156be31249-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:39:10 user nova-compute[72351]: DEBUG nova.compute.manager [req-c2fb789b-caea-47a9-8775-5c59d21907ae req-56be229d-c8f3-46a5-b7e5-13189f8a2dcf service nova] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] No waiting events found dispatching network-vif-plugged-d8727f3f-d838-442c-a9a8-278f38aa87d4 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:39:10 user nova-compute[72351]: WARNING nova.compute.manager [req-c2fb789b-caea-47a9-8775-5c59d21907ae req-56be229d-c8f3-46a5-b7e5-13189f8a2dcf service nova] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Received unexpected event network-vif-plugged-d8727f3f-d838-442c-a9a8-278f38aa87d4 for instance with vm_state deleted and task_state None.
Aug 09 02:39:10 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-e96ec814-fb2f-4b15-b00b-031952d2c02d tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:39:10 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-e96ec814-fb2f-4b15-b00b-031952d2c02d tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:39:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e96ec814-fb2f-4b15-b00b-031952d2c02d tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.252s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:39:10 user nova-compute[72351]: INFO nova.scheduler.client.report [None req-e96ec814-fb2f-4b15-b00b-031952d2c02d tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Deleted allocations for instance 7bef74f4-3bb3-4ad0-bff7-34156be31249
Aug 09 02:39:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e96ec814-fb2f-4b15-b00b-031952d2c02d tempest-SnapshotDataIntegrityTests-1458031403 tempest-SnapshotDataIntegrityTests-1458031403-project-member] Lock "7bef74f4-3bb3-4ad0-bff7-34156be31249" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.506s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:39:11 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:39:12 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:39:12 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Starting heal instance info cache {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9842}}
Aug 09 02:39:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "refresh_cache-a7f3d076-864b-4b68-9207-214ce2c21444" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:39:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquired lock "refresh_cache-a7f3d076-864b-4b68-9207-214ce2c21444" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:39:12 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Forcefully refreshing network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1996}}
Aug 09 02:39:13 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Updating instance_info_cache with network_info: [{"id": "64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f", "address": "fa:16:3e:ed:9c:88", "network": {"id": "9a452414-d61f-4045-a6cc-9be93c892c4c", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "321111f935dd4e8aab4a083206ea9a7e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap64bd80e2-6c", "ovs_interfaceid": "64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:39:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Releasing lock "refresh_cache-a7f3d076-864b-4b68-9207-214ce2c21444" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:39:13 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Updated the network info_cache for instance {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9913}}
Aug 09 02:39:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:39:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:39:13 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:39:13 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:39:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:39:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:39:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:39:13 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Auditing locally available compute resources for user (node: user) {{(pid=72351) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Aug 09 02:39:14 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdg (vda) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:39:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/41b5e305-3973-4937-b262-d32db1091e20/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:39:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/41b5e305-3973-4937-b262-d32db1091e20/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:39:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/41b5e305-3973-4937-b262-d32db1091e20/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:39:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/41b5e305-3973-4937-b262-d32db1091e20/disk --force-share --output=json" returned: 0 in 0.155s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:39:14 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdh (vda) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:39:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6de75cee-9bf1-4f9f-8fa3-977d9432b805/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:39:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6de75cee-9bf1-4f9f-8fa3-977d9432b805/disk --force-share --output=json" returned: 0 in 0.146s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:39:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6de75cee-9bf1-4f9f-8fa3-977d9432b805/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:39:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6de75cee-9bf1-4f9f-8fa3-977d9432b805/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:39:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:39:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json" returned: 0 in 0.132s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:39:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:39:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json" returned: 0 in 0.131s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:39:15 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:39:15 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:39:15 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Hypervisor/Node resource view: name=user free_ram=8138MB free_disk=25.115459442138672GB free_vcpus=7 pci_devices=[{"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72351) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Aug 09 02:39:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:39:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:39:15 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance a7f3d076-864b-4b68-9207-214ce2c21444 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:39:15 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 957c1510-1320-4492-9052-a58ff540319d actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:39:15 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 41b5e305-3973-4937-b262-d32db1091e20 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:39:15 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance bde1ef1e-163c-40eb-abb5-962b9064b7ab actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:39:15 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 6de75cee-9bf1-4f9f-8fa3-977d9432b805 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:39:15 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Total usable vcpus: 12, total allocated vcpus: 5 {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Aug 09 02:39:15 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Final resource view: name=user phys_ram=16011MB used_ram=3072MB phys_disk=40GB used_disk=3GB total_vcpus=12 used_vcpus=5 pci_stats=[] {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Aug 09 02:39:15 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Refreshing inventories for resource provider 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:804}}
Aug 09 02:39:15 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Updating ProviderTree inventory for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 from _refresh_and_get_inventory using data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) _refresh_and_get_inventory /opt/stack/nova/nova/scheduler/client/report.py:768}}
Aug 09 02:39:15 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Updating inventory in ProviderTree for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 with inventory: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:176}}
Aug 09 02:39:15 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Refreshing aggregate associations for resource provider 8eae9b37-1236-4e6d-91a5-b650de27b009, aggregates: None {{(pid=72351) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:813}}
Aug 09 02:39:15 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Refreshing trait associations for resource provider 8eae9b37-1236-4e6d-91a5-b650de27b009, traits: COMPUTE_VIOMMU_MODEL_INTEL,COMPUTE_STORAGE_BUS_IDE,COMPUTE_GRAPHICS_MODEL_NONE,COMPUTE_NET_VIF_MODEL_E1000E,COMPUTE_IMAGE_TYPE_AKI,COMPUTE_VOLUME_ATTACH_WITH_TAG,COMPUTE_NET_VIF_MODEL_VMXNET3,COMPUTE_SOCKET_PCI_NUMA_AFFINITY,COMPUTE_ACCELERATORS,HW_CPU_X86_SSE41,COMPUTE_NET_ATTACH_INTERFACE_WITH_TAG,COMPUTE_GRAPHICS_MODEL_VGA,COMPUTE_GRAPHICS_MODEL_QXL,COMPUTE_VIOMMU_MODEL_AUTO,COMPUTE_SECURITY_UEFI_SECURE_BOOT,COMPUTE_NET_VIF_MODEL_E1000,COMPUTE_TRUSTED_CERTS,COMPUTE_VOLUME_MULTI_ATTACH,COMPUTE_STORAGE_BUS_FDC,COMPUTE_VOLUME_EXTEND,COMPUTE_GRAPHICS_MODEL_CIRRUS,COMPUTE_GRAPHICS_MODEL_VMVGA,COMPUTE_GRAPHICS_MODEL_BOCHS,COMPUTE_DEVICE_TAGGING,COMPUTE_STORAGE_BUS_SCSI,COMPUTE_STORAGE_BUS_VIRTIO,HW_CPU_X86_SSSE3,HW_CPU_X86_SSE2,COMPUTE_IMAGE_TYPE_AMI,COMPUTE_STORAGE_BUS_USB,COMPUTE_NET_VIF_MODEL_PCNET,COMPUTE_NET_VIF_MODEL_VIRTIO,COMPUTE_IMAGE_TYPE_ISO,HW_CPU_X86_SSE42,HW_CPU_X86_SSE,COMPUTE_IMAGE_TYPE_ARI,COMPUTE_IMAGE_TYPE_RAW,COMPUTE_IMAGE_TYPE_QCOW2,COMPUTE_NET_VIF_MODEL_NE2K_PCI,COMPUTE_NODE,HW_CPU_X86_MMX,COMPUTE_NET_VIF_MODEL_RTL8139,COMPUTE_GRAPHICS_MODEL_VIRTIO,COMPUTE_NET_VIF_MODEL_SPAPR_VLAN,COMPUTE_STORAGE_BUS_SATA,COMPUTE_NET_ATTACH_INTERFACE,COMPUTE_RESCUE_BFV {{(pid=72351) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:825}}
Aug 09 02:39:15 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:39:15 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:39:15 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Compute_service record updated for user:user {{(pid=72351) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Aug 09 02:39:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.518s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:39:16 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:39:16 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:39:16 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:39:16 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:39:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:39:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:39:23 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691548748.7062414, 7bef74f4-3bb3-4ad0-bff7-34156be31249 => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:39:23 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] VM Stopped (Lifecycle Event)
Aug 09 02:39:23 user nova-compute[72351]: DEBUG nova.compute.manager [None req-b9ece95c-70f3-4920-9d87-6ec40a0ec9c9 None None] [instance: 7bef74f4-3bb3-4ad0-bff7-34156be31249] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:39:23 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:39:28 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:39:33 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:39:33 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:39:33 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5001 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 02:39:33 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:39:33 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:39:33 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:39:38 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:39:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:39:43 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:39:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:39:47 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:39:48 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:39:48 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:39:48 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:39:51 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Acquiring lock "c740d131-98a9-4b14-a75a-a33aa7c72b51" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:39:51 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "c740d131-98a9-4b14-a75a-a33aa7c72b51" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.003s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:39:51 user nova-compute[72351]: DEBUG nova.compute.manager [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Starting instance... {{(pid=72351) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2406}}
Aug 09 02:39:51 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:39:51 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:39:51 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72351) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Aug 09 02:39:51 user nova-compute[72351]: INFO nova.compute.claims [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Claim successful on node user
Aug 09 02:39:51 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:39:51 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:39:51 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.297s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:39:51 user nova-compute[72351]: DEBUG nova.compute.manager [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Start building networks asynchronously for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2803}}
Aug 09 02:39:51 user nova-compute[72351]: DEBUG nova.compute.manager [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Allocating IP information in the background. {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1956}}
Aug 09 02:39:51 user nova-compute[72351]: DEBUG nova.network.neutron [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] allocate_for_instance() {{(pid=72351) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Aug 09 02:39:51 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Aug 09 02:39:51 user nova-compute[72351]: DEBUG nova.compute.manager [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Start building block device mappings for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2838}}
Aug 09 02:39:52 user nova-compute[72351]: DEBUG nova.policy [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'ad8fc03436a84bdfb13ba6abecd4f27b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ff300d27c9a4dd3b7ecd507b3627d98', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72351) authorize /opt/stack/nova/nova/policy.py:203}}
Aug 09 02:39:52 user nova-compute[72351]: DEBUG nova.compute.manager [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Start spawning the instance on the hypervisor. {{(pid=72351) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2612}}
Aug 09 02:39:52 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Creating instance directory {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4717}}
Aug 09 02:39:52 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Creating image(s)
Aug 09 02:39:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Acquiring lock "/opt/stack/data/nova/instances/c740d131-98a9-4b14-a75a-a33aa7c72b51/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:39:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "/opt/stack/data/nova/instances/c740d131-98a9-4b14-a75a-a33aa7c72b51/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:39:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "/opt/stack/data/nova/instances/c740d131-98a9-4b14-a75a-a33aa7c72b51/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:39:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:39:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.139s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:39:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Acquiring lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:39:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:39:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:39:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.139s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:39:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/c740d131-98a9-4b14-a75a-a33aa7c72b51/disk 1073741824 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:39:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/c740d131-98a9-4b14-a75a-a33aa7c72b51/disk 1073741824" returned: 0 in 0.050s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:39:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.195s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:39:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:39:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.137s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:39:52 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Checking if we can resize image /opt/stack/data/nova/instances/c740d131-98a9-4b14-a75a-a33aa7c72b51/disk. size=1073741824 {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Aug 09 02:39:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c740d131-98a9-4b14-a75a-a33aa7c72b51/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:39:52 user nova-compute[72351]: DEBUG nova.network.neutron [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Successfully created port: ef71c011-3c08-4927-bf2b-763a63fe8c6f {{(pid=72351) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Aug 09 02:39:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c740d131-98a9-4b14-a75a-a33aa7c72b51/disk --force-share --output=json" returned: 0 in 0.144s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:39:52 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Cannot resize image /opt/stack/data/nova/instances/c740d131-98a9-4b14-a75a-a33aa7c72b51/disk to a smaller size. {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Aug 09 02:39:52 user nova-compute[72351]: DEBUG nova.objects.instance [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lazy-loading 'migration_context' on Instance uuid c740d131-98a9-4b14-a75a-a33aa7c72b51 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:39:52 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Created local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4851}}
Aug 09 02:39:52 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Ensure instance console log exists: /opt/stack/data/nova/instances/c740d131-98a9-4b14-a75a-a33aa7c72b51/console.log {{(pid=72351) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4603}}
Aug 09 02:39:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:39:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:39:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:39:53 user nova-compute[72351]: DEBUG nova.network.neutron [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Successfully updated port: ef71c011-3c08-4927-bf2b-763a63fe8c6f {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:39:53 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Acquiring lock "refresh_cache-c740d131-98a9-4b14-a75a-a33aa7c72b51" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:39:53 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Acquired lock "refresh_cache-c740d131-98a9-4b14-a75a-a33aa7c72b51" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:39:53 user nova-compute[72351]: DEBUG nova.network.neutron [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:39:53 user nova-compute[72351]: DEBUG nova.compute.manager [req-12793596-60de-45db-b049-1f95adca7a68 req-921b33ac-2ca6-4575-9c6e-a02bdbb5733b service nova] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Received event network-changed-ef71c011-3c08-4927-bf2b-763a63fe8c6f {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:39:53 user nova-compute[72351]: DEBUG nova.compute.manager [req-12793596-60de-45db-b049-1f95adca7a68 req-921b33ac-2ca6-4575-9c6e-a02bdbb5733b service nova] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Refreshing instance network info cache due to event network-changed-ef71c011-3c08-4927-bf2b-763a63fe8c6f. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:39:53 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-12793596-60de-45db-b049-1f95adca7a68 req-921b33ac-2ca6-4575-9c6e-a02bdbb5733b service nova] Acquiring lock "refresh_cache-c740d131-98a9-4b14-a75a-a33aa7c72b51" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:39:53 user nova-compute[72351]: DEBUG nova.network.neutron [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:39:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:39:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:39:53 user nova-compute[72351]: DEBUG nova.network.neutron [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Updating instance_info_cache with network_info: [{"id": "ef71c011-3c08-4927-bf2b-763a63fe8c6f", "address": "fa:16:3e:73:9f:90", "network": {"id": "da9be96c-ce20-4037-999c-230ca799fb06", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-661780305-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5ff300d27c9a4dd3b7ecd507b3627d98", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapef71c011-3c", "ovs_interfaceid": "ef71c011-3c08-4927-bf2b-763a63fe8c6f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:39:53 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Releasing lock "refresh_cache-c740d131-98a9-4b14-a75a-a33aa7c72b51" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:39:53 user nova-compute[72351]: DEBUG nova.compute.manager [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Instance network_info: |[{"id": "ef71c011-3c08-4927-bf2b-763a63fe8c6f", "address": "fa:16:3e:73:9f:90", "network": {"id": "da9be96c-ce20-4037-999c-230ca799fb06", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-661780305-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5ff300d27c9a4dd3b7ecd507b3627d98", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapef71c011-3c", "ovs_interfaceid": "ef71c011-3c08-4927-bf2b-763a63fe8c6f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1971}}
Aug 09 02:39:53 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-12793596-60de-45db-b049-1f95adca7a68 req-921b33ac-2ca6-4575-9c6e-a02bdbb5733b service nova] Acquired lock "refresh_cache-c740d131-98a9-4b14-a75a-a33aa7c72b51" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:39:53 user nova-compute[72351]: DEBUG nova.network.neutron [req-12793596-60de-45db-b049-1f95adca7a68 req-921b33ac-2ca6-4575-9c6e-a02bdbb5733b service nova] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Refreshing network info cache for port ef71c011-3c08-4927-bf2b-763a63fe8c6f {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:39:53 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Start _get_guest_xml network_info=[{"id": "ef71c011-3c08-4927-bf2b-763a63fe8c6f", "address": "fa:16:3e:73:9f:90", "network": {"id": "da9be96c-ce20-4037-999c-230ca799fb06", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-661780305-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5ff300d27c9a4dd3b7ecd507b3627d98", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapef71c011-3c", "ovs_interfaceid": "ef71c011-3c08-4927-bf2b-763a63fe8c6f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'size': 0, 'device_type': 'disk', 'encryption_secret_uuid': None, 'guest_format': None, 'encryption_options': None, 'disk_bus': 'virtio', 'encrypted': False, 'encryption_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'image_id': '18ccda7f-31d3-4978-ab9d-4ed2f74eff94'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:39:53 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:39:53 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:39:53 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:39:53 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:39:53 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:39:53 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:39:53 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:39:53 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:39:53 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:39:53 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:39:53 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:39:53 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:39:53 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:39:53 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:39:53 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:39:53 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:39:53 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:39:53 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:39:53 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:39:51Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-server-1075911547',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumeshelvetestjson-server-1075911547',id=45,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBAhm3wxNXc3qjoZ/WjO45nlSmWKHkz9mk2gcB1iSKP8F60j8gzonzjkrypbEG5qNiTKrTFWzLZuC0FgGJv+9yI6H0VXa1JB0Qe7iMGyt2g59NXDdngUdeDSufXPbcrFG6Q==',key_name='tempest-keypair-1369286576',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='5ff300d27c9a4dd3b7ecd507b3627d98',ramdisk_id='',reservation_id='r-gx6s8m4c',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeShelveTestJSON-1982056968',owner_user_name='tempest-AttachVolumeShelveTestJSON-1982056968-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:39:52Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='ad8fc03436a84bdfb13ba6abecd4f27b',uuid=c740d131-98a9-4b14-a75a-a33aa7c72b51,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "ef71c011-3c08-4927-bf2b-763a63fe8c6f", "address": "fa:16:3e:73:9f:90", "network": {"id": "da9be96c-ce20-4037-999c-230ca799fb06", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-661780305-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5ff300d27c9a4dd3b7ecd507b3627d98", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapef71c011-3c", "ovs_interfaceid": "ef71c011-3c08-4927-bf2b-763a63fe8c6f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:39:53 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Converting VIF {"id": "ef71c011-3c08-4927-bf2b-763a63fe8c6f", "address": "fa:16:3e:73:9f:90", "network": {"id": "da9be96c-ce20-4037-999c-230ca799fb06", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-661780305-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5ff300d27c9a4dd3b7ecd507b3627d98", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapef71c011-3c", "ovs_interfaceid": "ef71c011-3c08-4927-bf2b-763a63fe8c6f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:39:53 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:73:9f:90,bridge_name='br-int',has_traffic_filtering=True,id=ef71c011-3c08-4927-bf2b-763a63fe8c6f,network=Network(da9be96c-ce20-4037-999c-230ca799fb06),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapef71c011-3c') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:39:53 user nova-compute[72351]: DEBUG nova.objects.instance [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lazy-loading 'pci_devices' on Instance uuid c740d131-98a9-4b14-a75a-a33aa7c72b51 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:39:53 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:39:53 user nova-compute[72351]:   <uuid>c740d131-98a9-4b14-a75a-a33aa7c72b51</uuid>
Aug 09 02:39:53 user nova-compute[72351]:   <name>instance-0000002d</name>
Aug 09 02:39:53 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:39:53 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:39:53 user nova-compute[72351]:   <metadata>
Aug 09 02:39:53 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:39:53 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:39:53 user nova-compute[72351]:       <nova:name>tempest-AttachVolumeShelveTestJSON-server-1075911547</nova:name>
Aug 09 02:39:53 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:39:53</nova:creationTime>
Aug 09 02:39:53 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:39:53 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:39:53 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:39:53 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:39:53 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:39:53 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:39:53 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:39:53 user nova-compute[72351]:       <nova:owner>
Aug 09 02:39:53 user nova-compute[72351]:         <nova:user uuid="ad8fc03436a84bdfb13ba6abecd4f27b">tempest-AttachVolumeShelveTestJSON-1982056968-project-member</nova:user>
Aug 09 02:39:53 user nova-compute[72351]:         <nova:project uuid="5ff300d27c9a4dd3b7ecd507b3627d98">tempest-AttachVolumeShelveTestJSON-1982056968</nova:project>
Aug 09 02:39:53 user nova-compute[72351]:       </nova:owner>
Aug 09 02:39:53 user nova-compute[72351]:       <nova:root type="image" uuid="18ccda7f-31d3-4978-ab9d-4ed2f74eff94"/>
Aug 09 02:39:53 user nova-compute[72351]:       <nova:ports>
Aug 09 02:39:53 user nova-compute[72351]:         <nova:port uuid="ef71c011-3c08-4927-bf2b-763a63fe8c6f">
Aug 09 02:39:53 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.13" ipVersion="4"/>
Aug 09 02:39:53 user nova-compute[72351]:         </nova:port>
Aug 09 02:39:53 user nova-compute[72351]:       </nova:ports>
Aug 09 02:39:53 user nova-compute[72351]:     </nova:instance>
Aug 09 02:39:53 user nova-compute[72351]:   </metadata>
Aug 09 02:39:53 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:39:53 user nova-compute[72351]:     <system>
Aug 09 02:39:53 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:39:53 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:39:53 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:39:53 user nova-compute[72351]:       <entry name="serial">c740d131-98a9-4b14-a75a-a33aa7c72b51</entry>
Aug 09 02:39:53 user nova-compute[72351]:       <entry name="uuid">c740d131-98a9-4b14-a75a-a33aa7c72b51</entry>
Aug 09 02:39:53 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:39:53 user nova-compute[72351]:     </system>
Aug 09 02:39:53 user nova-compute[72351]:   </sysinfo>
Aug 09 02:39:53 user nova-compute[72351]:   <os>
Aug 09 02:39:53 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:39:53 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:39:53 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:39:53 user nova-compute[72351]:   </os>
Aug 09 02:39:53 user nova-compute[72351]:   <features>
Aug 09 02:39:53 user nova-compute[72351]:     <acpi/>
Aug 09 02:39:53 user nova-compute[72351]:     <apic/>
Aug 09 02:39:53 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:39:53 user nova-compute[72351]:   </features>
Aug 09 02:39:53 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:39:53 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:39:53 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:39:53 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:39:53 user nova-compute[72351]:   </clock>
Aug 09 02:39:53 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:39:53 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:39:53 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:39:53 user nova-compute[72351]:   </cpu>
Aug 09 02:39:53 user nova-compute[72351]:   <devices>
Aug 09 02:39:53 user nova-compute[72351]:     <disk type="file" device="disk">
Aug 09 02:39:53 user nova-compute[72351]:       <driver name="qemu" type="qcow2" cache="none"/>
Aug 09 02:39:53 user nova-compute[72351]:       <source file="/opt/stack/data/nova/instances/c740d131-98a9-4b14-a75a-a33aa7c72b51/disk"/>
Aug 09 02:39:53 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:39:53 user nova-compute[72351]:     </disk>
Aug 09 02:39:53 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:39:53 user nova-compute[72351]:       <mac address="fa:16:3e:73:9f:90"/>
Aug 09 02:39:53 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:39:53 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:39:53 user nova-compute[72351]:       <target dev="tapef71c011-3c"/>
Aug 09 02:39:53 user nova-compute[72351]:     </interface>
Aug 09 02:39:53 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:39:53 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/c740d131-98a9-4b14-a75a-a33aa7c72b51/console.log" append="off"/>
Aug 09 02:39:53 user nova-compute[72351]:     </serial>
Aug 09 02:39:53 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:39:53 user nova-compute[72351]:     <video>
Aug 09 02:39:53 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:39:53 user nova-compute[72351]:     </video>
Aug 09 02:39:53 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:39:53 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:39:53 user nova-compute[72351]:     </rng>
Aug 09 02:39:53 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:39:53 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:39:53 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:39:53 user nova-compute[72351]:     </memballoon>
Aug 09 02:39:53 user nova-compute[72351]:   </devices>
Aug 09 02:39:53 user nova-compute[72351]: </domain>
Aug 09 02:39:53 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:39:53 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:39:51Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-server-1075911547',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumeshelvetestjson-server-1075911547',id=45,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBAhm3wxNXc3qjoZ/WjO45nlSmWKHkz9mk2gcB1iSKP8F60j8gzonzjkrypbEG5qNiTKrTFWzLZuC0FgGJv+9yI6H0VXa1JB0Qe7iMGyt2g59NXDdngUdeDSufXPbcrFG6Q==',key_name='tempest-keypair-1369286576',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='5ff300d27c9a4dd3b7ecd507b3627d98',ramdisk_id='',reservation_id='r-gx6s8m4c',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeShelveTestJSON-1982056968',owner_user_name='tempest-AttachVolumeShelveTestJSON-1982056968-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:39:52Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='ad8fc03436a84bdfb13ba6abecd4f27b',uuid=c740d131-98a9-4b14-a75a-a33aa7c72b51,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "ef71c011-3c08-4927-bf2b-763a63fe8c6f", "address": "fa:16:3e:73:9f:90", "network": {"id": "da9be96c-ce20-4037-999c-230ca799fb06", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-661780305-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5ff300d27c9a4dd3b7ecd507b3627d98", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapef71c011-3c", "ovs_interfaceid": "ef71c011-3c08-4927-bf2b-763a63fe8c6f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:39:53 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Converting VIF {"id": "ef71c011-3c08-4927-bf2b-763a63fe8c6f", "address": "fa:16:3e:73:9f:90", "network": {"id": "da9be96c-ce20-4037-999c-230ca799fb06", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-661780305-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5ff300d27c9a4dd3b7ecd507b3627d98", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapef71c011-3c", "ovs_interfaceid": "ef71c011-3c08-4927-bf2b-763a63fe8c6f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:39:53 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:73:9f:90,bridge_name='br-int',has_traffic_filtering=True,id=ef71c011-3c08-4927-bf2b-763a63fe8c6f,network=Network(da9be96c-ce20-4037-999c-230ca799fb06),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapef71c011-3c') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:39:53 user nova-compute[72351]: DEBUG os_vif [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:73:9f:90,bridge_name='br-int',has_traffic_filtering=True,id=ef71c011-3c08-4927-bf2b-763a63fe8c6f,network=Network(da9be96c-ce20-4037-999c-230ca799fb06),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapef71c011-3c') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:39:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:39:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:39:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:39:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:39:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapef71c011-3c, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:39:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapef71c011-3c, col_values=(('external_ids', {'iface-id': 'ef71c011-3c08-4927-bf2b-763a63fe8c6f', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:73:9f:90', 'vm-uuid': 'c740d131-98a9-4b14-a75a-a33aa7c72b51'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:39:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:39:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:39:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:39:53 user nova-compute[72351]: INFO os_vif [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:73:9f:90,bridge_name='br-int',has_traffic_filtering=True,id=ef71c011-3c08-4927-bf2b-763a63fe8c6f,network=Network(da9be96c-ce20-4037-999c-230ca799fb06),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapef71c011-3c')
Aug 09 02:39:53 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:39:53 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] No VIF found with MAC fa:16:3e:73:9f:90, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:39:54 user nova-compute[72351]: DEBUG nova.network.neutron [req-12793596-60de-45db-b049-1f95adca7a68 req-921b33ac-2ca6-4575-9c6e-a02bdbb5733b service nova] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Updated VIF entry in instance network info cache for port ef71c011-3c08-4927-bf2b-763a63fe8c6f. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:39:54 user nova-compute[72351]: DEBUG nova.network.neutron [req-12793596-60de-45db-b049-1f95adca7a68 req-921b33ac-2ca6-4575-9c6e-a02bdbb5733b service nova] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Updating instance_info_cache with network_info: [{"id": "ef71c011-3c08-4927-bf2b-763a63fe8c6f", "address": "fa:16:3e:73:9f:90", "network": {"id": "da9be96c-ce20-4037-999c-230ca799fb06", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-661780305-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5ff300d27c9a4dd3b7ecd507b3627d98", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapef71c011-3c", "ovs_interfaceid": "ef71c011-3c08-4927-bf2b-763a63fe8c6f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:39:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-12793596-60de-45db-b049-1f95adca7a68 req-921b33ac-2ca6-4575-9c6e-a02bdbb5733b service nova] Releasing lock "refresh_cache-c740d131-98a9-4b14-a75a-a33aa7c72b51" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:39:55 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:39:55 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:39:55 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:39:55 user nova-compute[72351]: DEBUG nova.compute.manager [req-8c1a7eae-4a17-4bd5-b4c8-a3fe600a2619 req-f5656b13-65d1-4cec-87c0-bc6728d8ff98 service nova] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Received event network-vif-plugged-ef71c011-3c08-4927-bf2b-763a63fe8c6f {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:39:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-8c1a7eae-4a17-4bd5-b4c8-a3fe600a2619 req-f5656b13-65d1-4cec-87c0-bc6728d8ff98 service nova] Acquiring lock "c740d131-98a9-4b14-a75a-a33aa7c72b51-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:39:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-8c1a7eae-4a17-4bd5-b4c8-a3fe600a2619 req-f5656b13-65d1-4cec-87c0-bc6728d8ff98 service nova] Lock "c740d131-98a9-4b14-a75a-a33aa7c72b51-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:39:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-8c1a7eae-4a17-4bd5-b4c8-a3fe600a2619 req-f5656b13-65d1-4cec-87c0-bc6728d8ff98 service nova] Lock "c740d131-98a9-4b14-a75a-a33aa7c72b51-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:39:55 user nova-compute[72351]: DEBUG nova.compute.manager [req-8c1a7eae-4a17-4bd5-b4c8-a3fe600a2619 req-f5656b13-65d1-4cec-87c0-bc6728d8ff98 service nova] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] No waiting events found dispatching network-vif-plugged-ef71c011-3c08-4927-bf2b-763a63fe8c6f {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:39:55 user nova-compute[72351]: WARNING nova.compute.manager [req-8c1a7eae-4a17-4bd5-b4c8-a3fe600a2619 req-f5656b13-65d1-4cec-87c0-bc6728d8ff98 service nova] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Received unexpected event network-vif-plugged-ef71c011-3c08-4927-bf2b-763a63fe8c6f for instance with vm_state building and task_state spawning.
Aug 09 02:39:55 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:39:55 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:39:55 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:39:57 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548797.4456522, c740d131-98a9-4b14-a75a-a33aa7c72b51 => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:39:57 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] VM Resumed (Lifecycle Event)
Aug 09 02:39:57 user nova-compute[72351]: DEBUG nova.compute.manager [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:39:57 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Guest created on hypervisor {{(pid=72351) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4411}}
Aug 09 02:39:57 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Instance spawned successfully.
Aug 09 02:39:57 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:894}}
Aug 09 02:39:57 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:39:57 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:39:57 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Found default for hw_cdrom_bus of ide {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:39:57 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Found default for hw_disk_bus of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:39:57 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Found default for hw_input_bus of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:39:57 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Found default for hw_pointer_model of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:39:57 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Found default for hw_video_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:39:57 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Found default for hw_vif_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:39:57 user nova-compute[72351]: DEBUG nova.compute.manager [req-1b7738d9-e0df-4492-9b52-1e6df30a50ab req-fa2c3b7d-9921-40f6-9d7f-80bc95dc5d6e service nova] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Received event network-vif-plugged-ef71c011-3c08-4927-bf2b-763a63fe8c6f {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:39:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-1b7738d9-e0df-4492-9b52-1e6df30a50ab req-fa2c3b7d-9921-40f6-9d7f-80bc95dc5d6e service nova] Acquiring lock "c740d131-98a9-4b14-a75a-a33aa7c72b51-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:39:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-1b7738d9-e0df-4492-9b52-1e6df30a50ab req-fa2c3b7d-9921-40f6-9d7f-80bc95dc5d6e service nova] Lock "c740d131-98a9-4b14-a75a-a33aa7c72b51-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:39:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-1b7738d9-e0df-4492-9b52-1e6df30a50ab req-fa2c3b7d-9921-40f6-9d7f-80bc95dc5d6e service nova] Lock "c740d131-98a9-4b14-a75a-a33aa7c72b51-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:39:57 user nova-compute[72351]: DEBUG nova.compute.manager [req-1b7738d9-e0df-4492-9b52-1e6df30a50ab req-fa2c3b7d-9921-40f6-9d7f-80bc95dc5d6e service nova] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] No waiting events found dispatching network-vif-plugged-ef71c011-3c08-4927-bf2b-763a63fe8c6f {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:39:57 user nova-compute[72351]: WARNING nova.compute.manager [req-1b7738d9-e0df-4492-9b52-1e6df30a50ab req-fa2c3b7d-9921-40f6-9d7f-80bc95dc5d6e service nova] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Received unexpected event network-vif-plugged-ef71c011-3c08-4927-bf2b-763a63fe8c6f for instance with vm_state building and task_state spawning.
Aug 09 02:39:57 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:39:57 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548797.451222, c740d131-98a9-4b14-a75a-a33aa7c72b51 => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:39:57 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] VM Started (Lifecycle Event)
Aug 09 02:39:57 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:39:57 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:39:57 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:39:57 user nova-compute[72351]: INFO nova.compute.manager [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Took 5.46 seconds to spawn the instance on the hypervisor.
Aug 09 02:39:57 user nova-compute[72351]: DEBUG nova.compute.manager [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:39:57 user nova-compute[72351]: INFO nova.compute.manager [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Took 6.12 seconds to build instance.
Aug 09 02:39:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-0850e753-7c92-4896-a870-5398f97bf03f tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "c740d131-98a9-4b14-a75a-a33aa7c72b51" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 6.234s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:39:58 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:39:58 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:40:02 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:40:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:40:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:40:06 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:40:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:40:09 user nova-compute[72351]: DEBUG nova.compute.manager [req-1d5a14a9-91a6-4036-8285-eee5f454331f req-443eb3d3-1bfb-4ef2-886f-827059b21ce9 service nova] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Received event network-changed-bacefb00-adc9-4430-b7e1-83c6200d6144 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:40:09 user nova-compute[72351]: DEBUG nova.compute.manager [req-1d5a14a9-91a6-4036-8285-eee5f454331f req-443eb3d3-1bfb-4ef2-886f-827059b21ce9 service nova] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Refreshing instance network info cache due to event network-changed-bacefb00-adc9-4430-b7e1-83c6200d6144. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:40:09 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-1d5a14a9-91a6-4036-8285-eee5f454331f req-443eb3d3-1bfb-4ef2-886f-827059b21ce9 service nova] Acquiring lock "refresh_cache-6de75cee-9bf1-4f9f-8fa3-977d9432b805" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:40:09 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-1d5a14a9-91a6-4036-8285-eee5f454331f req-443eb3d3-1bfb-4ef2-886f-827059b21ce9 service nova] Acquired lock "refresh_cache-6de75cee-9bf1-4f9f-8fa3-977d9432b805" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:40:09 user nova-compute[72351]: DEBUG nova.network.neutron [req-1d5a14a9-91a6-4036-8285-eee5f454331f req-443eb3d3-1bfb-4ef2-886f-827059b21ce9 service nova] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Refreshing network info cache for port bacefb00-adc9-4430-b7e1-83c6200d6144 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:40:09 user nova-compute[72351]: DEBUG nova.network.neutron [req-1d5a14a9-91a6-4036-8285-eee5f454331f req-443eb3d3-1bfb-4ef2-886f-827059b21ce9 service nova] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Updated VIF entry in instance network info cache for port bacefb00-adc9-4430-b7e1-83c6200d6144. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:40:09 user nova-compute[72351]: DEBUG nova.network.neutron [req-1d5a14a9-91a6-4036-8285-eee5f454331f req-443eb3d3-1bfb-4ef2-886f-827059b21ce9 service nova] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Updating instance_info_cache with network_info: [{"id": "bacefb00-adc9-4430-b7e1-83c6200d6144", "address": "fa:16:3e:1a:c1:d6", "network": {"id": "47c43230-a244-4198-84e4-9ed98c005cd9", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1392403106-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.96", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c07a6b254414b41a673468238d3aa62", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbacefb00-ad", "ovs_interfaceid": "bacefb00-adc9-4430-b7e1-83c6200d6144", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:40:09 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-1d5a14a9-91a6-4036-8285-eee5f454331f req-443eb3d3-1bfb-4ef2-886f-827059b21ce9 service nova] Releasing lock "refresh_cache-6de75cee-9bf1-4f9f-8fa3-977d9432b805" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:40:09 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:40:10 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:40:10 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72351) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10461}}
Aug 09 02:40:12 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:40:12 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Starting heal instance info cache {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9842}}
Aug 09 02:40:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "refresh_cache-957c1510-1320-4492-9052-a58ff540319d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:40:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquired lock "refresh_cache-957c1510-1320-4492-9052-a58ff540319d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:40:12 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 957c1510-1320-4492-9052-a58ff540319d] Forcefully refreshing network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1996}}
Aug 09 02:40:13 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 957c1510-1320-4492-9052-a58ff540319d] Updating instance_info_cache with network_info: [{"id": "e679cfd9-123f-4b88-8fd2-397e4c8616cc", "address": "fa:16:3e:7f:d6:1b", "network": {"id": "9a452414-d61f-4045-a6cc-9be93c892c4c", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "321111f935dd4e8aab4a083206ea9a7e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape679cfd9-12", "ovs_interfaceid": "e679cfd9-123f-4b88-8fd2-397e4c8616cc", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:40:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Releasing lock "refresh_cache-957c1510-1320-4492-9052-a58ff540319d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:40:13 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 957c1510-1320-4492-9052-a58ff540319d] Updated the network info_cache for instance {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9913}}
Aug 09 02:40:13 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:40:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:40:13 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:40:13 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:40:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:40:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:40:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:40:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:40:13 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Auditing locally available compute resources for user (node: user) {{(pid=72351) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Aug 09 02:40:14 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdg (vda) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:40:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/41b5e305-3973-4937-b262-d32db1091e20/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:40:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/41b5e305-3973-4937-b262-d32db1091e20/disk --force-share --output=json" returned: 0 in 0.146s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:40:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/41b5e305-3973-4937-b262-d32db1091e20/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:40:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/41b5e305-3973-4937-b262-d32db1091e20/disk --force-share --output=json" returned: 0 in 0.153s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:40:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c740d131-98a9-4b14-a75a-a33aa7c72b51/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:40:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c740d131-98a9-4b14-a75a-a33aa7c72b51/disk --force-share --output=json" returned: 0 in 0.145s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:40:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c740d131-98a9-4b14-a75a-a33aa7c72b51/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:40:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c740d131-98a9-4b14-a75a-a33aa7c72b51/disk --force-share --output=json" returned: 0 in 0.179s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:40:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:40:14 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdh (vda) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:40:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6de75cee-9bf1-4f9f-8fa3-977d9432b805/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:40:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6de75cee-9bf1-4f9f-8fa3-977d9432b805/disk --force-share --output=json" returned: 0 in 0.146s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:40:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6de75cee-9bf1-4f9f-8fa3-977d9432b805/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:40:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6de75cee-9bf1-4f9f-8fa3-977d9432b805/disk --force-share --output=json" returned: 0 in 0.156s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:40:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:40:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json" returned: 0 in 0.142s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:40:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:40:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json" returned: 0 in 0.139s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:40:15 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:40:15 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:40:15 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Hypervisor/Node resource view: name=user free_ram=8124MB free_disk=25.084888458251953GB free_vcpus=6 pci_devices=[{"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72351) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Aug 09 02:40:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:40:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:40:16 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-615f0a79-e98d-4976-b318-14c05675187f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Acquiring lock "6de75cee-9bf1-4f9f-8fa3-977d9432b805" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:40:16 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-615f0a79-e98d-4976-b318-14c05675187f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "6de75cee-9bf1-4f9f-8fa3-977d9432b805" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:40:16 user nova-compute[72351]: DEBUG nova.objects.instance [None req-615f0a79-e98d-4976-b318-14c05675187f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lazy-loading 'flavor' on Instance uuid 6de75cee-9bf1-4f9f-8fa3-977d9432b805 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:40:16 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance a7f3d076-864b-4b68-9207-214ce2c21444 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:40:16 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 957c1510-1320-4492-9052-a58ff540319d actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:40:16 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 41b5e305-3973-4937-b262-d32db1091e20 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:40:16 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance bde1ef1e-163c-40eb-abb5-962b9064b7ab actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:40:16 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 6de75cee-9bf1-4f9f-8fa3-977d9432b805 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:40:16 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance c740d131-98a9-4b14-a75a-a33aa7c72b51 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:40:16 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Total usable vcpus: 12, total allocated vcpus: 6 {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Aug 09 02:40:16 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Final resource view: name=user phys_ram=16011MB used_ram=3584MB phys_disk=40GB used_disk=4GB total_vcpus=12 used_vcpus=6 pci_stats=[] {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Aug 09 02:40:16 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-615f0a79-e98d-4976-b318-14c05675187f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "6de75cee-9bf1-4f9f-8fa3-977d9432b805" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.054s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:40:16 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:40:16 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-615f0a79-e98d-4976-b318-14c05675187f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Acquiring lock "6de75cee-9bf1-4f9f-8fa3-977d9432b805" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:40:16 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-615f0a79-e98d-4976-b318-14c05675187f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "6de75cee-9bf1-4f9f-8fa3-977d9432b805" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:40:16 user nova-compute[72351]: INFO nova.compute.manager [None req-615f0a79-e98d-4976-b318-14c05675187f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Attaching volume af0b86fc-20d0-49f5-bf92-ee3c8c5783e4 to /dev/vdb
Aug 09 02:40:16 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:40:16 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Compute_service record updated for user:user {{(pid=72351) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Aug 09 02:40:16 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.705s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:40:16 user nova-compute[72351]: DEBUG os_brick.utils [None req-615f0a79-e98d-4976-b318-14c05675187f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:40:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:40:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:40:16 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[066cd425-f33f-4fb8-ba20-d8564d8bc7f4]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:a52269602ff4\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:40:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:40:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:40:16 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[35dac00d-7ae7-4796-af4c-f2105c7b32eb]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:40:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:40:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:40:16 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[60122f7d-cdb4-4d59-8485-c4f641776fa5]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:40:16 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[939f8d33-d8bf-4440-ab25-48fe618f8e99]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:40:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-615f0a79-e98d-4976-b318-14c05675187f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Running cmd (subprocess): nvme version {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:40:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-615f0a79-e98d-4976-b318-14c05675187f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] 'nvme version' failed. Not Retrying. {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:40:16 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.nvmeof [None req-615f0a79-e98d-4976-b318-14c05675187f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] nvme not present on system {{(pid=72351) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Aug 09 02:40:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:40:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:40:16 user nova-compute[72351]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Aug 09 02:40:16 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a3506829-85b3-4d90-a7ff-638a26a6d6b0]: (4, '') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:40:16 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-615f0a79-e98d-4976-b318-14c05675187f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72351) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Aug 09 02:40:16 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-615f0a79-e98d-4976-b318-14c05675187f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Aug 09 02:40:16 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-615f0a79-e98d-4976-b318-14c05675187f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] LIGHTOS: no hostnqn found. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Aug 09 02:40:16 user nova-compute[72351]: DEBUG os_brick.utils [None req-615f0a79-e98d-4976-b318-14c05675187f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] <== get_connector_properties: return (129ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:a52269602ff4', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:40:16 user nova-compute[72351]: DEBUG nova.virt.block_device [None req-615f0a79-e98d-4976-b318-14c05675187f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Updating existing volume attachment record: f37baaf0-0e01-4cf0-b08f-ac4fbc34aec6 {{(pid=72351) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Aug 09 02:40:18 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:40:18 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:40:18 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:40:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:40:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:40:23 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:40:23 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:40:24 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-615f0a79-e98d-4976-b318-14c05675187f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 02:40:24 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-615f0a79-e98d-4976-b318-14c05675187f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:af0b86fc-20d0-49f5-bf92-ee3c8c5783e4'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'WnmTZzkQ', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:40:24 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-615f0a79-e98d-4976-b318-14c05675187f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:40:24 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-615f0a79-e98d-4976-b318-14c05675187f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:40:24 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-615f0a79-e98d-4976-b318-14c05675187f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:40:24 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-615f0a79-e98d-4976-b318-14c05675187f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:40:24 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-615f0a79-e98d-4976-b318-14c05675187f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:af0b86fc-20d0-49f5-bf92-ee3c8c5783e4" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:40:24 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-615f0a79-e98d-4976-b318-14c05675187f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:af0b86fc-20d0-49f5-bf92-ee3c8c5783e4" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:40:24 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-615f0a79-e98d-4976-b318-14c05675187f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 02:40:24 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:af0b86fc-20d0-49f5-bf92-ee3c8c5783e4 -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:40:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:af0b86fc-20d0-49f5-bf92-ee3c8c5783e4 -p 172.16.0.220:3260" returned: 21 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:40:25 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[7b5a1ab7-e842-4bec-a5fe-0b9497f2df51]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:40:25 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-615f0a79-e98d-4976-b318-14c05675187f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Aug 09 02:40:25 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:40:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:af0b86fc-20d0-49f5-bf92-ee3c8c5783e4 -p 172.16.0.220:3260 --interface default --op new {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:40:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:af0b86fc-20d0-49f5-bf92-ee3c8c5783e4 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:40:25 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[0f363867-4e3f-4231-a82c-648c4bd0f061]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:af0b86fc-20d0-49f5-bf92-ee3c8c5783e4] added\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:40:25 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-615f0a79-e98d-4976-b318-14c05675187f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:af0b86fc-20d0-49f5-bf92-ee3c8c5783e4] added
Aug 09 02:40:25 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:40:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:af0b86fc-20d0-49f5-bf92-ee3c8c5783e4 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:40:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:af0b86fc-20d0-49f5-bf92-ee3c8c5783e4 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.044s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:40:25 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[fd296542-604c-404e-bf89-ffd7aa7aa19c]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:40:25 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-615f0a79-e98d-4976-b318-14c05675187f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:40:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:af0b86fc-20d0-49f5-bf92-ee3c8c5783e4 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:40:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:af0b86fc-20d0-49f5-bf92-ee3c8c5783e4 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:40:25 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[1aa3b380-3f00-48a3-a31c-f5dce6984c48]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:40:25 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-615f0a79-e98d-4976-b318-14c05675187f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:40:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:af0b86fc-20d0-49f5-bf92-ee3c8c5783e4 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v WnmTZzkQ {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:40:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:af0b86fc-20d0-49f5-bf92-ee3c8c5783e4 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v WnmTZzkQ" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:40:25 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[78314579-e669-4dbe-af9c-801004350179]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:40:25 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-615f0a79-e98d-4976-b318-14c05675187f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'WnmTZzkQ'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:40:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:af0b86fc-20d0-49f5-bf92-ee3c8c5783e4 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:40:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:af0b86fc-20d0-49f5-bf92-ee3c8c5783e4 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:40:25 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[fc079155-e3c9-47d1-9eb9-3c4558d8c369]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:40:25 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-615f0a79-e98d-4976-b318-14c05675187f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:40:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:40:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.028s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:40:25 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a95f3f28-646a-4210-8b99-22edde43a93a]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:40:25 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-615f0a79-e98d-4976-b318-14c05675187f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:40:25 user nova-compute[72351]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df (non-flash)
Aug 09 02:40:25 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:40:25 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-615f0a79-e98d-4976-b318-14c05675187f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:40:25 user nova-compute[72351]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df (non-flash)
Aug 09 02:40:25 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:40:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:af0b86fc-20d0-49f5-bf92-ee3c8c5783e4 -p 172.16.0.220:3260 --login {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:40:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:af0b86fc-20d0-49f5-bf92-ee3c8c5783e4 -p 172.16.0.220:3260 --login" returned: 0 in 0.057s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:40:25 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[1ebb2c7e-cc06-4e82-b65b-6e4f2e9c95f8]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:af0b86fc-20d0-49f5-bf92-ee3c8c5783e4, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:af0b86fc-20d0-49f5-bf92-ee3c8c5783e4, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:40:25 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-615f0a79-e98d-4976-b318-14c05675187f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:af0b86fc-20d0-49f5-bf92-ee3c8c5783e4, portal: 172.16.0.220,3260]
Aug 09 02:40:25 user nova-compute[72351]: Login to [iface: default, target: iqn.2010-10.org.openstack:af0b86fc-20d0-49f5-bf92-ee3c8c5783e4, portal: 172.16.0.220,3260] successful.
Aug 09 02:40:25 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:40:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:af0b86fc-20d0-49f5-bf92-ee3c8c5783e4 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:40:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:af0b86fc-20d0-49f5-bf92-ee3c8c5783e4 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:40:25 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d1b847d8-1519-4492-b20f-93dd797c19c9]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:40:25 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-615f0a79-e98d-4976-b318-14c05675187f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:40:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:40:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:40:25 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[42315adf-b2db-498a-bedd-28648a410f21]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df (non-flash)\ntcp: [56] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:af0b86fc-20d0-49f5-bf92-ee3c8c5783e4 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:40:25 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-615f0a79-e98d-4976-b318-14c05675187f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:40:25 user nova-compute[72351]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df (non-flash)
Aug 09 02:40:25 user nova-compute[72351]: tcp: [56] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:af0b86fc-20d0-49f5-bf92-ee3c8c5783e4 (non-flash)
Aug 09 02:40:25 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:40:25 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-615f0a79-e98d-4976-b318-14c05675187f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:40:25 user nova-compute[72351]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df (non-flash)
Aug 09 02:40:25 user nova-compute[72351]: tcp: [56] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:af0b86fc-20d0-49f5-bf92-ee3c8c5783e4 (non-flash)
Aug 09 02:40:25 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:40:25 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-615f0a79-e98d-4976-b318-14c05675187f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:af0b86fc-20d0-49f5-bf92-ee3c8c5783e4" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.310s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:40:25 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-615f0a79-e98d-4976-b318-14c05675187f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 02:40:25 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-615f0a79-e98d-4976-b318-14c05675187f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] HCTL ('33', '-', '-', 0) found on session 56 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 02:40:25 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-615f0a79-e98d-4976-b318-14c05675187f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 02:40:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:40:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:40:25 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[8d6a24ab-3419-499a-afbe-1c5e1775c40a]: (4, ('- - 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:40:25 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-615f0a79-e98d-4976-b318-14c05675187f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Searching for a device in session 56 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:40:25 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-615f0a79-e98d-4976-b318-14c05675187f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Connected to sdb using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'WnmTZzkQ', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 02:40:26 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-615f0a79-e98d-4976-b318-14c05675187f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.343s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:40:26 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-615f0a79-e98d-4976-b318-14c05675187f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] <== connect_volume: return (1344ms) {'type': 'block', 'scsi_wwn': '26234306164613964', 'path': '/dev/sdb'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:40:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-615f0a79-e98d-4976-b318-14c05675187f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26234306164613964', 'path': '/dev/sdb'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 02:40:26 user nova-compute[72351]: DEBUG nova.objects.instance [None req-615f0a79-e98d-4976-b318-14c05675187f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lazy-loading 'flavor' on Instance uuid 6de75cee-9bf1-4f9f-8fa3-977d9432b805 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:40:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-615f0a79-e98d-4976-b318-14c05675187f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] attach device xml: <disk type="block" device="disk">
Aug 09 02:40:26 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:40:26 user nova-compute[72351]:   <source dev="/dev/sdb"/>
Aug 09 02:40:26 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:40:26 user nova-compute[72351]:   <serial>af0b86fc-20d0-49f5-bf92-ee3c8c5783e4</serial>
Aug 09 02:40:26 user nova-compute[72351]: </disk>
Aug 09 02:40:26 user nova-compute[72351]:  {{(pid=72351) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Aug 09 02:40:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-615f0a79-e98d-4976-b318-14c05675187f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:40:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-615f0a79-e98d-4976-b318-14c05675187f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:40:26 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-615f0a79-e98d-4976-b318-14c05675187f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] No VIF found with MAC fa:16:3e:1a:c1:d6, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:40:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-615f0a79-e98d-4976-b318-14c05675187f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "6de75cee-9bf1-4f9f-8fa3-977d9432b805" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 10.490s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:40:28 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:40:28 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:40:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Acquiring lock "ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:40:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:40:29 user nova-compute[72351]: DEBUG nova.compute.manager [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Starting instance... {{(pid=72351) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2406}}
Aug 09 02:40:30 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a06e630c-6db0-4af5-9e2a-49ad648fda97 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Acquiring lock "bde1ef1e-163c-40eb-abb5-962b9064b7ab" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:40:30 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a06e630c-6db0-4af5-9e2a-49ad648fda97 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "bde1ef1e-163c-40eb-abb5-962b9064b7ab" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:40:30 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a06e630c-6db0-4af5-9e2a-49ad648fda97 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Acquiring lock "bde1ef1e-163c-40eb-abb5-962b9064b7ab-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:40:30 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a06e630c-6db0-4af5-9e2a-49ad648fda97 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "bde1ef1e-163c-40eb-abb5-962b9064b7ab-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:40:30 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a06e630c-6db0-4af5-9e2a-49ad648fda97 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "bde1ef1e-163c-40eb-abb5-962b9064b7ab-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:40:30 user nova-compute[72351]: INFO nova.compute.manager [None req-a06e630c-6db0-4af5-9e2a-49ad648fda97 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Terminating instance
Aug 09 02:40:30 user nova-compute[72351]: DEBUG nova.compute.manager [None req-a06e630c-6db0-4af5-9e2a-49ad648fda97 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Start destroying the instance on the hypervisor. {{(pid=72351) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3124}}
Aug 09 02:40:30 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:40:30 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:40:30 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72351) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Aug 09 02:40:30 user nova-compute[72351]: INFO nova.compute.claims [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Claim successful on node user
Aug 09 02:40:30 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:40:30 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:40:30 user nova-compute[72351]: DEBUG nova.compute.manager [req-41fa67bf-697f-41e8-abaa-cc5b6dcfe343 req-2d8d35fb-0326-47cf-8b28-d3afc0039747 service nova] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Received event network-vif-unplugged-8424518d-87b7-4fc4-8a58-1b4f094a5ca0 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:40:30 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-41fa67bf-697f-41e8-abaa-cc5b6dcfe343 req-2d8d35fb-0326-47cf-8b28-d3afc0039747 service nova] Acquiring lock "bde1ef1e-163c-40eb-abb5-962b9064b7ab-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:40:30 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-41fa67bf-697f-41e8-abaa-cc5b6dcfe343 req-2d8d35fb-0326-47cf-8b28-d3afc0039747 service nova] Lock "bde1ef1e-163c-40eb-abb5-962b9064b7ab-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:40:30 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-41fa67bf-697f-41e8-abaa-cc5b6dcfe343 req-2d8d35fb-0326-47cf-8b28-d3afc0039747 service nova] Lock "bde1ef1e-163c-40eb-abb5-962b9064b7ab-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:40:30 user nova-compute[72351]: DEBUG nova.compute.manager [req-41fa67bf-697f-41e8-abaa-cc5b6dcfe343 req-2d8d35fb-0326-47cf-8b28-d3afc0039747 service nova] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] No waiting events found dispatching network-vif-unplugged-8424518d-87b7-4fc4-8a58-1b4f094a5ca0 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:40:30 user nova-compute[72351]: DEBUG nova.compute.manager [req-41fa67bf-697f-41e8-abaa-cc5b6dcfe343 req-2d8d35fb-0326-47cf-8b28-d3afc0039747 service nova] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Received event network-vif-unplugged-8424518d-87b7-4fc4-8a58-1b4f094a5ca0 for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:40:30 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:40:30 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:40:30 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:40:30 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:40:30 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.394s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:40:30 user nova-compute[72351]: DEBUG nova.compute.manager [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Start building networks asynchronously for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2803}}
Aug 09 02:40:30 user nova-compute[72351]: DEBUG nova.compute.manager [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Allocating IP information in the background. {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1956}}
Aug 09 02:40:30 user nova-compute[72351]: DEBUG nova.network.neutron [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] allocate_for_instance() {{(pid=72351) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Aug 09 02:40:30 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Aug 09 02:40:30 user nova-compute[72351]: DEBUG nova.compute.manager [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Start building block device mappings for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2838}}
Aug 09 02:40:30 user nova-compute[72351]: DEBUG nova.policy [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '17f7007adf02474e9b019cd8af099ff8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c07a6b254414b41a673468238d3aa62', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72351) authorize /opt/stack/nova/nova/policy.py:203}}
Aug 09 02:40:30 user nova-compute[72351]: DEBUG nova.compute.manager [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Start spawning the instance on the hypervisor. {{(pid=72351) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2612}}
Aug 09 02:40:30 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Creating instance directory {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4717}}
Aug 09 02:40:30 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Creating image(s)
Aug 09 02:40:30 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Acquiring lock "/opt/stack/data/nova/instances/ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:40:30 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "/opt/stack/data/nova/instances/ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:40:30 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "/opt/stack/data/nova/instances/ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:40:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:40:30 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Instance destroyed successfully.
Aug 09 02:40:30 user nova-compute[72351]: DEBUG nova.objects.instance [None req-a06e630c-6db0-4af5-9e2a-49ad648fda97 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lazy-loading 'resources' on Instance uuid bde1ef1e-163c-40eb-abb5-962b9064b7ab {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:40:30 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-a06e630c-6db0-4af5-9e2a-49ad648fda97 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:34:56Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-1202876429',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-1202876429',id=43,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:36:57Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='321111f935dd4e8aab4a083206ea9a7e',ramdisk_id='',reservation_id='r-p0t08coj',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-2108329302',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:38:44Z,user_data=None,user_id='73d05c48be5640c28455e8ba8b05ef31',uuid=bde1ef1e-163c-40eb-abb5-962b9064b7ab,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "8424518d-87b7-4fc4-8a58-1b4f094a5ca0", "address": "fa:16:3e:08:61:83", "network": {"id": "9a452414-d61f-4045-a6cc-9be93c892c4c", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "321111f935dd4e8aab4a083206ea9a7e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8424518d-87", "ovs_interfaceid": "8424518d-87b7-4fc4-8a58-1b4f094a5ca0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:40:30 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-a06e630c-6db0-4af5-9e2a-49ad648fda97 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Converting VIF {"id": "8424518d-87b7-4fc4-8a58-1b4f094a5ca0", "address": "fa:16:3e:08:61:83", "network": {"id": "9a452414-d61f-4045-a6cc-9be93c892c4c", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "321111f935dd4e8aab4a083206ea9a7e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8424518d-87", "ovs_interfaceid": "8424518d-87b7-4fc4-8a58-1b4f094a5ca0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:40:30 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-a06e630c-6db0-4af5-9e2a-49ad648fda97 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:08:61:83,bridge_name='br-int',has_traffic_filtering=True,id=8424518d-87b7-4fc4-8a58-1b4f094a5ca0,network=Network(9a452414-d61f-4045-a6cc-9be93c892c4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8424518d-87') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:40:30 user nova-compute[72351]: DEBUG os_vif [None req-a06e630c-6db0-4af5-9e2a-49ad648fda97 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:08:61:83,bridge_name='br-int',has_traffic_filtering=True,id=8424518d-87b7-4fc4-8a58-1b4f094a5ca0,network=Network(9a452414-d61f-4045-a6cc-9be93c892c4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8424518d-87') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:40:30 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:40:30 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap8424518d-87, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:40:30 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:40:30 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:40:30 user nova-compute[72351]: INFO os_vif [None req-a06e630c-6db0-4af5-9e2a-49ad648fda97 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:08:61:83,bridge_name='br-int',has_traffic_filtering=True,id=8424518d-87b7-4fc4-8a58-1b4f094a5ca0,network=Network(9a452414-d61f-4045-a6cc-9be93c892c4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8424518d-87')
Aug 09 02:40:30 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-a06e630c-6db0-4af5-9e2a-49ad648fda97 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 02:40:30 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-a06e630c-6db0-4af5-9e2a-49ad648fda97 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'LNfcKqMo', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdh'}, None), 'kwargs': True}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:40:30 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-a06e630c-6db0-4af5-9e2a-49ad648fda97 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:40:30 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-a06e630c-6db0-4af5-9e2a-49ad648fda97 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:40:30 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-a06e630c-6db0-4af5-9e2a-49ad648fda97 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:40:30 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-a06e630c-6db0-4af5-9e2a-49ad648fda97 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 02:40:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:40:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.155s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:40:30 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Acquiring lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:40:30 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:40:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:40:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:40:30 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[243a0d1e-9b30-4cb3-a0a5-7b7379cc7513]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:af0b86fc-20d0-49f5-bf92-ee3c8c5783e4\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:40:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:40:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:40:30 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[9b9e07c1-9344-499e-8b89-f6be6eaf40d1]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df (non-flash)\ntcp: [56] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:af0b86fc-20d0-49f5-bf92-ee3c8c5783e4 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:40:30 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-a06e630c-6db0-4af5-9e2a-49ad648fda97 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:40:30 user nova-compute[72351]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df (non-flash)
Aug 09 02:40:30 user nova-compute[72351]: tcp: [56] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:af0b86fc-20d0-49f5-bf92-ee3c8c5783e4 (non-flash)
Aug 09 02:40:30 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:40:30 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-a06e630c-6db0-4af5-9e2a-49ad648fda97 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:40:30 user nova-compute[72351]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df (non-flash)
Aug 09 02:40:30 user nova-compute[72351]: tcp: [56] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:af0b86fc-20d0-49f5-bf92-ee3c8c5783e4 (non-flash)
Aug 09 02:40:30 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:40:30 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-a06e630c-6db0-4af5-9e2a-49ad648fda97 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fd00921bd00>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df'): ({'sdh'}, set())}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 02:40:30 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-a06e630c-6db0-4af5-9e2a-49ad648fda97 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Removing single pathed devices sdh {{(pid=72351) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Aug 09 02:40:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:40:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:40:31 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[38472e0c-42fd-45e3-afc7-0ea63274c2a2]: (4, ('path checker states:\nup                  3\n\npaths: 0\nbusy: False\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:40:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdh {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:40:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdh" returned: 0 in 0.025s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:40:31 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[10620f74-bed0-47c4-8cbd-7f9ea9384dd3]: (4, ('ok\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:40:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.154s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:40:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f/disk 1073741824 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:40:31 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-a06e630c-6db0-4af5-9e2a-49ad648fda97 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Flushing IO for device /dev/sdh {{(pid=72351) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Aug 09 02:40:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdh {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:40:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f/disk 1073741824" returned: 0 in 0.084s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:40:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.241s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:40:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:40:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdh" returned: 0 in 0.029s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:40:31 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[654b5b1b-1566-40d7-b877-808fb70ab46b]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:40:31 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-a06e630c-6db0-4af5-9e2a-49ad648fda97 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Remove SCSI device /dev/sdh with /sys/block/sdh/device/delete {{(pid=72351) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Aug 09 02:40:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdh/device/delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:40:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdh/device/delete" returned: 0 in 0.032s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:40:31 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[aa05f117-8f04-468a-985c-d45fe407d323]: (4, ('1', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:40:31 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-a06e630c-6db0-4af5-9e2a-49ad648fda97 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Checking to see if SCSI volumes sdh have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Aug 09 02:40:31 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-a06e630c-6db0-4af5-9e2a-49ad648fda97 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] SCSI volumes sdh have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Aug 09 02:40:31 user nova-compute[72351]: DEBUG os_brick.privileged.rootwrap [-] Unlinking ('/dev/disk/by-id/scsi-SSCST_BIO_Storage_c8ed23b1', '/dev/disk/by-id/scsi-1SCST_BIOc8ed23b1-3366a9dce4991b45', '/dev/disk/by-id/scsi-23333363661396463') {{(pid=83366) unlink_root /usr/local/lib/python3.10/dist-packages/os_brick/privileged/rootwrap.py:219}}
Aug 09 02:40:31 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ebac5c8c-16a6-4447-a9ad-275b64fa608a]: (4, None) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:40:31 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-a06e630c-6db0-4af5-9e2a-49ad648fda97 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df')] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 02:40:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:40:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.036s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:40:31 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[024c5935-e62a-4f1d-83c3-73d5d3bd626d]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:40:31 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-a06e630c-6db0-4af5-9e2a-49ad648fda97 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:40:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df -p 172.16.0.220:3260 --logout {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:40:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.144s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:40:31 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Checking if we can resize image /opt/stack/data/nova/instances/ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f/disk. size=1073741824 {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Aug 09 02:40:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:40:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df -p 172.16.0.220:3260 --logout" returned: 0 in 0.073s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:40:31 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[bb3c8443-c1c5-4ef1-93aa-dfd4a9b8ea1e]: (4, ('Logging out of session [sid: 48, target: iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df, portal: 172.16.0.220,3260]\nLogout of [sid: 48, target: iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:40:31 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-a06e630c-6db0-4af5-9e2a-49ad648fda97 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 48, target: iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df, portal: 172.16.0.220,3260]
Aug 09 02:40:31 user nova-compute[72351]: Logout of [sid: 48, target: iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df, portal: 172.16.0.220,3260] successful.
Aug 09 02:40:31 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:40:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df -p 172.16.0.220:3260 --op delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:40:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b5789e9a-6f06-4239-8523-0e8607f350df -p 172.16.0.220:3260 --op delete" returned: 0 in 0.027s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:40:31 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[22f88bb0-7129-48e4-be1f-fd5abba64e5f]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:40:31 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-a06e630c-6db0-4af5-9e2a-49ad648fda97 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:40:31 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-a06e630c-6db0-4af5-9e2a-49ad648fda97 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.509s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:40:31 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-a06e630c-6db0-4af5-9e2a-49ad648fda97 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] <== disconnect_volume: return (511ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:40:31 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-a06e630c-6db0-4af5-9e2a-49ad648fda97 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 02:40:31 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-a06e630c-6db0-4af5-9e2a-49ad648fda97 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Deleting instance files /opt/stack/data/nova/instances/bde1ef1e-163c-40eb-abb5-962b9064b7ab_del
Aug 09 02:40:31 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-a06e630c-6db0-4af5-9e2a-49ad648fda97 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Deletion of /opt/stack/data/nova/instances/bde1ef1e-163c-40eb-abb5-962b9064b7ab_del complete
Aug 09 02:40:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f/disk --force-share --output=json" returned: 0 in 0.146s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:40:31 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Cannot resize image /opt/stack/data/nova/instances/ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f/disk to a smaller size. {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Aug 09 02:40:31 user nova-compute[72351]: DEBUG nova.objects.instance [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lazy-loading 'migration_context' on Instance uuid ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:40:31 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Created local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4851}}
Aug 09 02:40:31 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Ensure instance console log exists: /opt/stack/data/nova/instances/ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f/console.log {{(pid=72351) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4603}}
Aug 09 02:40:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:40:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:40:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:40:31 user nova-compute[72351]: INFO nova.compute.manager [None req-a06e630c-6db0-4af5-9e2a-49ad648fda97 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Took 1.43 seconds to destroy the instance on the hypervisor.
Aug 09 02:40:31 user nova-compute[72351]: DEBUG oslo.service.loopingcall [None req-a06e630c-6db0-4af5-9e2a-49ad648fda97 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72351) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Aug 09 02:40:31 user nova-compute[72351]: DEBUG nova.compute.manager [-] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Deallocating network for instance {{(pid=72351) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2263}}
Aug 09 02:40:31 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] deallocate_for_instance() {{(pid=72351) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1795}}
Aug 09 02:40:31 user nova-compute[72351]: DEBUG nova.network.neutron [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Successfully created port: 6295a96f-893b-4ae1-9452-18ef59c6e544 {{(pid=72351) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Aug 09 02:40:32 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:40:32 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Took 0.63 seconds to deallocate network for instance.
Aug 09 02:40:32 user nova-compute[72351]: DEBUG nova.compute.manager [req-bbd24596-e9d5-49f3-938d-9381149622c5 req-a0fd5ff3-807e-4d8b-a5f7-ab682f67979d service nova] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Received event network-vif-deleted-8424518d-87b7-4fc4-8a58-1b4f094a5ca0 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:40:32 user nova-compute[72351]: DEBUG nova.compute.manager [req-2fdbaf12-184f-4e93-b26d-ca0d816026d7 req-0ff73be6-d364-4877-94cc-c81dcb8c1d66 service nova] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Received event network-vif-plugged-8424518d-87b7-4fc4-8a58-1b4f094a5ca0 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:40:32 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-2fdbaf12-184f-4e93-b26d-ca0d816026d7 req-0ff73be6-d364-4877-94cc-c81dcb8c1d66 service nova] Acquiring lock "bde1ef1e-163c-40eb-abb5-962b9064b7ab-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:40:32 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-2fdbaf12-184f-4e93-b26d-ca0d816026d7 req-0ff73be6-d364-4877-94cc-c81dcb8c1d66 service nova] Lock "bde1ef1e-163c-40eb-abb5-962b9064b7ab-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:40:32 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-2fdbaf12-184f-4e93-b26d-ca0d816026d7 req-0ff73be6-d364-4877-94cc-c81dcb8c1d66 service nova] Lock "bde1ef1e-163c-40eb-abb5-962b9064b7ab-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:40:32 user nova-compute[72351]: DEBUG nova.compute.manager [req-2fdbaf12-184f-4e93-b26d-ca0d816026d7 req-0ff73be6-d364-4877-94cc-c81dcb8c1d66 service nova] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] No waiting events found dispatching network-vif-plugged-8424518d-87b7-4fc4-8a58-1b4f094a5ca0 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:40:32 user nova-compute[72351]: WARNING nova.compute.manager [req-2fdbaf12-184f-4e93-b26d-ca0d816026d7 req-0ff73be6-d364-4877-94cc-c81dcb8c1d66 service nova] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Received unexpected event network-vif-plugged-8424518d-87b7-4fc4-8a58-1b4f094a5ca0 for instance with vm_state active and task_state deleting.
Aug 09 02:40:32 user nova-compute[72351]: DEBUG nova.network.neutron [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Successfully updated port: 6295a96f-893b-4ae1-9452-18ef59c6e544 {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:40:32 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Acquiring lock "refresh_cache-ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:40:32 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Acquired lock "refresh_cache-ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:40:32 user nova-compute[72351]: DEBUG nova.network.neutron [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:40:32 user nova-compute[72351]: DEBUG nova.compute.manager [req-69228e66-79c1-4b1b-a083-d5b53cc42cd3 req-10e4f41b-de42-43d2-aaea-6d70f8434230 service nova] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Received event network-changed-6295a96f-893b-4ae1-9452-18ef59c6e544 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:40:32 user nova-compute[72351]: DEBUG nova.compute.manager [req-69228e66-79c1-4b1b-a083-d5b53cc42cd3 req-10e4f41b-de42-43d2-aaea-6d70f8434230 service nova] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Refreshing instance network info cache due to event network-changed-6295a96f-893b-4ae1-9452-18ef59c6e544. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:40:32 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-69228e66-79c1-4b1b-a083-d5b53cc42cd3 req-10e4f41b-de42-43d2-aaea-6d70f8434230 service nova] Acquiring lock "refresh_cache-ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:40:32 user nova-compute[72351]: DEBUG nova.network.neutron [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:40:32 user nova-compute[72351]: DEBUG nova.network.neutron [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Updating instance_info_cache with network_info: [{"id": "6295a96f-893b-4ae1-9452-18ef59c6e544", "address": "fa:16:3e:f2:06:f7", "network": {"id": "47c43230-a244-4198-84e4-9ed98c005cd9", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1392403106-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c07a6b254414b41a673468238d3aa62", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6295a96f-89", "ovs_interfaceid": "6295a96f-893b-4ae1-9452-18ef59c6e544", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:40:32 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Releasing lock "refresh_cache-ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:40:32 user nova-compute[72351]: DEBUG nova.compute.manager [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Instance network_info: |[{"id": "6295a96f-893b-4ae1-9452-18ef59c6e544", "address": "fa:16:3e:f2:06:f7", "network": {"id": "47c43230-a244-4198-84e4-9ed98c005cd9", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1392403106-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c07a6b254414b41a673468238d3aa62", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6295a96f-89", "ovs_interfaceid": "6295a96f-893b-4ae1-9452-18ef59c6e544", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1971}}
Aug 09 02:40:32 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-69228e66-79c1-4b1b-a083-d5b53cc42cd3 req-10e4f41b-de42-43d2-aaea-6d70f8434230 service nova] Acquired lock "refresh_cache-ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:40:32 user nova-compute[72351]: DEBUG nova.network.neutron [req-69228e66-79c1-4b1b-a083-d5b53cc42cd3 req-10e4f41b-de42-43d2-aaea-6d70f8434230 service nova] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Refreshing network info cache for port 6295a96f-893b-4ae1-9452-18ef59c6e544 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:40:32 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Start _get_guest_xml network_info=[{"id": "6295a96f-893b-4ae1-9452-18ef59c6e544", "address": "fa:16:3e:f2:06:f7", "network": {"id": "47c43230-a244-4198-84e4-9ed98c005cd9", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1392403106-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c07a6b254414b41a673468238d3aa62", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6295a96f-89", "ovs_interfaceid": "6295a96f-893b-4ae1-9452-18ef59c6e544", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'size': 0, 'device_type': 'disk', 'encryption_secret_uuid': None, 'guest_format': None, 'encryption_options': None, 'disk_bus': 'virtio', 'encrypted': False, 'encryption_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'image_id': '18ccda7f-31d3-4978-ab9d-4ed2f74eff94'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:40:32 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:40:32 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:40:32 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:40:32 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:40:32 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:40:32 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:40:32 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:40:32 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:40:32 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:40:32 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:40:32 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:40:32 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:40:32 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:40:32 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:40:32 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:40:32 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:40:32 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:40:32 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:40:32 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:40:29Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachVolumeNegativeTest-server-2139317293',display_name='tempest-AttachVolumeNegativeTest-server-2139317293',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumenegativetest-server-2139317293',id=46,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBuIWu8zq6eGYrevOSXDRX6Ih5Nw4qjL0TAWTPVT1ZTrLHHnTDwcq06u6Q0X+/PGvqpZxqOnKLnPwr180t9o7ApGu+r9prljsGX7oJiP+ZAJgHUACauCzbsO9HoK5YKlvw==',key_name='tempest-keypair-1826810697',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='8c07a6b254414b41a673468238d3aa62',ramdisk_id='',reservation_id='r-v0g22x6o',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeNegativeTest-758771161',owner_user_name='tempest-AttachVolumeNegativeTest-758771161-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:40:31Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='17f7007adf02474e9b019cd8af099ff8',uuid=ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "6295a96f-893b-4ae1-9452-18ef59c6e544", "address": "fa:16:3e:f2:06:f7", "network": {"id": "47c43230-a244-4198-84e4-9ed98c005cd9", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1392403106-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c07a6b254414b41a673468238d3aa62", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6295a96f-89", "ovs_interfaceid": "6295a96f-893b-4ae1-9452-18ef59c6e544", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:40:32 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Converting VIF {"id": "6295a96f-893b-4ae1-9452-18ef59c6e544", "address": "fa:16:3e:f2:06:f7", "network": {"id": "47c43230-a244-4198-84e4-9ed98c005cd9", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1392403106-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c07a6b254414b41a673468238d3aa62", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6295a96f-89", "ovs_interfaceid": "6295a96f-893b-4ae1-9452-18ef59c6e544", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:40:32 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:f2:06:f7,bridge_name='br-int',has_traffic_filtering=True,id=6295a96f-893b-4ae1-9452-18ef59c6e544,network=Network(47c43230-a244-4198-84e4-9ed98c005cd9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6295a96f-89') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:40:32 user nova-compute[72351]: DEBUG nova.objects.instance [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lazy-loading 'pci_devices' on Instance uuid ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:40:33 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:40:33 user nova-compute[72351]:   <uuid>ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f</uuid>
Aug 09 02:40:33 user nova-compute[72351]:   <name>instance-0000002e</name>
Aug 09 02:40:33 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:40:33 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:40:33 user nova-compute[72351]:   <metadata>
Aug 09 02:40:33 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:40:33 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:40:33 user nova-compute[72351]:       <nova:name>tempest-AttachVolumeNegativeTest-server-2139317293</nova:name>
Aug 09 02:40:33 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:40:32</nova:creationTime>
Aug 09 02:40:33 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:40:33 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:40:33 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:40:33 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:40:33 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:40:33 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:40:33 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:40:33 user nova-compute[72351]:       <nova:owner>
Aug 09 02:40:33 user nova-compute[72351]:         <nova:user uuid="17f7007adf02474e9b019cd8af099ff8">tempest-AttachVolumeNegativeTest-758771161-project-member</nova:user>
Aug 09 02:40:33 user nova-compute[72351]:         <nova:project uuid="8c07a6b254414b41a673468238d3aa62">tempest-AttachVolumeNegativeTest-758771161</nova:project>
Aug 09 02:40:33 user nova-compute[72351]:       </nova:owner>
Aug 09 02:40:33 user nova-compute[72351]:       <nova:root type="image" uuid="18ccda7f-31d3-4978-ab9d-4ed2f74eff94"/>
Aug 09 02:40:33 user nova-compute[72351]:       <nova:ports>
Aug 09 02:40:33 user nova-compute[72351]:         <nova:port uuid="6295a96f-893b-4ae1-9452-18ef59c6e544">
Aug 09 02:40:33 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.4" ipVersion="4"/>
Aug 09 02:40:33 user nova-compute[72351]:         </nova:port>
Aug 09 02:40:33 user nova-compute[72351]:       </nova:ports>
Aug 09 02:40:33 user nova-compute[72351]:     </nova:instance>
Aug 09 02:40:33 user nova-compute[72351]:   </metadata>
Aug 09 02:40:33 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:40:33 user nova-compute[72351]:     <system>
Aug 09 02:40:33 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:40:33 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:40:33 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:40:33 user nova-compute[72351]:       <entry name="serial">ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f</entry>
Aug 09 02:40:33 user nova-compute[72351]:       <entry name="uuid">ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f</entry>
Aug 09 02:40:33 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:40:33 user nova-compute[72351]:     </system>
Aug 09 02:40:33 user nova-compute[72351]:   </sysinfo>
Aug 09 02:40:33 user nova-compute[72351]:   <os>
Aug 09 02:40:33 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:40:33 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:40:33 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:40:33 user nova-compute[72351]:   </os>
Aug 09 02:40:33 user nova-compute[72351]:   <features>
Aug 09 02:40:33 user nova-compute[72351]:     <acpi/>
Aug 09 02:40:33 user nova-compute[72351]:     <apic/>
Aug 09 02:40:33 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:40:33 user nova-compute[72351]:   </features>
Aug 09 02:40:33 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:40:33 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:40:33 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:40:33 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:40:33 user nova-compute[72351]:   </clock>
Aug 09 02:40:33 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:40:33 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:40:33 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:40:33 user nova-compute[72351]:   </cpu>
Aug 09 02:40:33 user nova-compute[72351]:   <devices>
Aug 09 02:40:33 user nova-compute[72351]:     <disk type="file" device="disk">
Aug 09 02:40:33 user nova-compute[72351]:       <driver name="qemu" type="qcow2" cache="none"/>
Aug 09 02:40:33 user nova-compute[72351]:       <source file="/opt/stack/data/nova/instances/ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f/disk"/>
Aug 09 02:40:33 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:40:33 user nova-compute[72351]:     </disk>
Aug 09 02:40:33 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:40:33 user nova-compute[72351]:       <mac address="fa:16:3e:f2:06:f7"/>
Aug 09 02:40:33 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:40:33 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:40:33 user nova-compute[72351]:       <target dev="tap6295a96f-89"/>
Aug 09 02:40:33 user nova-compute[72351]:     </interface>
Aug 09 02:40:33 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:40:33 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f/console.log" append="off"/>
Aug 09 02:40:33 user nova-compute[72351]:     </serial>
Aug 09 02:40:33 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:40:33 user nova-compute[72351]:     <video>
Aug 09 02:40:33 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:40:33 user nova-compute[72351]:     </video>
Aug 09 02:40:33 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:40:33 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:40:33 user nova-compute[72351]:     </rng>
Aug 09 02:40:33 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:40:33 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:40:33 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:40:33 user nova-compute[72351]:     </memballoon>
Aug 09 02:40:33 user nova-compute[72351]:   </devices>
Aug 09 02:40:33 user nova-compute[72351]: </domain>
Aug 09 02:40:33 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:40:33 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:40:29Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachVolumeNegativeTest-server-2139317293',display_name='tempest-AttachVolumeNegativeTest-server-2139317293',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumenegativetest-server-2139317293',id=46,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBuIWu8zq6eGYrevOSXDRX6Ih5Nw4qjL0TAWTPVT1ZTrLHHnTDwcq06u6Q0X+/PGvqpZxqOnKLnPwr180t9o7ApGu+r9prljsGX7oJiP+ZAJgHUACauCzbsO9HoK5YKlvw==',key_name='tempest-keypair-1826810697',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='8c07a6b254414b41a673468238d3aa62',ramdisk_id='',reservation_id='r-v0g22x6o',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeNegativeTest-758771161',owner_user_name='tempest-AttachVolumeNegativeTest-758771161-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:40:31Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='17f7007adf02474e9b019cd8af099ff8',uuid=ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "6295a96f-893b-4ae1-9452-18ef59c6e544", "address": "fa:16:3e:f2:06:f7", "network": {"id": "47c43230-a244-4198-84e4-9ed98c005cd9", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1392403106-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c07a6b254414b41a673468238d3aa62", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6295a96f-89", "ovs_interfaceid": "6295a96f-893b-4ae1-9452-18ef59c6e544", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:40:33 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Converting VIF {"id": "6295a96f-893b-4ae1-9452-18ef59c6e544", "address": "fa:16:3e:f2:06:f7", "network": {"id": "47c43230-a244-4198-84e4-9ed98c005cd9", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1392403106-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c07a6b254414b41a673468238d3aa62", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6295a96f-89", "ovs_interfaceid": "6295a96f-893b-4ae1-9452-18ef59c6e544", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:40:33 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:f2:06:f7,bridge_name='br-int',has_traffic_filtering=True,id=6295a96f-893b-4ae1-9452-18ef59c6e544,network=Network(47c43230-a244-4198-84e4-9ed98c005cd9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6295a96f-89') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:40:33 user nova-compute[72351]: DEBUG os_vif [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:f2:06:f7,bridge_name='br-int',has_traffic_filtering=True,id=6295a96f-893b-4ae1-9452-18ef59c6e544,network=Network(47c43230-a244-4198-84e4-9ed98c005cd9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6295a96f-89') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:40:33 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:40:33 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:40:33 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:40:33 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:40:33 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap6295a96f-89, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:40:33 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap6295a96f-89, col_values=(('external_ids', {'iface-id': '6295a96f-893b-4ae1-9452-18ef59c6e544', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:f2:06:f7', 'vm-uuid': 'ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:40:33 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:40:33 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:40:33 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:40:33 user nova-compute[72351]: INFO os_vif [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:f2:06:f7,bridge_name='br-int',has_traffic_filtering=True,id=6295a96f-893b-4ae1-9452-18ef59c6e544,network=Network(47c43230-a244-4198-84e4-9ed98c005cd9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6295a96f-89')
Aug 09 02:40:33 user nova-compute[72351]: INFO nova.compute.manager [None req-a06e630c-6db0-4af5-9e2a-49ad648fda97 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Took 0.95 seconds to detach 1 volumes for instance.
Aug 09 02:40:33 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:40:33 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] No VIF found with MAC fa:16:3e:f2:06:f7, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:40:33 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a06e630c-6db0-4af5-9e2a-49ad648fda97 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:40:33 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a06e630c-6db0-4af5-9e2a-49ad648fda97 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:40:33 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-a06e630c-6db0-4af5-9e2a-49ad648fda97 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:40:33 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-a06e630c-6db0-4af5-9e2a-49ad648fda97 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:40:33 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a06e630c-6db0-4af5-9e2a-49ad648fda97 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.262s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:40:33 user nova-compute[72351]: INFO nova.scheduler.client.report [None req-a06e630c-6db0-4af5-9e2a-49ad648fda97 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Deleted allocations for instance bde1ef1e-163c-40eb-abb5-962b9064b7ab
Aug 09 02:40:33 user nova-compute[72351]: DEBUG nova.network.neutron [req-69228e66-79c1-4b1b-a083-d5b53cc42cd3 req-10e4f41b-de42-43d2-aaea-6d70f8434230 service nova] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Updated VIF entry in instance network info cache for port 6295a96f-893b-4ae1-9452-18ef59c6e544. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:40:33 user nova-compute[72351]: DEBUG nova.network.neutron [req-69228e66-79c1-4b1b-a083-d5b53cc42cd3 req-10e4f41b-de42-43d2-aaea-6d70f8434230 service nova] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Updating instance_info_cache with network_info: [{"id": "6295a96f-893b-4ae1-9452-18ef59c6e544", "address": "fa:16:3e:f2:06:f7", "network": {"id": "47c43230-a244-4198-84e4-9ed98c005cd9", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1392403106-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c07a6b254414b41a673468238d3aa62", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6295a96f-89", "ovs_interfaceid": "6295a96f-893b-4ae1-9452-18ef59c6e544", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:40:33 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-69228e66-79c1-4b1b-a083-d5b53cc42cd3 req-10e4f41b-de42-43d2-aaea-6d70f8434230 service nova] Releasing lock "refresh_cache-ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:40:33 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-a06e630c-6db0-4af5-9e2a-49ad648fda97 tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "bde1ef1e-163c-40eb-abb5-962b9064b7ab" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 3.460s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:40:34 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:40:34 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:40:34 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:40:34 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:40:34 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:40:34 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:40:34 user nova-compute[72351]: DEBUG nova.compute.manager [req-5a430168-538a-4367-a98c-0ab9f1775134 req-d04c481a-5866-4b07-9337-3a58bed932d7 service nova] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Received event network-vif-plugged-6295a96f-893b-4ae1-9452-18ef59c6e544 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:40:34 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-5a430168-538a-4367-a98c-0ab9f1775134 req-d04c481a-5866-4b07-9337-3a58bed932d7 service nova] Acquiring lock "ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:40:34 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-5a430168-538a-4367-a98c-0ab9f1775134 req-d04c481a-5866-4b07-9337-3a58bed932d7 service nova] Lock "ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:40:34 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-5a430168-538a-4367-a98c-0ab9f1775134 req-d04c481a-5866-4b07-9337-3a58bed932d7 service nova] Lock "ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:40:34 user nova-compute[72351]: DEBUG nova.compute.manager [req-5a430168-538a-4367-a98c-0ab9f1775134 req-d04c481a-5866-4b07-9337-3a58bed932d7 service nova] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] No waiting events found dispatching network-vif-plugged-6295a96f-893b-4ae1-9452-18ef59c6e544 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:40:34 user nova-compute[72351]: WARNING nova.compute.manager [req-5a430168-538a-4367-a98c-0ab9f1775134 req-d04c481a-5866-4b07-9337-3a58bed932d7 service nova] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Received unexpected event network-vif-plugged-6295a96f-893b-4ae1-9452-18ef59c6e544 for instance with vm_state building and task_state spawning.
Aug 09 02:40:36 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548836.58462, ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:40:36 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] VM Resumed (Lifecycle Event)
Aug 09 02:40:36 user nova-compute[72351]: DEBUG nova.compute.manager [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:40:36 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Guest created on hypervisor {{(pid=72351) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4411}}
Aug 09 02:40:36 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Instance spawned successfully.
Aug 09 02:40:36 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:894}}
Aug 09 02:40:36 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:40:36 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:40:36 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Found default for hw_cdrom_bus of ide {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:40:36 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Found default for hw_disk_bus of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:40:36 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Found default for hw_input_bus of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:40:36 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Found default for hw_pointer_model of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:40:36 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Found default for hw_video_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:40:36 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Found default for hw_vif_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:40:36 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:40:36 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691548836.5872128, ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:40:36 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] VM Started (Lifecycle Event)
Aug 09 02:40:36 user nova-compute[72351]: DEBUG nova.compute.manager [req-3090b0eb-cfe4-482e-9fb2-62e084a8c4af req-bad2050d-9103-44a6-92fc-8e5e0504e0c7 service nova] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Received event network-vif-plugged-6295a96f-893b-4ae1-9452-18ef59c6e544 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:40:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-3090b0eb-cfe4-482e-9fb2-62e084a8c4af req-bad2050d-9103-44a6-92fc-8e5e0504e0c7 service nova] Acquiring lock "ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:40:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-3090b0eb-cfe4-482e-9fb2-62e084a8c4af req-bad2050d-9103-44a6-92fc-8e5e0504e0c7 service nova] Lock "ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:40:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-3090b0eb-cfe4-482e-9fb2-62e084a8c4af req-bad2050d-9103-44a6-92fc-8e5e0504e0c7 service nova] Lock "ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:40:36 user nova-compute[72351]: DEBUG nova.compute.manager [req-3090b0eb-cfe4-482e-9fb2-62e084a8c4af req-bad2050d-9103-44a6-92fc-8e5e0504e0c7 service nova] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] No waiting events found dispatching network-vif-plugged-6295a96f-893b-4ae1-9452-18ef59c6e544 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:40:36 user nova-compute[72351]: WARNING nova.compute.manager [req-3090b0eb-cfe4-482e-9fb2-62e084a8c4af req-bad2050d-9103-44a6-92fc-8e5e0504e0c7 service nova] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Received unexpected event network-vif-plugged-6295a96f-893b-4ae1-9452-18ef59c6e544 for instance with vm_state building and task_state spawning.
Aug 09 02:40:36 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:40:36 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:40:36 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:40:36 user nova-compute[72351]: INFO nova.compute.manager [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Took 6.02 seconds to spawn the instance on the hypervisor.
Aug 09 02:40:36 user nova-compute[72351]: DEBUG nova.compute.manager [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:40:36 user nova-compute[72351]: INFO nova.compute.manager [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Took 6.77 seconds to build instance.
Aug 09 02:40:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-41bd038e-868c-44ff-9529-6f7f9a5ed4d8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 6.893s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:40:38 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:40:38 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:40:43 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:40:43 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:40:45 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691548830.8157973, bde1ef1e-163c-40eb-abb5-962b9064b7ab => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:40:45 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] VM Stopped (Lifecycle Event)
Aug 09 02:40:45 user nova-compute[72351]: DEBUG nova.compute.manager [None req-b2d3a86b-5518-41f3-9fd7-425b47f94390 None None] [instance: bde1ef1e-163c-40eb-abb5-962b9064b7ab] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:40:48 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:40:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:40:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:40:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 02:40:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:40:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:40:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:40:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:40:58 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:40:58 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:41:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:41:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:41:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:41:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:41:10 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:41:12 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:41:12 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72351) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10461}}
Aug 09 02:41:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:41:13 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:41:13 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:41:14 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:41:14 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Starting heal instance info cache {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9842}}
Aug 09 02:41:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "refresh_cache-41b5e305-3973-4937-b262-d32db1091e20" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:41:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquired lock "refresh_cache-41b5e305-3973-4937-b262-d32db1091e20" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:41:15 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Forcefully refreshing network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1996}}
Aug 09 02:41:15 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Updating instance_info_cache with network_info: [{"id": "e0032717-884b-4439-b7fc-42803aa7a802", "address": "fa:16:3e:49:57:14", "network": {"id": "9a452414-d61f-4045-a6cc-9be93c892c4c", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "321111f935dd4e8aab4a083206ea9a7e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape0032717-88", "ovs_interfaceid": "e0032717-884b-4439-b7fc-42803aa7a802", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:41:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Releasing lock "refresh_cache-41b5e305-3973-4937-b262-d32db1091e20" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:41:15 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Updated the network info_cache for instance {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9913}}
Aug 09 02:41:15 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:41:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:41:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:41:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:41:15 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Auditing locally available compute resources for user (node: user) {{(pid=72351) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Aug 09 02:41:15 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdg (vda) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:41:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/41b5e305-3973-4937-b262-d32db1091e20/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:41:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/41b5e305-3973-4937-b262-d32db1091e20/disk --force-share --output=json" returned: 0 in 0.140s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:41:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/41b5e305-3973-4937-b262-d32db1091e20/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:41:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/41b5e305-3973-4937-b262-d32db1091e20/disk --force-share --output=json" returned: 0 in 0.149s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:41:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c740d131-98a9-4b14-a75a-a33aa7c72b51/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:41:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c740d131-98a9-4b14-a75a-a33aa7c72b51/disk --force-share --output=json" returned: 0 in 0.144s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:41:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c740d131-98a9-4b14-a75a-a33aa7c72b51/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:41:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c740d131-98a9-4b14-a75a-a33aa7c72b51/disk --force-share --output=json" returned: 0 in 0.143s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:41:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:41:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f/disk --force-share --output=json" returned: 0 in 0.144s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:41:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:41:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:41:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6de75cee-9bf1-4f9f-8fa3-977d9432b805/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:41:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6de75cee-9bf1-4f9f-8fa3-977d9432b805/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:41:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6de75cee-9bf1-4f9f-8fa3-977d9432b805/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:41:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6de75cee-9bf1-4f9f-8fa3-977d9432b805/disk --force-share --output=json" returned: 0 in 0.131s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:41:16 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdb (vdb) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:41:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:41:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json" returned: 0 in 0.148s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:41:16 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:41:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:41:17 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:41:17 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:41:17 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Hypervisor/Node resource view: name=user free_ram=8203MB free_disk=25.044273376464844GB free_vcpus=6 pci_devices=[{"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72351) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Aug 09 02:41:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:41:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:41:17 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance a7f3d076-864b-4b68-9207-214ce2c21444 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:41:17 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 957c1510-1320-4492-9052-a58ff540319d actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:41:17 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 41b5e305-3973-4937-b262-d32db1091e20 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:41:17 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 6de75cee-9bf1-4f9f-8fa3-977d9432b805 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:41:17 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance c740d131-98a9-4b14-a75a-a33aa7c72b51 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:41:17 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:41:17 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Total usable vcpus: 12, total allocated vcpus: 6 {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Aug 09 02:41:17 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Final resource view: name=user phys_ram=16011MB used_ram=3584MB phys_disk=40GB used_disk=5GB total_vcpus=12 used_vcpus=6 pci_stats=[] {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Aug 09 02:41:17 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:41:17 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:41:17 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Compute_service record updated for user:user {{(pid=72351) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Aug 09 02:41:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.361s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:41:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:41:19 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:41:19 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:41:19 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:41:19 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:41:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-8ccdb767-2d45-47d8-ad03-bfd1a86e0ace tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Acquiring lock "41b5e305-3973-4937-b262-d32db1091e20" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:41:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-8ccdb767-2d45-47d8-ad03-bfd1a86e0ace tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "41b5e305-3973-4937-b262-d32db1091e20" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:41:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-8ccdb767-2d45-47d8-ad03-bfd1a86e0ace tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Acquiring lock "41b5e305-3973-4937-b262-d32db1091e20-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:41:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-8ccdb767-2d45-47d8-ad03-bfd1a86e0ace tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "41b5e305-3973-4937-b262-d32db1091e20-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:41:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-8ccdb767-2d45-47d8-ad03-bfd1a86e0ace tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "41b5e305-3973-4937-b262-d32db1091e20-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:41:20 user nova-compute[72351]: INFO nova.compute.manager [None req-8ccdb767-2d45-47d8-ad03-bfd1a86e0ace tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Terminating instance
Aug 09 02:41:20 user nova-compute[72351]: DEBUG nova.compute.manager [None req-8ccdb767-2d45-47d8-ad03-bfd1a86e0ace tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Start destroying the instance on the hypervisor. {{(pid=72351) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3124}}
Aug 09 02:41:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:41:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:41:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:41:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:41:21 user nova-compute[72351]: DEBUG nova.compute.manager [req-b72592af-3234-441d-a1b3-564582548218 req-ba09e59c-b021-4011-93f4-75ad89a86c8c service nova] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Received event network-vif-unplugged-e0032717-884b-4439-b7fc-42803aa7a802 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:41:21 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-b72592af-3234-441d-a1b3-564582548218 req-ba09e59c-b021-4011-93f4-75ad89a86c8c service nova] Acquiring lock "41b5e305-3973-4937-b262-d32db1091e20-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:41:21 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-b72592af-3234-441d-a1b3-564582548218 req-ba09e59c-b021-4011-93f4-75ad89a86c8c service nova] Lock "41b5e305-3973-4937-b262-d32db1091e20-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:41:21 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-b72592af-3234-441d-a1b3-564582548218 req-ba09e59c-b021-4011-93f4-75ad89a86c8c service nova] Lock "41b5e305-3973-4937-b262-d32db1091e20-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:41:21 user nova-compute[72351]: DEBUG nova.compute.manager [req-b72592af-3234-441d-a1b3-564582548218 req-ba09e59c-b021-4011-93f4-75ad89a86c8c service nova] [instance: 41b5e305-3973-4937-b262-d32db1091e20] No waiting events found dispatching network-vif-unplugged-e0032717-884b-4439-b7fc-42803aa7a802 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:41:21 user nova-compute[72351]: DEBUG nova.compute.manager [req-b72592af-3234-441d-a1b3-564582548218 req-ba09e59c-b021-4011-93f4-75ad89a86c8c service nova] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Received event network-vif-unplugged-e0032717-884b-4439-b7fc-42803aa7a802 for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:41:21 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Instance destroyed successfully.
Aug 09 02:41:21 user nova-compute[72351]: DEBUG nova.objects.instance [None req-8ccdb767-2d45-47d8-ad03-bfd1a86e0ace tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lazy-loading 'resources' on Instance uuid 41b5e305-3973-4937-b262-d32db1091e20 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:41:21 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-8ccdb767-2d45-47d8-ad03-bfd1a86e0ace tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:31:58Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-427830094',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-427830094',id=40,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:32:07Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='321111f935dd4e8aab4a083206ea9a7e',ramdisk_id='',reservation_id='r-iurnxcqs',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-2108329302',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:33:55Z,user_data=None,user_id='73d05c48be5640c28455e8ba8b05ef31',uuid=41b5e305-3973-4937-b262-d32db1091e20,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "e0032717-884b-4439-b7fc-42803aa7a802", "address": "fa:16:3e:49:57:14", "network": {"id": "9a452414-d61f-4045-a6cc-9be93c892c4c", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "321111f935dd4e8aab4a083206ea9a7e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape0032717-88", "ovs_interfaceid": "e0032717-884b-4439-b7fc-42803aa7a802", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:41:21 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-8ccdb767-2d45-47d8-ad03-bfd1a86e0ace tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Converting VIF {"id": "e0032717-884b-4439-b7fc-42803aa7a802", "address": "fa:16:3e:49:57:14", "network": {"id": "9a452414-d61f-4045-a6cc-9be93c892c4c", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "321111f935dd4e8aab4a083206ea9a7e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape0032717-88", "ovs_interfaceid": "e0032717-884b-4439-b7fc-42803aa7a802", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:41:21 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-8ccdb767-2d45-47d8-ad03-bfd1a86e0ace tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:49:57:14,bridge_name='br-int',has_traffic_filtering=True,id=e0032717-884b-4439-b7fc-42803aa7a802,network=Network(9a452414-d61f-4045-a6cc-9be93c892c4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape0032717-88') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:41:21 user nova-compute[72351]: DEBUG os_vif [None req-8ccdb767-2d45-47d8-ad03-bfd1a86e0ace tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:49:57:14,bridge_name='br-int',has_traffic_filtering=True,id=e0032717-884b-4439-b7fc-42803aa7a802,network=Network(9a452414-d61f-4045-a6cc-9be93c892c4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape0032717-88') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:41:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:41:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tape0032717-88, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:41:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:41:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:41:21 user nova-compute[72351]: INFO os_vif [None req-8ccdb767-2d45-47d8-ad03-bfd1a86e0ace tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:49:57:14,bridge_name='br-int',has_traffic_filtering=True,id=e0032717-884b-4439-b7fc-42803aa7a802,network=Network(9a452414-d61f-4045-a6cc-9be93c892c4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape0032717-88')
Aug 09 02:41:21 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-8ccdb767-2d45-47d8-ad03-bfd1a86e0ace tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Deleting instance files /opt/stack/data/nova/instances/41b5e305-3973-4937-b262-d32db1091e20_del
Aug 09 02:41:21 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-8ccdb767-2d45-47d8-ad03-bfd1a86e0ace tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Deletion of /opt/stack/data/nova/instances/41b5e305-3973-4937-b262-d32db1091e20_del complete
Aug 09 02:41:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:41:21 user nova-compute[72351]: INFO nova.compute.manager [None req-8ccdb767-2d45-47d8-ad03-bfd1a86e0ace tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Took 0.72 seconds to destroy the instance on the hypervisor.
Aug 09 02:41:21 user nova-compute[72351]: DEBUG oslo.service.loopingcall [None req-8ccdb767-2d45-47d8-ad03-bfd1a86e0ace tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72351) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Aug 09 02:41:21 user nova-compute[72351]: DEBUG nova.compute.manager [-] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Deallocating network for instance {{(pid=72351) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2263}}
Aug 09 02:41:21 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 41b5e305-3973-4937-b262-d32db1091e20] deallocate_for_instance() {{(pid=72351) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1795}}
Aug 09 02:41:22 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:41:22 user nova-compute[72351]: DEBUG nova.compute.manager [req-dac90af9-a13b-4e8f-bad1-fddf18ed56ea req-2d1e802e-90e8-4e3c-9ceb-49143d04341d service nova] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Received event network-vif-deleted-e0032717-884b-4439-b7fc-42803aa7a802 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:41:22 user nova-compute[72351]: INFO nova.compute.manager [req-dac90af9-a13b-4e8f-bad1-fddf18ed56ea req-2d1e802e-90e8-4e3c-9ceb-49143d04341d service nova] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Neutron deleted interface e0032717-884b-4439-b7fc-42803aa7a802; detaching it from the instance and deleting it from the info cache
Aug 09 02:41:22 user nova-compute[72351]: DEBUG nova.network.neutron [req-dac90af9-a13b-4e8f-bad1-fddf18ed56ea req-2d1e802e-90e8-4e3c-9ceb-49143d04341d service nova] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:41:22 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Took 0.68 seconds to deallocate network for instance.
Aug 09 02:41:22 user nova-compute[72351]: DEBUG nova.compute.manager [req-dac90af9-a13b-4e8f-bad1-fddf18ed56ea req-2d1e802e-90e8-4e3c-9ceb-49143d04341d service nova] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Detach interface failed, port_id=e0032717-884b-4439-b7fc-42803aa7a802, reason: Instance 41b5e305-3973-4937-b262-d32db1091e20 could not be found. {{(pid=72351) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10866}}
Aug 09 02:41:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-8ccdb767-2d45-47d8-ad03-bfd1a86e0ace tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:41:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-8ccdb767-2d45-47d8-ad03-bfd1a86e0ace tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:41:22 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-8ccdb767-2d45-47d8-ad03-bfd1a86e0ace tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:41:22 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-8ccdb767-2d45-47d8-ad03-bfd1a86e0ace tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:41:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-8ccdb767-2d45-47d8-ad03-bfd1a86e0ace tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.225s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:41:22 user nova-compute[72351]: INFO nova.scheduler.client.report [None req-8ccdb767-2d45-47d8-ad03-bfd1a86e0ace tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Deleted allocations for instance 41b5e305-3973-4937-b262-d32db1091e20
Aug 09 02:41:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-8ccdb767-2d45-47d8-ad03-bfd1a86e0ace tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "41b5e305-3973-4937-b262-d32db1091e20" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 1.825s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:41:23 user nova-compute[72351]: DEBUG nova.compute.manager [req-f457d6b8-fae1-46df-abd3-17b6707f806a req-0768186c-9dcf-4edb-b0ef-e3d3f43f4703 service nova] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Received event network-vif-plugged-e0032717-884b-4439-b7fc-42803aa7a802 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:41:23 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-f457d6b8-fae1-46df-abd3-17b6707f806a req-0768186c-9dcf-4edb-b0ef-e3d3f43f4703 service nova] Acquiring lock "41b5e305-3973-4937-b262-d32db1091e20-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:41:23 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-f457d6b8-fae1-46df-abd3-17b6707f806a req-0768186c-9dcf-4edb-b0ef-e3d3f43f4703 service nova] Lock "41b5e305-3973-4937-b262-d32db1091e20-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:41:23 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-f457d6b8-fae1-46df-abd3-17b6707f806a req-0768186c-9dcf-4edb-b0ef-e3d3f43f4703 service nova] Lock "41b5e305-3973-4937-b262-d32db1091e20-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:41:23 user nova-compute[72351]: DEBUG nova.compute.manager [req-f457d6b8-fae1-46df-abd3-17b6707f806a req-0768186c-9dcf-4edb-b0ef-e3d3f43f4703 service nova] [instance: 41b5e305-3973-4937-b262-d32db1091e20] No waiting events found dispatching network-vif-plugged-e0032717-884b-4439-b7fc-42803aa7a802 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:41:23 user nova-compute[72351]: WARNING nova.compute.manager [req-f457d6b8-fae1-46df-abd3-17b6707f806a req-0768186c-9dcf-4edb-b0ef-e3d3f43f4703 service nova] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Received unexpected event network-vif-plugged-e0032717-884b-4439-b7fc-42803aa7a802 for instance with vm_state deleted and task_state None.
Aug 09 02:41:26 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:41:31 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:41:36 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691548881.4080212, 41b5e305-3973-4937-b262-d32db1091e20 => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:41:36 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 41b5e305-3973-4937-b262-d32db1091e20] VM Stopped (Lifecycle Event)
Aug 09 02:41:36 user nova-compute[72351]: DEBUG nova.compute.manager [None req-0aae4b7f-3b57-4873-a084-97ccabeadf75 None None] [instance: 41b5e305-3973-4937-b262-d32db1091e20] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:41:36 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:41:41 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:41:42 user nova-compute[72351]: DEBUG nova.compute.manager [req-dff15f67-ae33-4d22-aa45-cb1e4f3e5137 req-26f25d4c-a7ee-4d69-80a9-28e5c0911672 service nova] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Received event network-changed-ef71c011-3c08-4927-bf2b-763a63fe8c6f {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:41:42 user nova-compute[72351]: DEBUG nova.compute.manager [req-dff15f67-ae33-4d22-aa45-cb1e4f3e5137 req-26f25d4c-a7ee-4d69-80a9-28e5c0911672 service nova] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Refreshing instance network info cache due to event network-changed-ef71c011-3c08-4927-bf2b-763a63fe8c6f. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:41:42 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-dff15f67-ae33-4d22-aa45-cb1e4f3e5137 req-26f25d4c-a7ee-4d69-80a9-28e5c0911672 service nova] Acquiring lock "refresh_cache-c740d131-98a9-4b14-a75a-a33aa7c72b51" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:41:42 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-dff15f67-ae33-4d22-aa45-cb1e4f3e5137 req-26f25d4c-a7ee-4d69-80a9-28e5c0911672 service nova] Acquired lock "refresh_cache-c740d131-98a9-4b14-a75a-a33aa7c72b51" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:41:42 user nova-compute[72351]: DEBUG nova.network.neutron [req-dff15f67-ae33-4d22-aa45-cb1e4f3e5137 req-26f25d4c-a7ee-4d69-80a9-28e5c0911672 service nova] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Refreshing network info cache for port ef71c011-3c08-4927-bf2b-763a63fe8c6f {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:41:42 user nova-compute[72351]: DEBUG nova.network.neutron [req-dff15f67-ae33-4d22-aa45-cb1e4f3e5137 req-26f25d4c-a7ee-4d69-80a9-28e5c0911672 service nova] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Updated VIF entry in instance network info cache for port ef71c011-3c08-4927-bf2b-763a63fe8c6f. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:41:42 user nova-compute[72351]: DEBUG nova.network.neutron [req-dff15f67-ae33-4d22-aa45-cb1e4f3e5137 req-26f25d4c-a7ee-4d69-80a9-28e5c0911672 service nova] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Updating instance_info_cache with network_info: [{"id": "ef71c011-3c08-4927-bf2b-763a63fe8c6f", "address": "fa:16:3e:73:9f:90", "network": {"id": "da9be96c-ce20-4037-999c-230ca799fb06", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-661780305-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.69", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5ff300d27c9a4dd3b7ecd507b3627d98", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapef71c011-3c", "ovs_interfaceid": "ef71c011-3c08-4927-bf2b-763a63fe8c6f", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:41:42 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-dff15f67-ae33-4d22-aa45-cb1e4f3e5137 req-26f25d4c-a7ee-4d69-80a9-28e5c0911672 service nova] Releasing lock "refresh_cache-c740d131-98a9-4b14-a75a-a33aa7c72b51" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:41:43 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:41:46 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:41:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-cbb4a837-ef11-4874-9094-15561bf76b0a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Acquiring lock "c740d131-98a9-4b14-a75a-a33aa7c72b51" by "nova.compute.manager.ComputeManager.shelve_instance.<locals>.do_shelve_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:41:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-cbb4a837-ef11-4874-9094-15561bf76b0a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "c740d131-98a9-4b14-a75a-a33aa7c72b51" acquired by "nova.compute.manager.ComputeManager.shelve_instance.<locals>.do_shelve_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:41:47 user nova-compute[72351]: INFO nova.compute.manager [None req-cbb4a837-ef11-4874-9094-15561bf76b0a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Shelving
Aug 09 02:41:47 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:41:47 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:41:47 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:41:47 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:41:47 user nova-compute[72351]: DEBUG nova.compute.manager [req-aa93bdfc-8bd9-4525-9142-67baa9da8a88 req-e30aa1c0-d91a-47d1-96f2-f1c49a369023 service nova] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Received event network-vif-unplugged-ef71c011-3c08-4927-bf2b-763a63fe8c6f {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:41:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-aa93bdfc-8bd9-4525-9142-67baa9da8a88 req-e30aa1c0-d91a-47d1-96f2-f1c49a369023 service nova] Acquiring lock "c740d131-98a9-4b14-a75a-a33aa7c72b51-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:41:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-aa93bdfc-8bd9-4525-9142-67baa9da8a88 req-e30aa1c0-d91a-47d1-96f2-f1c49a369023 service nova] Lock "c740d131-98a9-4b14-a75a-a33aa7c72b51-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:41:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-aa93bdfc-8bd9-4525-9142-67baa9da8a88 req-e30aa1c0-d91a-47d1-96f2-f1c49a369023 service nova] Lock "c740d131-98a9-4b14-a75a-a33aa7c72b51-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:41:47 user nova-compute[72351]: DEBUG nova.compute.manager [req-aa93bdfc-8bd9-4525-9142-67baa9da8a88 req-e30aa1c0-d91a-47d1-96f2-f1c49a369023 service nova] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] No waiting events found dispatching network-vif-unplugged-ef71c011-3c08-4927-bf2b-763a63fe8c6f {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:41:47 user nova-compute[72351]: WARNING nova.compute.manager [req-aa93bdfc-8bd9-4525-9142-67baa9da8a88 req-e30aa1c0-d91a-47d1-96f2-f1c49a369023 service nova] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Received unexpected event network-vif-unplugged-ef71c011-3c08-4927-bf2b-763a63fe8c6f for instance with vm_state active and task_state shelving.
Aug 09 02:41:47 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Instance destroyed successfully.
Aug 09 02:41:47 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-cbb4a837-ef11-4874-9094-15561bf76b0a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Beginning cold snapshot process
Aug 09 02:41:48 user nova-compute[72351]: DEBUG nova.privsep.utils [None req-cbb4a837-ef11-4874-9094-15561bf76b0a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=72351) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Aug 09 02:41:48 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-cbb4a837-ef11-4874-9094-15561bf76b0a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Running cmd (subprocess): qemu-img convert -t none -O qcow2 -f qcow2 /opt/stack/data/nova/instances/c740d131-98a9-4b14-a75a-a33aa7c72b51/disk /opt/stack/data/nova/instances/snapshots/tmp7064rpy4/e7dc78794cbb48c6835cc67799ce68c2 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:41:48 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-cbb4a837-ef11-4874-9094-15561bf76b0a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] CMD "qemu-img convert -t none -O qcow2 -f qcow2 /opt/stack/data/nova/instances/c740d131-98a9-4b14-a75a-a33aa7c72b51/disk /opt/stack/data/nova/instances/snapshots/tmp7064rpy4/e7dc78794cbb48c6835cc67799ce68c2" returned: 0 in 0.555s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:41:48 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-cbb4a837-ef11-4874-9094-15561bf76b0a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Snapshot extracted, beginning image upload
Aug 09 02:41:48 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:41:49 user nova-compute[72351]: DEBUG nova.compute.manager [req-3a73097d-1baa-4bfb-8ac9-073c58d05979 req-35d55200-548e-480f-8e9b-6b664b800c7b service nova] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Received event network-vif-plugged-ef71c011-3c08-4927-bf2b-763a63fe8c6f {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:41:49 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-3a73097d-1baa-4bfb-8ac9-073c58d05979 req-35d55200-548e-480f-8e9b-6b664b800c7b service nova] Acquiring lock "c740d131-98a9-4b14-a75a-a33aa7c72b51-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:41:49 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-3a73097d-1baa-4bfb-8ac9-073c58d05979 req-35d55200-548e-480f-8e9b-6b664b800c7b service nova] Lock "c740d131-98a9-4b14-a75a-a33aa7c72b51-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:41:49 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-3a73097d-1baa-4bfb-8ac9-073c58d05979 req-35d55200-548e-480f-8e9b-6b664b800c7b service nova] Lock "c740d131-98a9-4b14-a75a-a33aa7c72b51-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:41:49 user nova-compute[72351]: DEBUG nova.compute.manager [req-3a73097d-1baa-4bfb-8ac9-073c58d05979 req-35d55200-548e-480f-8e9b-6b664b800c7b service nova] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] No waiting events found dispatching network-vif-plugged-ef71c011-3c08-4927-bf2b-763a63fe8c6f {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:41:49 user nova-compute[72351]: WARNING nova.compute.manager [req-3a73097d-1baa-4bfb-8ac9-073c58d05979 req-35d55200-548e-480f-8e9b-6b664b800c7b service nova] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Received unexpected event network-vif-plugged-ef71c011-3c08-4927-bf2b-763a63fe8c6f for instance with vm_state active and task_state shelving_image_uploading.
Aug 09 02:41:50 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-cbb4a837-ef11-4874-9094-15561bf76b0a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Snapshot image upload complete
Aug 09 02:41:50 user nova-compute[72351]: DEBUG nova.compute.manager [None req-cbb4a837-ef11-4874-9094-15561bf76b0a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:41:50 user nova-compute[72351]: INFO nova.compute.manager [None req-cbb4a837-ef11-4874-9094-15561bf76b0a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Shelve offloading
Aug 09 02:41:50 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Instance destroyed successfully.
Aug 09 02:41:50 user nova-compute[72351]: DEBUG nova.compute.manager [None req-cbb4a837-ef11-4874-9094-15561bf76b0a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:41:50 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-cbb4a837-ef11-4874-9094-15561bf76b0a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Acquiring lock "refresh_cache-c740d131-98a9-4b14-a75a-a33aa7c72b51" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:41:50 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-cbb4a837-ef11-4874-9094-15561bf76b0a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Acquired lock "refresh_cache-c740d131-98a9-4b14-a75a-a33aa7c72b51" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:41:50 user nova-compute[72351]: DEBUG nova.network.neutron [None req-cbb4a837-ef11-4874-9094-15561bf76b0a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:41:51 user nova-compute[72351]: DEBUG nova.network.neutron [None req-cbb4a837-ef11-4874-9094-15561bf76b0a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Updating instance_info_cache with network_info: [{"id": "ef71c011-3c08-4927-bf2b-763a63fe8c6f", "address": "fa:16:3e:73:9f:90", "network": {"id": "da9be96c-ce20-4037-999c-230ca799fb06", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-661780305-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.69", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5ff300d27c9a4dd3b7ecd507b3627d98", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapef71c011-3c", "ovs_interfaceid": "ef71c011-3c08-4927-bf2b-763a63fe8c6f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:41:51 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:41:51 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-cbb4a837-ef11-4874-9094-15561bf76b0a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Releasing lock "refresh_cache-c740d131-98a9-4b14-a75a-a33aa7c72b51" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:41:52 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Instance destroyed successfully.
Aug 09 02:41:52 user nova-compute[72351]: DEBUG nova.objects.instance [None req-cbb4a837-ef11-4874-9094-15561bf76b0a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lazy-loading 'resources' on Instance uuid c740d131-98a9-4b14-a75a-a33aa7c72b51 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:41:52 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-cbb4a837-ef11-4874-9094-15561bf76b0a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:39:51Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-server-1075911547',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumeshelvetestjson-server-1075911547',id=45,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBAhm3wxNXc3qjoZ/WjO45nlSmWKHkz9mk2gcB1iSKP8F60j8gzonzjkrypbEG5qNiTKrTFWzLZuC0FgGJv+9yI6H0VXa1JB0Qe7iMGyt2g59NXDdngUdeDSufXPbcrFG6Q==',key_name='tempest-keypair-1369286576',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:39:57Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=<?>,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=4,progress=0,project_id='5ff300d27c9a4dd3b7ecd507b3627d98',ramdisk_id='',reservation_id='r-gx6s8m4c',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeShelveTestJSON-1982056968',owner_user_name='tempest-AttachVolumeShelveTestJSON-1982056968-project-member',shelved_at='2023-08-09T02:41:50.820347',shelved_host='user',shelved_image_id='6195b054-3ba1-4222-8581-110cbefd136a'},tags=<?>,task_state='shelving_offloading',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:41:49Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='ad8fc03436a84bdfb13ba6abecd4f27b',uuid=c740d131-98a9-4b14-a75a-a33aa7c72b51,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='shelved') vif={"id": "ef71c011-3c08-4927-bf2b-763a63fe8c6f", "address": "fa:16:3e:73:9f:90", "network": {"id": "da9be96c-ce20-4037-999c-230ca799fb06", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-661780305-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.69", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5ff300d27c9a4dd3b7ecd507b3627d98", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapef71c011-3c", "ovs_interfaceid": "ef71c011-3c08-4927-bf2b-763a63fe8c6f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:41:52 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-cbb4a837-ef11-4874-9094-15561bf76b0a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Converting VIF {"id": "ef71c011-3c08-4927-bf2b-763a63fe8c6f", "address": "fa:16:3e:73:9f:90", "network": {"id": "da9be96c-ce20-4037-999c-230ca799fb06", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-661780305-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.69", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5ff300d27c9a4dd3b7ecd507b3627d98", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapef71c011-3c", "ovs_interfaceid": "ef71c011-3c08-4927-bf2b-763a63fe8c6f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:41:52 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-cbb4a837-ef11-4874-9094-15561bf76b0a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:73:9f:90,bridge_name='br-int',has_traffic_filtering=True,id=ef71c011-3c08-4927-bf2b-763a63fe8c6f,network=Network(da9be96c-ce20-4037-999c-230ca799fb06),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapef71c011-3c') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:41:52 user nova-compute[72351]: DEBUG os_vif [None req-cbb4a837-ef11-4874-9094-15561bf76b0a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:73:9f:90,bridge_name='br-int',has_traffic_filtering=True,id=ef71c011-3c08-4927-bf2b-763a63fe8c6f,network=Network(da9be96c-ce20-4037-999c-230ca799fb06),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapef71c011-3c') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:41:52 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:41:52 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapef71c011-3c, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:41:52 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:41:52 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:41:52 user nova-compute[72351]: INFO os_vif [None req-cbb4a837-ef11-4874-9094-15561bf76b0a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:73:9f:90,bridge_name='br-int',has_traffic_filtering=True,id=ef71c011-3c08-4927-bf2b-763a63fe8c6f,network=Network(da9be96c-ce20-4037-999c-230ca799fb06),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapef71c011-3c')
Aug 09 02:41:52 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-cbb4a837-ef11-4874-9094-15561bf76b0a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Deleting instance files /opt/stack/data/nova/instances/c740d131-98a9-4b14-a75a-a33aa7c72b51_del
Aug 09 02:41:52 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-cbb4a837-ef11-4874-9094-15561bf76b0a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Deletion of /opt/stack/data/nova/instances/c740d131-98a9-4b14-a75a-a33aa7c72b51_del complete
Aug 09 02:41:52 user nova-compute[72351]: DEBUG nova.compute.manager [req-0eaed999-f6c7-46ba-ac87-3853156cdf56 req-72ec619b-3fb4-4078-b896-308bc0d4d18f service nova] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Received event network-changed-ef71c011-3c08-4927-bf2b-763a63fe8c6f {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:41:52 user nova-compute[72351]: DEBUG nova.compute.manager [req-0eaed999-f6c7-46ba-ac87-3853156cdf56 req-72ec619b-3fb4-4078-b896-308bc0d4d18f service nova] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Refreshing instance network info cache due to event network-changed-ef71c011-3c08-4927-bf2b-763a63fe8c6f. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:41:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-0eaed999-f6c7-46ba-ac87-3853156cdf56 req-72ec619b-3fb4-4078-b896-308bc0d4d18f service nova] Acquiring lock "refresh_cache-c740d131-98a9-4b14-a75a-a33aa7c72b51" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:41:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-0eaed999-f6c7-46ba-ac87-3853156cdf56 req-72ec619b-3fb4-4078-b896-308bc0d4d18f service nova] Acquired lock "refresh_cache-c740d131-98a9-4b14-a75a-a33aa7c72b51" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:41:52 user nova-compute[72351]: DEBUG nova.network.neutron [req-0eaed999-f6c7-46ba-ac87-3853156cdf56 req-72ec619b-3fb4-4078-b896-308bc0d4d18f service nova] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Refreshing network info cache for port ef71c011-3c08-4927-bf2b-763a63fe8c6f {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:41:52 user nova-compute[72351]: INFO nova.scheduler.client.report [None req-cbb4a837-ef11-4874-9094-15561bf76b0a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Deleted allocations for instance c740d131-98a9-4b14-a75a-a33aa7c72b51
Aug 09 02:41:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-cbb4a837-ef11-4874-9094-15561bf76b0a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:41:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-cbb4a837-ef11-4874-9094-15561bf76b0a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:41:52 user nova-compute[72351]: DEBUG nova.objects.instance [None req-cbb4a837-ef11-4874-9094-15561bf76b0a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lazy-loading 'numa_topology' on Instance uuid c740d131-98a9-4b14-a75a-a33aa7c72b51 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:41:52 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-cbb4a837-ef11-4874-9094-15561bf76b0a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:41:52 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-cbb4a837-ef11-4874-9094-15561bf76b0a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:41:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-cbb4a837-ef11-4874-9094-15561bf76b0a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.194s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:41:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-cbb4a837-ef11-4874-9094-15561bf76b0a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "c740d131-98a9-4b14-a75a-a33aa7c72b51" "released" by "nova.compute.manager.ComputeManager.shelve_instance.<locals>.do_shelve_instance" :: held 5.473s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:41:52 user nova-compute[72351]: DEBUG nova.network.neutron [req-0eaed999-f6c7-46ba-ac87-3853156cdf56 req-72ec619b-3fb4-4078-b896-308bc0d4d18f service nova] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Updated VIF entry in instance network info cache for port ef71c011-3c08-4927-bf2b-763a63fe8c6f. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:41:52 user nova-compute[72351]: DEBUG nova.network.neutron [req-0eaed999-f6c7-46ba-ac87-3853156cdf56 req-72ec619b-3fb4-4078-b896-308bc0d4d18f service nova] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Updating instance_info_cache with network_info: [{"id": "ef71c011-3c08-4927-bf2b-763a63fe8c6f", "address": "fa:16:3e:73:9f:90", "network": {"id": "da9be96c-ce20-4037-999c-230ca799fb06", "bridge": null, "label": "tempest-AttachVolumeShelveTestJSON-661780305-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.69", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5ff300d27c9a4dd3b7ecd507b3627d98", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "unbound", "details": {}, "devname": "tapef71c011-3c", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:41:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-0eaed999-f6c7-46ba-ac87-3853156cdf56 req-72ec619b-3fb4-4078-b896-308bc0d4d18f service nova] Releasing lock "refresh_cache-c740d131-98a9-4b14-a75a-a33aa7c72b51" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:41:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:41:57 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:41:58 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:42:02 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:42:02 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691548907.892313, c740d131-98a9-4b14-a75a-a33aa7c72b51 => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:42:02 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] VM Stopped (Lifecycle Event)
Aug 09 02:42:02 user nova-compute[72351]: DEBUG nova.compute.manager [None req-9367a35d-4024-4e22-9f8f-4b15afb34521 None None] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:42:07 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:42:07 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:42:07 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 02:42:07 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:42:07 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:42:07 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:42:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b00b1330-04aa-4d3f-93ba-1be0780ea8cc tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Acquiring lock "957c1510-1320-4492-9052-a58ff540319d" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:42:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b00b1330-04aa-4d3f-93ba-1be0780ea8cc tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "957c1510-1320-4492-9052-a58ff540319d" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:42:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b00b1330-04aa-4d3f-93ba-1be0780ea8cc tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Acquiring lock "957c1510-1320-4492-9052-a58ff540319d-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:42:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b00b1330-04aa-4d3f-93ba-1be0780ea8cc tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "957c1510-1320-4492-9052-a58ff540319d-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:42:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b00b1330-04aa-4d3f-93ba-1be0780ea8cc tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "957c1510-1320-4492-9052-a58ff540319d-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:42:11 user nova-compute[72351]: INFO nova.compute.manager [None req-b00b1330-04aa-4d3f-93ba-1be0780ea8cc tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 957c1510-1320-4492-9052-a58ff540319d] Terminating instance
Aug 09 02:42:11 user nova-compute[72351]: DEBUG nova.compute.manager [None req-b00b1330-04aa-4d3f-93ba-1be0780ea8cc tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 957c1510-1320-4492-9052-a58ff540319d] Start destroying the instance on the hypervisor. {{(pid=72351) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3124}}
Aug 09 02:42:11 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:42:11 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:42:11 user nova-compute[72351]: DEBUG nova.compute.manager [req-8a828171-979a-4cd3-8ba6-f5554a399ae3 req-a8efd22a-caa1-4d85-ad0a-6d7a15f7d643 service nova] [instance: 957c1510-1320-4492-9052-a58ff540319d] Received event network-vif-unplugged-e679cfd9-123f-4b88-8fd2-397e4c8616cc {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:42:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-8a828171-979a-4cd3-8ba6-f5554a399ae3 req-a8efd22a-caa1-4d85-ad0a-6d7a15f7d643 service nova] Acquiring lock "957c1510-1320-4492-9052-a58ff540319d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:42:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-8a828171-979a-4cd3-8ba6-f5554a399ae3 req-a8efd22a-caa1-4d85-ad0a-6d7a15f7d643 service nova] Lock "957c1510-1320-4492-9052-a58ff540319d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:42:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-8a828171-979a-4cd3-8ba6-f5554a399ae3 req-a8efd22a-caa1-4d85-ad0a-6d7a15f7d643 service nova] Lock "957c1510-1320-4492-9052-a58ff540319d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:42:11 user nova-compute[72351]: DEBUG nova.compute.manager [req-8a828171-979a-4cd3-8ba6-f5554a399ae3 req-a8efd22a-caa1-4d85-ad0a-6d7a15f7d643 service nova] [instance: 957c1510-1320-4492-9052-a58ff540319d] No waiting events found dispatching network-vif-unplugged-e679cfd9-123f-4b88-8fd2-397e4c8616cc {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:42:11 user nova-compute[72351]: DEBUG nova.compute.manager [req-8a828171-979a-4cd3-8ba6-f5554a399ae3 req-a8efd22a-caa1-4d85-ad0a-6d7a15f7d643 service nova] [instance: 957c1510-1320-4492-9052-a58ff540319d] Received event network-vif-unplugged-e679cfd9-123f-4b88-8fd2-397e4c8616cc for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:42:11 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:42:11 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:42:11 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 957c1510-1320-4492-9052-a58ff540319d] Instance destroyed successfully.
Aug 09 02:42:11 user nova-compute[72351]: DEBUG nova.objects.instance [None req-b00b1330-04aa-4d3f-93ba-1be0780ea8cc tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lazy-loading 'resources' on Instance uuid 957c1510-1320-4492-9052-a58ff540319d {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:42:11 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-b00b1330-04aa-4d3f-93ba-1be0780ea8cc tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:26:24Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-821495844',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-821495844',id=28,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:28:25Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='321111f935dd4e8aab4a083206ea9a7e',ramdisk_id='',reservation_id='r-081hk0tj',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-2108329302',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:30:12Z,user_data=None,user_id='73d05c48be5640c28455e8ba8b05ef31',uuid=957c1510-1320-4492-9052-a58ff540319d,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "e679cfd9-123f-4b88-8fd2-397e4c8616cc", "address": "fa:16:3e:7f:d6:1b", "network": {"id": "9a452414-d61f-4045-a6cc-9be93c892c4c", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "321111f935dd4e8aab4a083206ea9a7e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape679cfd9-12", "ovs_interfaceid": "e679cfd9-123f-4b88-8fd2-397e4c8616cc", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:42:11 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-b00b1330-04aa-4d3f-93ba-1be0780ea8cc tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Converting VIF {"id": "e679cfd9-123f-4b88-8fd2-397e4c8616cc", "address": "fa:16:3e:7f:d6:1b", "network": {"id": "9a452414-d61f-4045-a6cc-9be93c892c4c", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "321111f935dd4e8aab4a083206ea9a7e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape679cfd9-12", "ovs_interfaceid": "e679cfd9-123f-4b88-8fd2-397e4c8616cc", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:42:11 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-b00b1330-04aa-4d3f-93ba-1be0780ea8cc tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:7f:d6:1b,bridge_name='br-int',has_traffic_filtering=True,id=e679cfd9-123f-4b88-8fd2-397e4c8616cc,network=Network(9a452414-d61f-4045-a6cc-9be93c892c4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape679cfd9-12') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:42:11 user nova-compute[72351]: DEBUG os_vif [None req-b00b1330-04aa-4d3f-93ba-1be0780ea8cc tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:7f:d6:1b,bridge_name='br-int',has_traffic_filtering=True,id=e679cfd9-123f-4b88-8fd2-397e4c8616cc,network=Network(9a452414-d61f-4045-a6cc-9be93c892c4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape679cfd9-12') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:42:11 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:42:11 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tape679cfd9-12, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:42:11 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:42:11 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:42:11 user nova-compute[72351]: INFO os_vif [None req-b00b1330-04aa-4d3f-93ba-1be0780ea8cc tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:7f:d6:1b,bridge_name='br-int',has_traffic_filtering=True,id=e679cfd9-123f-4b88-8fd2-397e4c8616cc,network=Network(9a452414-d61f-4045-a6cc-9be93c892c4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape679cfd9-12')
Aug 09 02:42:11 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b00b1330-04aa-4d3f-93ba-1be0780ea8cc tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 957c1510-1320-4492-9052-a58ff540319d] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 02:42:11 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-b00b1330-04aa-4d3f-93ba-1be0780ea8cc tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'MbqJXivK', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdg'}, None), 'kwargs': True}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:42:11 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-b00b1330-04aa-4d3f-93ba-1be0780ea8cc tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:42:11 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-b00b1330-04aa-4d3f-93ba-1be0780ea8cc tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:42:11 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-b00b1330-04aa-4d3f-93ba-1be0780ea8cc tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:42:11 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-b00b1330-04aa-4d3f-93ba-1be0780ea8cc tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 02:42:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:42:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:42:12 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e5b98e4d-2510-4064-a263-8bf10c2c884b]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:af0b86fc-20d0-49f5-bf92-ee3c8c5783e4\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:42:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:42:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:42:12 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[98e5e1e7-089e-423f-82e2-d2ef7ed45ebf]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)\ntcp: [56] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:af0b86fc-20d0-49f5-bf92-ee3c8c5783e4 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:42:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-b00b1330-04aa-4d3f-93ba-1be0780ea8cc tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:42:12 user nova-compute[72351]: tcp: [56] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:af0b86fc-20d0-49f5-bf92-ee3c8c5783e4 (non-flash)
Aug 09 02:42:12 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:42:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-b00b1330-04aa-4d3f-93ba-1be0780ea8cc tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 (non-flash)
Aug 09 02:42:12 user nova-compute[72351]: tcp: [56] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:af0b86fc-20d0-49f5-bf92-ee3c8c5783e4 (non-flash)
Aug 09 02:42:12 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:42:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-b00b1330-04aa-4d3f-93ba-1be0780ea8cc tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fcfbc52e9e0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235'): ({'sdg'}, set())}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 02:42:12 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-b00b1330-04aa-4d3f-93ba-1be0780ea8cc tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Removing single pathed devices sdg {{(pid=72351) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Aug 09 02:42:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:42:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:42:12 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[eb11d8d1-86a4-4ab5-91a8-3492313a5e55]: (4, ('path checker states:\nup                  2\n\npaths: 0\nbusy: False\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:42:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdg {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:42:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdg" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:42:12 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[70df5cce-0a4d-4285-8e2f-d85fc4c8a337]: (4, ('ok\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:42:12 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-b00b1330-04aa-4d3f-93ba-1be0780ea8cc tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Flushing IO for device /dev/sdg {{(pid=72351) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Aug 09 02:42:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdg {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:42:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdg" returned: 0 in 0.014s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:42:12 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d358f5fe-e8c4-44ed-986d-244bc811e7a8]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:42:12 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-b00b1330-04aa-4d3f-93ba-1be0780ea8cc tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Remove SCSI device /dev/sdg with /sys/block/sdg/device/delete {{(pid=72351) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Aug 09 02:42:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdg/device/delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:42:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdg/device/delete" returned: 0 in 0.038s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:42:12 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[468e18e5-30fa-46e0-b4d9-2011c7af14ab]: (4, ('1', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:42:12 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-b00b1330-04aa-4d3f-93ba-1be0780ea8cc tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Checking to see if SCSI volumes sdg have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Aug 09 02:42:12 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-b00b1330-04aa-4d3f-93ba-1be0780ea8cc tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] SCSI volumes sdg have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Aug 09 02:42:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-b00b1330-04aa-4d3f-93ba-1be0780ea8cc tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235')] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 02:42:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:42:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:42:12 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[8d57e4b0-1029-4458-92a9-0467f1213860]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:42:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-b00b1330-04aa-4d3f-93ba-1be0780ea8cc tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:42:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 -p 172.16.0.220:3260 --logout {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:42:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 -p 172.16.0.220:3260 --logout" returned: 0 in 0.058s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:42:12 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[09232326-e50a-4727-a4fa-15771a93c31d]: (4, ('Logging out of session [sid: 26, target: iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235, portal: 172.16.0.220,3260]\nLogout of [sid: 26, target: iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:42:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-b00b1330-04aa-4d3f-93ba-1be0780ea8cc tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 26, target: iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235, portal: 172.16.0.220,3260]
Aug 09 02:42:12 user nova-compute[72351]: Logout of [sid: 26, target: iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235, portal: 172.16.0.220,3260] successful.
Aug 09 02:42:12 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:42:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 -p 172.16.0.220:3260 --op delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:42:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:492c8c70-5cca-4ba1-954d-e1a7ebea4235 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:42:12 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[6d754266-fb18-4705-bca4-d2a8d9cf27bd]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:42:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-b00b1330-04aa-4d3f-93ba-1be0780ea8cc tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:42:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-b00b1330-04aa-4d3f-93ba-1be0780ea8cc tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.271s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:42:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-b00b1330-04aa-4d3f-93ba-1be0780ea8cc tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] <== disconnect_volume: return (272ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:42:12 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b00b1330-04aa-4d3f-93ba-1be0780ea8cc tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 957c1510-1320-4492-9052-a58ff540319d] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 02:42:12 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-b00b1330-04aa-4d3f-93ba-1be0780ea8cc tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 957c1510-1320-4492-9052-a58ff540319d] Deleting instance files /opt/stack/data/nova/instances/957c1510-1320-4492-9052-a58ff540319d_del
Aug 09 02:42:12 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-b00b1330-04aa-4d3f-93ba-1be0780ea8cc tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 957c1510-1320-4492-9052-a58ff540319d] Deletion of /opt/stack/data/nova/instances/957c1510-1320-4492-9052-a58ff540319d_del complete
Aug 09 02:42:12 user nova-compute[72351]: INFO nova.compute.manager [None req-b00b1330-04aa-4d3f-93ba-1be0780ea8cc tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 957c1510-1320-4492-9052-a58ff540319d] Took 0.92 seconds to destroy the instance on the hypervisor.
Aug 09 02:42:12 user nova-compute[72351]: DEBUG oslo.service.loopingcall [None req-b00b1330-04aa-4d3f-93ba-1be0780ea8cc tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72351) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Aug 09 02:42:12 user nova-compute[72351]: DEBUG nova.compute.manager [-] [instance: 957c1510-1320-4492-9052-a58ff540319d] Deallocating network for instance {{(pid=72351) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2263}}
Aug 09 02:42:12 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 957c1510-1320-4492-9052-a58ff540319d] deallocate_for_instance() {{(pid=72351) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1795}}
Aug 09 02:42:12 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:42:13 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 957c1510-1320-4492-9052-a58ff540319d] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:42:13 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 957c1510-1320-4492-9052-a58ff540319d] Took 0.73 seconds to deallocate network for instance.
Aug 09 02:42:13 user nova-compute[72351]: DEBUG nova.compute.manager [req-4d907b5d-f2ca-4105-9a69-ff9d8c541527 req-29ed8712-1bba-43c9-827d-bdb71b2462cb service nova] [instance: 957c1510-1320-4492-9052-a58ff540319d] Received event network-vif-plugged-e679cfd9-123f-4b88-8fd2-397e4c8616cc {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:42:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-4d907b5d-f2ca-4105-9a69-ff9d8c541527 req-29ed8712-1bba-43c9-827d-bdb71b2462cb service nova] Acquiring lock "957c1510-1320-4492-9052-a58ff540319d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:42:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-4d907b5d-f2ca-4105-9a69-ff9d8c541527 req-29ed8712-1bba-43c9-827d-bdb71b2462cb service nova] Lock "957c1510-1320-4492-9052-a58ff540319d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:42:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-4d907b5d-f2ca-4105-9a69-ff9d8c541527 req-29ed8712-1bba-43c9-827d-bdb71b2462cb service nova] Lock "957c1510-1320-4492-9052-a58ff540319d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:42:13 user nova-compute[72351]: DEBUG nova.compute.manager [req-4d907b5d-f2ca-4105-9a69-ff9d8c541527 req-29ed8712-1bba-43c9-827d-bdb71b2462cb service nova] [instance: 957c1510-1320-4492-9052-a58ff540319d] No waiting events found dispatching network-vif-plugged-e679cfd9-123f-4b88-8fd2-397e4c8616cc {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:42:13 user nova-compute[72351]: WARNING nova.compute.manager [req-4d907b5d-f2ca-4105-9a69-ff9d8c541527 req-29ed8712-1bba-43c9-827d-bdb71b2462cb service nova] [instance: 957c1510-1320-4492-9052-a58ff540319d] Received unexpected event network-vif-plugged-e679cfd9-123f-4b88-8fd2-397e4c8616cc for instance with vm_state active and task_state deleting.
Aug 09 02:42:13 user nova-compute[72351]: DEBUG nova.compute.manager [req-4d907b5d-f2ca-4105-9a69-ff9d8c541527 req-29ed8712-1bba-43c9-827d-bdb71b2462cb service nova] [instance: 957c1510-1320-4492-9052-a58ff540319d] Received event network-vif-deleted-e679cfd9-123f-4b88-8fd2-397e4c8616cc {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:42:13 user nova-compute[72351]: INFO nova.compute.manager [None req-b00b1330-04aa-4d3f-93ba-1be0780ea8cc tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: 957c1510-1320-4492-9052-a58ff540319d] Took 0.70 seconds to detach 1 volumes for instance.
Aug 09 02:42:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:42:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b00b1330-04aa-4d3f-93ba-1be0780ea8cc tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:42:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b00b1330-04aa-4d3f-93ba-1be0780ea8cc tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:42:13 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:42:13 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:42:13 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72351) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10461}}
Aug 09 02:42:14 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-b00b1330-04aa-4d3f-93ba-1be0780ea8cc tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:42:14 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-b00b1330-04aa-4d3f-93ba-1be0780ea8cc tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:42:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b00b1330-04aa-4d3f-93ba-1be0780ea8cc tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.262s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:42:14 user nova-compute[72351]: INFO nova.scheduler.client.report [None req-b00b1330-04aa-4d3f-93ba-1be0780ea8cc tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Deleted allocations for instance 957c1510-1320-4492-9052-a58ff540319d
Aug 09 02:42:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-b00b1330-04aa-4d3f-93ba-1be0780ea8cc tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "957c1510-1320-4492-9052-a58ff540319d" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.879s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:42:14 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:42:14 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:42:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:42:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:42:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:42:14 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Auditing locally available compute resources for user (node: user) {{(pid=72351) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Aug 09 02:42:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6de75cee-9bf1-4f9f-8fa3-977d9432b805/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:42:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6de75cee-9bf1-4f9f-8fa3-977d9432b805/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:42:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6de75cee-9bf1-4f9f-8fa3-977d9432b805/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:42:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6de75cee-9bf1-4f9f-8fa3-977d9432b805/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:42:15 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdb (vdb) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:42:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:42:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:42:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:42:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444/disk --force-share --output=json" returned: 0 in 0.148s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:42:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:42:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:42:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:42:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:42:16 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:42:16 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:42:16 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Hypervisor/Node resource view: name=user free_ram=8516MB free_disk=25.14734649658203GB free_vcpus=9 pci_devices=[{"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72351) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Aug 09 02:42:16 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:42:16 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:42:16 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance a7f3d076-864b-4b68-9207-214ce2c21444 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:42:16 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 6de75cee-9bf1-4f9f-8fa3-977d9432b805 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:42:16 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:42:16 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Total usable vcpus: 12, total allocated vcpus: 3 {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Aug 09 02:42:16 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Final resource view: name=user phys_ram=16011MB used_ram=2048MB phys_disk=40GB used_disk=3GB total_vcpus=12 used_vcpus=3 pci_stats=[] {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Aug 09 02:42:16 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:42:16 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:42:16 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Compute_service record updated for user:user {{(pid=72351) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Aug 09 02:42:16 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.312s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:42:16 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:42:17 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:42:17 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Starting heal instance info cache {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9842}}
Aug 09 02:42:17 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Didn't find any instances for network info cache update. {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9928}}
Aug 09 02:42:17 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:42:18 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:42:18 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:42:21 user nova-compute[72351]: DEBUG nova.compute.manager [req-0def91aa-9bc9-46df-85d3-85bcd54a8310 req-483ce109-98b4-4dcd-95e9-aaa7104c40c8 service nova] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Received event network-changed-6295a96f-893b-4ae1-9452-18ef59c6e544 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:42:21 user nova-compute[72351]: DEBUG nova.compute.manager [req-0def91aa-9bc9-46df-85d3-85bcd54a8310 req-483ce109-98b4-4dcd-95e9-aaa7104c40c8 service nova] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Refreshing instance network info cache due to event network-changed-6295a96f-893b-4ae1-9452-18ef59c6e544. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:42:21 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-0def91aa-9bc9-46df-85d3-85bcd54a8310 req-483ce109-98b4-4dcd-95e9-aaa7104c40c8 service nova] Acquiring lock "refresh_cache-ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:42:21 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-0def91aa-9bc9-46df-85d3-85bcd54a8310 req-483ce109-98b4-4dcd-95e9-aaa7104c40c8 service nova] Acquired lock "refresh_cache-ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:42:21 user nova-compute[72351]: DEBUG nova.network.neutron [req-0def91aa-9bc9-46df-85d3-85bcd54a8310 req-483ce109-98b4-4dcd-95e9-aaa7104c40c8 service nova] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Refreshing network info cache for port 6295a96f-893b-4ae1-9452-18ef59c6e544 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:42:21 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-98d693a3-641c-4fdd-9b81-c6369fc70856 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Acquiring lock "ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:42:21 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-98d693a3-641c-4fdd-9b81-c6369fc70856 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:42:21 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-98d693a3-641c-4fdd-9b81-c6369fc70856 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Acquiring lock "ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:42:21 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-98d693a3-641c-4fdd-9b81-c6369fc70856 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:42:21 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-98d693a3-641c-4fdd-9b81-c6369fc70856 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:42:21 user nova-compute[72351]: INFO nova.compute.manager [None req-98d693a3-641c-4fdd-9b81-c6369fc70856 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Terminating instance
Aug 09 02:42:21 user nova-compute[72351]: DEBUG nova.compute.manager [None req-98d693a3-641c-4fdd-9b81-c6369fc70856 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Start destroying the instance on the hypervisor. {{(pid=72351) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3124}}
Aug 09 02:42:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:42:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:42:21 user nova-compute[72351]: DEBUG nova.network.neutron [req-0def91aa-9bc9-46df-85d3-85bcd54a8310 req-483ce109-98b4-4dcd-95e9-aaa7104c40c8 service nova] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Updated VIF entry in instance network info cache for port 6295a96f-893b-4ae1-9452-18ef59c6e544. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:42:21 user nova-compute[72351]: DEBUG nova.network.neutron [req-0def91aa-9bc9-46df-85d3-85bcd54a8310 req-483ce109-98b4-4dcd-95e9-aaa7104c40c8 service nova] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Updating instance_info_cache with network_info: [{"id": "6295a96f-893b-4ae1-9452-18ef59c6e544", "address": "fa:16:3e:f2:06:f7", "network": {"id": "47c43230-a244-4198-84e4-9ed98c005cd9", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1392403106-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.80", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c07a6b254414b41a673468238d3aa62", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6295a96f-89", "ovs_interfaceid": "6295a96f-893b-4ae1-9452-18ef59c6e544", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:42:21 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-0def91aa-9bc9-46df-85d3-85bcd54a8310 req-483ce109-98b4-4dcd-95e9-aaa7104c40c8 service nova] Releasing lock "refresh_cache-ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:42:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:42:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:42:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:42:22 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Instance destroyed successfully.
Aug 09 02:42:22 user nova-compute[72351]: DEBUG nova.objects.instance [None req-98d693a3-641c-4fdd-9b81-c6369fc70856 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lazy-loading 'resources' on Instance uuid ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:42:22 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-98d693a3-641c-4fdd-9b81-c6369fc70856 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:40:29Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachVolumeNegativeTest-server-2139317293',display_name='tempest-AttachVolumeNegativeTest-server-2139317293',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumenegativetest-server-2139317293',id=46,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBuIWu8zq6eGYrevOSXDRX6Ih5Nw4qjL0TAWTPVT1ZTrLHHnTDwcq06u6Q0X+/PGvqpZxqOnKLnPwr180t9o7ApGu+r9prljsGX7oJiP+ZAJgHUACauCzbsO9HoK5YKlvw==',key_name='tempest-keypair-1826810697',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:40:36Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='8c07a6b254414b41a673468238d3aa62',ramdisk_id='',reservation_id='r-v0g22x6o',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeNegativeTest-758771161',owner_user_name='tempest-AttachVolumeNegativeTest-758771161-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:40:37Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='17f7007adf02474e9b019cd8af099ff8',uuid=ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "6295a96f-893b-4ae1-9452-18ef59c6e544", "address": "fa:16:3e:f2:06:f7", "network": {"id": "47c43230-a244-4198-84e4-9ed98c005cd9", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1392403106-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c07a6b254414b41a673468238d3aa62", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6295a96f-89", "ovs_interfaceid": "6295a96f-893b-4ae1-9452-18ef59c6e544", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:42:22 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-98d693a3-641c-4fdd-9b81-c6369fc70856 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Converting VIF {"id": "6295a96f-893b-4ae1-9452-18ef59c6e544", "address": "fa:16:3e:f2:06:f7", "network": {"id": "47c43230-a244-4198-84e4-9ed98c005cd9", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1392403106-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c07a6b254414b41a673468238d3aa62", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6295a96f-89", "ovs_interfaceid": "6295a96f-893b-4ae1-9452-18ef59c6e544", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:42:22 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-98d693a3-641c-4fdd-9b81-c6369fc70856 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:f2:06:f7,bridge_name='br-int',has_traffic_filtering=True,id=6295a96f-893b-4ae1-9452-18ef59c6e544,network=Network(47c43230-a244-4198-84e4-9ed98c005cd9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6295a96f-89') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:42:22 user nova-compute[72351]: DEBUG os_vif [None req-98d693a3-641c-4fdd-9b81-c6369fc70856 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:f2:06:f7,bridge_name='br-int',has_traffic_filtering=True,id=6295a96f-893b-4ae1-9452-18ef59c6e544,network=Network(47c43230-a244-4198-84e4-9ed98c005cd9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6295a96f-89') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:42:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:42:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap6295a96f-89, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:42:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:42:22 user nova-compute[72351]: INFO os_vif [None req-98d693a3-641c-4fdd-9b81-c6369fc70856 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:f2:06:f7,bridge_name='br-int',has_traffic_filtering=True,id=6295a96f-893b-4ae1-9452-18ef59c6e544,network=Network(47c43230-a244-4198-84e4-9ed98c005cd9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6295a96f-89')
Aug 09 02:42:22 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-98d693a3-641c-4fdd-9b81-c6369fc70856 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Deleting instance files /opt/stack/data/nova/instances/ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f_del
Aug 09 02:42:22 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-98d693a3-641c-4fdd-9b81-c6369fc70856 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Deletion of /opt/stack/data/nova/instances/ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f_del complete
Aug 09 02:42:22 user nova-compute[72351]: INFO nova.compute.manager [None req-98d693a3-641c-4fdd-9b81-c6369fc70856 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Took 0.64 seconds to destroy the instance on the hypervisor.
Aug 09 02:42:22 user nova-compute[72351]: DEBUG oslo.service.loopingcall [None req-98d693a3-641c-4fdd-9b81-c6369fc70856 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72351) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Aug 09 02:42:22 user nova-compute[72351]: DEBUG nova.compute.manager [-] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Deallocating network for instance {{(pid=72351) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2263}}
Aug 09 02:42:22 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] deallocate_for_instance() {{(pid=72351) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1795}}
Aug 09 02:42:23 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:42:23 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:42:23 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Took 0.91 seconds to deallocate network for instance.
Aug 09 02:42:23 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-98d693a3-641c-4fdd-9b81-c6369fc70856 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:42:23 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-98d693a3-641c-4fdd-9b81-c6369fc70856 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:42:23 user nova-compute[72351]: DEBUG nova.compute.manager [req-6c685761-3ace-4233-be32-497e2f62f617 req-cf010f70-f70a-4641-892a-4b4c5033a511 service nova] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Received event network-vif-unplugged-6295a96f-893b-4ae1-9452-18ef59c6e544 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:42:23 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-6c685761-3ace-4233-be32-497e2f62f617 req-cf010f70-f70a-4641-892a-4b4c5033a511 service nova] Acquiring lock "ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:42:23 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-6c685761-3ace-4233-be32-497e2f62f617 req-cf010f70-f70a-4641-892a-4b4c5033a511 service nova] Lock "ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:42:23 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-6c685761-3ace-4233-be32-497e2f62f617 req-cf010f70-f70a-4641-892a-4b4c5033a511 service nova] Lock "ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:42:23 user nova-compute[72351]: DEBUG nova.compute.manager [req-6c685761-3ace-4233-be32-497e2f62f617 req-cf010f70-f70a-4641-892a-4b4c5033a511 service nova] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] No waiting events found dispatching network-vif-unplugged-6295a96f-893b-4ae1-9452-18ef59c6e544 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:42:23 user nova-compute[72351]: WARNING nova.compute.manager [req-6c685761-3ace-4233-be32-497e2f62f617 req-cf010f70-f70a-4641-892a-4b4c5033a511 service nova] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Received unexpected event network-vif-unplugged-6295a96f-893b-4ae1-9452-18ef59c6e544 for instance with vm_state deleted and task_state None.
Aug 09 02:42:23 user nova-compute[72351]: DEBUG nova.compute.manager [req-6c685761-3ace-4233-be32-497e2f62f617 req-cf010f70-f70a-4641-892a-4b4c5033a511 service nova] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Received event network-vif-plugged-6295a96f-893b-4ae1-9452-18ef59c6e544 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:42:23 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-6c685761-3ace-4233-be32-497e2f62f617 req-cf010f70-f70a-4641-892a-4b4c5033a511 service nova] Acquiring lock "ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:42:23 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-6c685761-3ace-4233-be32-497e2f62f617 req-cf010f70-f70a-4641-892a-4b4c5033a511 service nova] Lock "ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:42:23 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-6c685761-3ace-4233-be32-497e2f62f617 req-cf010f70-f70a-4641-892a-4b4c5033a511 service nova] Lock "ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:42:23 user nova-compute[72351]: DEBUG nova.compute.manager [req-6c685761-3ace-4233-be32-497e2f62f617 req-cf010f70-f70a-4641-892a-4b4c5033a511 service nova] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] No waiting events found dispatching network-vif-plugged-6295a96f-893b-4ae1-9452-18ef59c6e544 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:42:23 user nova-compute[72351]: WARNING nova.compute.manager [req-6c685761-3ace-4233-be32-497e2f62f617 req-cf010f70-f70a-4641-892a-4b4c5033a511 service nova] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Received unexpected event network-vif-plugged-6295a96f-893b-4ae1-9452-18ef59c6e544 for instance with vm_state deleted and task_state None.
Aug 09 02:42:23 user nova-compute[72351]: DEBUG nova.compute.manager [req-6c685761-3ace-4233-be32-497e2f62f617 req-cf010f70-f70a-4641-892a-4b4c5033a511 service nova] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Received event network-vif-deleted-6295a96f-893b-4ae1-9452-18ef59c6e544 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:42:23 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-98d693a3-641c-4fdd-9b81-c6369fc70856 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:42:23 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-98d693a3-641c-4fdd-9b81-c6369fc70856 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:42:23 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-98d693a3-641c-4fdd-9b81-c6369fc70856 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.158s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:42:23 user nova-compute[72351]: INFO nova.scheduler.client.report [None req-98d693a3-641c-4fdd-9b81-c6369fc70856 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Deleted allocations for instance ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f
Aug 09 02:42:23 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-98d693a3-641c-4fdd-9b81-c6369fc70856 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 1.886s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:42:23 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:42:26 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691548931.931324, 957c1510-1320-4492-9052-a58ff540319d => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:42:26 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 957c1510-1320-4492-9052-a58ff540319d] VM Stopped (Lifecycle Event)
Aug 09 02:42:26 user nova-compute[72351]: DEBUG nova.compute.manager [None req-dac815c9-069c-402a-acce-96718493c743 None None] [instance: 957c1510-1320-4492-9052-a58ff540319d] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:42:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:42:32 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:42:37 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691548942.0453486, ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:42:37 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] VM Stopped (Lifecycle Event)
Aug 09 02:42:37 user nova-compute[72351]: DEBUG nova.compute.manager [None req-4a64f8bf-6e17-406f-8a1f-dfee6776b419 None None] [instance: ad1a83c8-5c01-43bd-879b-dd00fcd3ae4f] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:42:37 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:42:42 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:42:47 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:42:52 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:42:57 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:42:57 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:42:57 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 02:42:57 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:42:57 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:42:57 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:43:02 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:43:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-cb1281e0-448f-4ee6-87fe-010bef42595d tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Acquiring lock "a7f3d076-864b-4b68-9207-214ce2c21444" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:43:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-cb1281e0-448f-4ee6-87fe-010bef42595d tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "a7f3d076-864b-4b68-9207-214ce2c21444" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:43:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-cb1281e0-448f-4ee6-87fe-010bef42595d tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Acquiring lock "a7f3d076-864b-4b68-9207-214ce2c21444-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:43:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-cb1281e0-448f-4ee6-87fe-010bef42595d tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "a7f3d076-864b-4b68-9207-214ce2c21444-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:43:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-cb1281e0-448f-4ee6-87fe-010bef42595d tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "a7f3d076-864b-4b68-9207-214ce2c21444-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:43:02 user nova-compute[72351]: INFO nova.compute.manager [None req-cb1281e0-448f-4ee6-87fe-010bef42595d tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Terminating instance
Aug 09 02:43:02 user nova-compute[72351]: DEBUG nova.compute.manager [None req-cb1281e0-448f-4ee6-87fe-010bef42595d tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Start destroying the instance on the hypervisor. {{(pid=72351) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3124}}
Aug 09 02:43:02 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:43:02 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:43:02 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:43:02 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:43:02 user nova-compute[72351]: DEBUG nova.compute.manager [req-d50052fe-30fb-4de4-902f-132e64662f74 req-8b36a49a-3f47-4309-a285-6c7dddca99c0 service nova] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Received event network-vif-unplugged-64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:43:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d50052fe-30fb-4de4-902f-132e64662f74 req-8b36a49a-3f47-4309-a285-6c7dddca99c0 service nova] Acquiring lock "a7f3d076-864b-4b68-9207-214ce2c21444-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:43:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d50052fe-30fb-4de4-902f-132e64662f74 req-8b36a49a-3f47-4309-a285-6c7dddca99c0 service nova] Lock "a7f3d076-864b-4b68-9207-214ce2c21444-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:43:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d50052fe-30fb-4de4-902f-132e64662f74 req-8b36a49a-3f47-4309-a285-6c7dddca99c0 service nova] Lock "a7f3d076-864b-4b68-9207-214ce2c21444-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:43:02 user nova-compute[72351]: DEBUG nova.compute.manager [req-d50052fe-30fb-4de4-902f-132e64662f74 req-8b36a49a-3f47-4309-a285-6c7dddca99c0 service nova] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] No waiting events found dispatching network-vif-unplugged-64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:43:02 user nova-compute[72351]: DEBUG nova.compute.manager [req-d50052fe-30fb-4de4-902f-132e64662f74 req-8b36a49a-3f47-4309-a285-6c7dddca99c0 service nova] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Received event network-vif-unplugged-64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:43:02 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:43:02 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:43:02 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:43:02 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:43:02 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Instance destroyed successfully.
Aug 09 02:43:02 user nova-compute[72351]: DEBUG nova.objects.instance [None req-cb1281e0-448f-4ee6-87fe-010bef42595d tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lazy-loading 'resources' on Instance uuid a7f3d076-864b-4b68-9207-214ce2c21444 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:43:02 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-cb1281e0-448f-4ee6-87fe-010bef42595d tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:22:33Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-578493690',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-578493690',id=20,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:22:41Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='321111f935dd4e8aab4a083206ea9a7e',ramdisk_id='',reservation_id='r-ovw0qy0n',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-2108329302',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:24:33Z,user_data=None,user_id='73d05c48be5640c28455e8ba8b05ef31',uuid=a7f3d076-864b-4b68-9207-214ce2c21444,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f", "address": "fa:16:3e:ed:9c:88", "network": {"id": "9a452414-d61f-4045-a6cc-9be93c892c4c", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "321111f935dd4e8aab4a083206ea9a7e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap64bd80e2-6c", "ovs_interfaceid": "64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:43:02 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-cb1281e0-448f-4ee6-87fe-010bef42595d tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Converting VIF {"id": "64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f", "address": "fa:16:3e:ed:9c:88", "network": {"id": "9a452414-d61f-4045-a6cc-9be93c892c4c", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1382817150-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "321111f935dd4e8aab4a083206ea9a7e", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap64bd80e2-6c", "ovs_interfaceid": "64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:43:02 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-cb1281e0-448f-4ee6-87fe-010bef42595d tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:ed:9c:88,bridge_name='br-int',has_traffic_filtering=True,id=64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f,network=Network(9a452414-d61f-4045-a6cc-9be93c892c4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap64bd80e2-6c') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:43:02 user nova-compute[72351]: DEBUG os_vif [None req-cb1281e0-448f-4ee6-87fe-010bef42595d tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:ed:9c:88,bridge_name='br-int',has_traffic_filtering=True,id=64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f,network=Network(9a452414-d61f-4045-a6cc-9be93c892c4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap64bd80e2-6c') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:43:02 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:43:02 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap64bd80e2-6c, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:43:02 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:43:02 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:43:02 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:43:02 user nova-compute[72351]: INFO os_vif [None req-cb1281e0-448f-4ee6-87fe-010bef42595d tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:ed:9c:88,bridge_name='br-int',has_traffic_filtering=True,id=64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f,network=Network(9a452414-d61f-4045-a6cc-9be93c892c4c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap64bd80e2-6c')
Aug 09 02:43:02 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-cb1281e0-448f-4ee6-87fe-010bef42595d tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Deleting instance files /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444_del
Aug 09 02:43:02 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-cb1281e0-448f-4ee6-87fe-010bef42595d tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Deletion of /opt/stack/data/nova/instances/a7f3d076-864b-4b68-9207-214ce2c21444_del complete
Aug 09 02:43:03 user nova-compute[72351]: INFO nova.compute.manager [None req-cb1281e0-448f-4ee6-87fe-010bef42595d tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Took 0.88 seconds to destroy the instance on the hypervisor.
Aug 09 02:43:03 user nova-compute[72351]: DEBUG oslo.service.loopingcall [None req-cb1281e0-448f-4ee6-87fe-010bef42595d tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72351) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Aug 09 02:43:03 user nova-compute[72351]: DEBUG nova.compute.manager [-] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Deallocating network for instance {{(pid=72351) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2263}}
Aug 09 02:43:03 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] deallocate_for_instance() {{(pid=72351) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1795}}
Aug 09 02:43:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:43:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:43:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:43:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:43:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:43:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:43:04 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:43:04 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Took 1.06 seconds to deallocate network for instance.
Aug 09 02:43:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-cb1281e0-448f-4ee6-87fe-010bef42595d tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:43:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-cb1281e0-448f-4ee6-87fe-010bef42595d tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:43:04 user nova-compute[72351]: DEBUG nova.compute.manager [req-f28be441-8957-4132-862f-ea18e3fb073e req-2330a979-eb60-49bd-ad25-4a34f8438796 service nova] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Received event network-vif-deleted-64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:43:04 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-cb1281e0-448f-4ee6-87fe-010bef42595d tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:43:04 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-cb1281e0-448f-4ee6-87fe-010bef42595d tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:43:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-cb1281e0-448f-4ee6-87fe-010bef42595d tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.142s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:43:04 user nova-compute[72351]: INFO nova.scheduler.client.report [None req-cb1281e0-448f-4ee6-87fe-010bef42595d tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Deleted allocations for instance a7f3d076-864b-4b68-9207-214ce2c21444
Aug 09 02:43:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-cb1281e0-448f-4ee6-87fe-010bef42595d tempest-ServerBootFromVolumeStableRescueTest-2108329302 tempest-ServerBootFromVolumeStableRescueTest-2108329302-project-member] Lock "a7f3d076-864b-4b68-9207-214ce2c21444" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.267s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:43:04 user nova-compute[72351]: DEBUG nova.compute.manager [req-547235e6-5d37-4aab-aa36-349ce3bf583c req-67161dbb-819c-4440-8414-404a9366745f service nova] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Received event network-vif-plugged-64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:43:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-547235e6-5d37-4aab-aa36-349ce3bf583c req-67161dbb-819c-4440-8414-404a9366745f service nova] Acquiring lock "a7f3d076-864b-4b68-9207-214ce2c21444-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:43:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-547235e6-5d37-4aab-aa36-349ce3bf583c req-67161dbb-819c-4440-8414-404a9366745f service nova] Lock "a7f3d076-864b-4b68-9207-214ce2c21444-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:43:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-547235e6-5d37-4aab-aa36-349ce3bf583c req-67161dbb-819c-4440-8414-404a9366745f service nova] Lock "a7f3d076-864b-4b68-9207-214ce2c21444-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:43:04 user nova-compute[72351]: DEBUG nova.compute.manager [req-547235e6-5d37-4aab-aa36-349ce3bf583c req-67161dbb-819c-4440-8414-404a9366745f service nova] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] No waiting events found dispatching network-vif-plugged-64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:43:04 user nova-compute[72351]: WARNING nova.compute.manager [req-547235e6-5d37-4aab-aa36-349ce3bf583c req-67161dbb-819c-4440-8414-404a9366745f service nova] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Received unexpected event network-vif-plugged-64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f for instance with vm_state deleted and task_state None.
Aug 09 02:43:04 user nova-compute[72351]: DEBUG nova.compute.manager [req-547235e6-5d37-4aab-aa36-349ce3bf583c req-67161dbb-819c-4440-8414-404a9366745f service nova] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Received event network-vif-plugged-64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:43:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-547235e6-5d37-4aab-aa36-349ce3bf583c req-67161dbb-819c-4440-8414-404a9366745f service nova] Acquiring lock "a7f3d076-864b-4b68-9207-214ce2c21444-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:43:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-547235e6-5d37-4aab-aa36-349ce3bf583c req-67161dbb-819c-4440-8414-404a9366745f service nova] Lock "a7f3d076-864b-4b68-9207-214ce2c21444-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:43:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-547235e6-5d37-4aab-aa36-349ce3bf583c req-67161dbb-819c-4440-8414-404a9366745f service nova] Lock "a7f3d076-864b-4b68-9207-214ce2c21444-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:43:04 user nova-compute[72351]: DEBUG nova.compute.manager [req-547235e6-5d37-4aab-aa36-349ce3bf583c req-67161dbb-819c-4440-8414-404a9366745f service nova] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] No waiting events found dispatching network-vif-plugged-64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:43:04 user nova-compute[72351]: WARNING nova.compute.manager [req-547235e6-5d37-4aab-aa36-349ce3bf583c req-67161dbb-819c-4440-8414-404a9366745f service nova] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Received unexpected event network-vif-plugged-64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f for instance with vm_state deleted and task_state None.
Aug 09 02:43:04 user nova-compute[72351]: DEBUG nova.compute.manager [req-547235e6-5d37-4aab-aa36-349ce3bf583c req-67161dbb-819c-4440-8414-404a9366745f service nova] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Received event network-vif-plugged-64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:43:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-547235e6-5d37-4aab-aa36-349ce3bf583c req-67161dbb-819c-4440-8414-404a9366745f service nova] Acquiring lock "a7f3d076-864b-4b68-9207-214ce2c21444-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:43:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-547235e6-5d37-4aab-aa36-349ce3bf583c req-67161dbb-819c-4440-8414-404a9366745f service nova] Lock "a7f3d076-864b-4b68-9207-214ce2c21444-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:43:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-547235e6-5d37-4aab-aa36-349ce3bf583c req-67161dbb-819c-4440-8414-404a9366745f service nova] Lock "a7f3d076-864b-4b68-9207-214ce2c21444-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:43:04 user nova-compute[72351]: DEBUG nova.compute.manager [req-547235e6-5d37-4aab-aa36-349ce3bf583c req-67161dbb-819c-4440-8414-404a9366745f service nova] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] No waiting events found dispatching network-vif-plugged-64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:43:04 user nova-compute[72351]: WARNING nova.compute.manager [req-547235e6-5d37-4aab-aa36-349ce3bf583c req-67161dbb-819c-4440-8414-404a9366745f service nova] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Received unexpected event network-vif-plugged-64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f for instance with vm_state deleted and task_state None.
Aug 09 02:43:04 user nova-compute[72351]: DEBUG nova.compute.manager [req-547235e6-5d37-4aab-aa36-349ce3bf583c req-67161dbb-819c-4440-8414-404a9366745f service nova] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Received event network-vif-unplugged-64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:43:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-547235e6-5d37-4aab-aa36-349ce3bf583c req-67161dbb-819c-4440-8414-404a9366745f service nova] Acquiring lock "a7f3d076-864b-4b68-9207-214ce2c21444-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:43:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-547235e6-5d37-4aab-aa36-349ce3bf583c req-67161dbb-819c-4440-8414-404a9366745f service nova] Lock "a7f3d076-864b-4b68-9207-214ce2c21444-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:43:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-547235e6-5d37-4aab-aa36-349ce3bf583c req-67161dbb-819c-4440-8414-404a9366745f service nova] Lock "a7f3d076-864b-4b68-9207-214ce2c21444-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:43:04 user nova-compute[72351]: DEBUG nova.compute.manager [req-547235e6-5d37-4aab-aa36-349ce3bf583c req-67161dbb-819c-4440-8414-404a9366745f service nova] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] No waiting events found dispatching network-vif-unplugged-64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:43:04 user nova-compute[72351]: WARNING nova.compute.manager [req-547235e6-5d37-4aab-aa36-349ce3bf583c req-67161dbb-819c-4440-8414-404a9366745f service nova] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Received unexpected event network-vif-unplugged-64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f for instance with vm_state deleted and task_state None.
Aug 09 02:43:04 user nova-compute[72351]: DEBUG nova.compute.manager [req-547235e6-5d37-4aab-aa36-349ce3bf583c req-67161dbb-819c-4440-8414-404a9366745f service nova] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Received event network-vif-plugged-64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:43:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-547235e6-5d37-4aab-aa36-349ce3bf583c req-67161dbb-819c-4440-8414-404a9366745f service nova] Acquiring lock "a7f3d076-864b-4b68-9207-214ce2c21444-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:43:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-547235e6-5d37-4aab-aa36-349ce3bf583c req-67161dbb-819c-4440-8414-404a9366745f service nova] Lock "a7f3d076-864b-4b68-9207-214ce2c21444-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:43:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-547235e6-5d37-4aab-aa36-349ce3bf583c req-67161dbb-819c-4440-8414-404a9366745f service nova] Lock "a7f3d076-864b-4b68-9207-214ce2c21444-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:43:04 user nova-compute[72351]: DEBUG nova.compute.manager [req-547235e6-5d37-4aab-aa36-349ce3bf583c req-67161dbb-819c-4440-8414-404a9366745f service nova] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] No waiting events found dispatching network-vif-plugged-64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:43:04 user nova-compute[72351]: WARNING nova.compute.manager [req-547235e6-5d37-4aab-aa36-349ce3bf583c req-67161dbb-819c-4440-8414-404a9366745f service nova] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Received unexpected event network-vif-plugged-64bd80e2-6cf2-4b5f-8f06-ed12d7227b3f for instance with vm_state deleted and task_state None.
Aug 09 02:43:07 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:43:12 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:43:12 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._cleanup_expired_console_auth_tokens {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:43:12 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:43:12 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:43:12 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 02:43:12 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:43:12 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:43:12 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:43:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ad85ea60-47a0-4bfc-8a63-3403441a885f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Acquiring lock "6de75cee-9bf1-4f9f-8fa3-977d9432b805" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:43:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ad85ea60-47a0-4bfc-8a63-3403441a885f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "6de75cee-9bf1-4f9f-8fa3-977d9432b805" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:43:13 user nova-compute[72351]: INFO nova.compute.manager [None req-ad85ea60-47a0-4bfc-8a63-3403441a885f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Detaching volume af0b86fc-20d0-49f5-bf92-ee3c8c5783e4
Aug 09 02:43:13 user nova-compute[72351]: INFO nova.virt.block_device [None req-ad85ea60-47a0-4bfc-8a63-3403441a885f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Attempting to driver detach volume af0b86fc-20d0-49f5-bf92-ee3c8c5783e4 from mountpoint /dev/vdb
Aug 09 02:43:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-ad85ea60-47a0-4bfc-8a63-3403441a885f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Attempting to detach device vdb from instance 6de75cee-9bf1-4f9f-8fa3-977d9432b805 from the persistent domain config. {{(pid=72351) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2477}}
Aug 09 02:43:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-ad85ea60-47a0-4bfc-8a63-3403441a885f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 02:43:13 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:43:13 user nova-compute[72351]:   <source dev="/dev/sdb"/>
Aug 09 02:43:13 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:43:13 user nova-compute[72351]:   <serial>af0b86fc-20d0-49f5-bf92-ee3c8c5783e4</serial>
Aug 09 02:43:13 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Aug 09 02:43:13 user nova-compute[72351]: </disk>
Aug 09 02:43:13 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 02:43:13 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-ad85ea60-47a0-4bfc-8a63-3403441a885f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Successfully detached device vdb from instance 6de75cee-9bf1-4f9f-8fa3-977d9432b805 from the persistent domain config.
Aug 09 02:43:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-ad85ea60-47a0-4bfc-8a63-3403441a885f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 6de75cee-9bf1-4f9f-8fa3-977d9432b805 from the live domain config. {{(pid=72351) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2513}}
Aug 09 02:43:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-ad85ea60-47a0-4bfc-8a63-3403441a885f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 02:43:13 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:43:13 user nova-compute[72351]:   <source dev="/dev/sdb"/>
Aug 09 02:43:13 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:43:13 user nova-compute[72351]:   <serial>af0b86fc-20d0-49f5-bf92-ee3c8c5783e4</serial>
Aug 09 02:43:13 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Aug 09 02:43:13 user nova-compute[72351]: </disk>
Aug 09 02:43:13 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 02:43:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Received event <DeviceRemovedEvent: 1691548993.7178931, 6de75cee-9bf1-4f9f-8fa3-977d9432b805 => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=72351) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2360}}
Aug 09 02:43:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-ad85ea60-47a0-4bfc-8a63-3403441a885f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 6de75cee-9bf1-4f9f-8fa3-977d9432b805 {{(pid=72351) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2589}}
Aug 09 02:43:13 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-ad85ea60-47a0-4bfc-8a63-3403441a885f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Successfully detached device vdb from instance 6de75cee-9bf1-4f9f-8fa3-977d9432b805 from the live domain config.
Aug 09 02:43:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-ad85ea60-47a0-4bfc-8a63-3403441a885f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 02:43:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-ad85ea60-47a0-4bfc-8a63-3403441a885f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:af0b86fc-20d0-49f5-bf92-ee3c8c5783e4'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'WnmTZzkQ', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb'}, None), 'kwargs': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:43:13 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-ad85ea60-47a0-4bfc-8a63-3403441a885f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:43:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-ad85ea60-47a0-4bfc-8a63-3403441a885f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:43:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-ad85ea60-47a0-4bfc-8a63-3403441a885f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:43:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-ad85ea60-47a0-4bfc-8a63-3403441a885f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:af0b86fc-20d0-49f5-bf92-ee3c8c5783e4', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 02:43:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:43:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:43:13 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[9a2c2119-1224-489e-b2f4-76da8f688943]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:af0b86fc-20d0-49f5-bf92-ee3c8c5783e4\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:43:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:43:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:43:13 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a5cda2a8-0478-4942-ba27-6f229ffed764]: (4, ('tcp: [56] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:af0b86fc-20d0-49f5-bf92-ee3c8c5783e4 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:43:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-ad85ea60-47a0-4bfc-8a63-3403441a885f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [56] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:af0b86fc-20d0-49f5-bf92-ee3c8c5783e4 (non-flash)
Aug 09 02:43:13 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:43:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-ad85ea60-47a0-4bfc-8a63-3403441a885f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] iscsi session list stdout=tcp: [56] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:af0b86fc-20d0-49f5-bf92-ee3c8c5783e4 (non-flash)
Aug 09 02:43:13 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:43:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-ad85ea60-47a0-4bfc-8a63-3403441a885f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fcfdc1a4280>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:af0b86fc-20d0-49f5-bf92-ee3c8c5783e4'): ({'sdb'}, set())}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 02:43:13 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-ad85ea60-47a0-4bfc-8a63-3403441a885f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Removing single pathed devices sdb {{(pid=72351) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Aug 09 02:43:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:43:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:43:13 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[9232fd44-aa0d-4989-ac20-1d507e6240d6]: (4, ('path checker states:\nup                  1\n\npaths: 0\nbusy: False\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:43:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:43:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:43:13 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e6306350-9262-449c-9ba0-886e15141d1f]: (4, ('ok\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:43:13 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-ad85ea60-47a0-4bfc-8a63-3403441a885f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Flushing IO for device /dev/sdb {{(pid=72351) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Aug 09 02:43:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:43:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.014s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:43:13 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[95bf8056-7347-4f6d-9ece-9b31482daef8]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:43:13 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-ad85ea60-47a0-4bfc-8a63-3403441a885f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=72351) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Aug 09 02:43:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:43:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.043s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:43:13 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[9a13a68a-4467-4a86-a37c-79c4fbbdb527]: (4, ('1', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:43:13 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-ad85ea60-47a0-4bfc-8a63-3403441a885f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Aug 09 02:43:13 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-ad85ea60-47a0-4bfc-8a63-3403441a885f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] SCSI volumes sdb have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Aug 09 02:43:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-ad85ea60-47a0-4bfc-8a63-3403441a885f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:af0b86fc-20d0-49f5-bf92-ee3c8c5783e4')] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 02:43:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:af0b86fc-20d0-49f5-bf92-ee3c8c5783e4 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:43:13 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:43:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:af0b86fc-20d0-49f5-bf92-ee3c8c5783e4 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:43:13 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[aa24a91f-ae39-481c-92b5-7b2a322025e2]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:43:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-ad85ea60-47a0-4bfc-8a63-3403441a885f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:43:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:af0b86fc-20d0-49f5-bf92-ee3c8c5783e4 -p 172.16.0.220:3260 --logout {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:43:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:af0b86fc-20d0-49f5-bf92-ee3c8c5783e4 -p 172.16.0.220:3260 --logout" returned: 0 in 0.048s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:43:13 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[be054c8d-f7d3-4906-89bf-37b4575ab99b]: (4, ('Logging out of session [sid: 56, target: iqn.2010-10.org.openstack:af0b86fc-20d0-49f5-bf92-ee3c8c5783e4, portal: 172.16.0.220,3260]\nLogout of [sid: 56, target: iqn.2010-10.org.openstack:af0b86fc-20d0-49f5-bf92-ee3c8c5783e4, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:43:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-ad85ea60-47a0-4bfc-8a63-3403441a885f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 56, target: iqn.2010-10.org.openstack:af0b86fc-20d0-49f5-bf92-ee3c8c5783e4, portal: 172.16.0.220,3260]
Aug 09 02:43:13 user nova-compute[72351]: Logout of [sid: 56, target: iqn.2010-10.org.openstack:af0b86fc-20d0-49f5-bf92-ee3c8c5783e4, portal: 172.16.0.220,3260] successful.
Aug 09 02:43:13 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:43:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:af0b86fc-20d0-49f5-bf92-ee3c8c5783e4 -p 172.16.0.220:3260 --op delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:43:13 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:af0b86fc-20d0-49f5-bf92-ee3c8c5783e4 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:43:13 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[2cbe0451-932a-4036-a9ce-3eb099d5e7fa]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:43:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-ad85ea60-47a0-4bfc-8a63-3403441a885f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:43:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-ad85ea60-47a0-4bfc-8a63-3403441a885f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.258s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:43:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-ad85ea60-47a0-4bfc-8a63-3403441a885f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] <== disconnect_volume: return (259ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:43:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-ad85ea60-47a0-4bfc-8a63-3403441a885f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 02:43:14 user nova-compute[72351]: DEBUG nova.objects.instance [None req-ad85ea60-47a0-4bfc-8a63-3403441a885f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lazy-loading 'flavor' on Instance uuid 6de75cee-9bf1-4f9f-8fa3-977d9432b805 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:43:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ad85ea60-47a0-4bfc-8a63-3403441a885f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "6de75cee-9bf1-4f9f-8fa3-977d9432b805" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.575s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:43:14 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:43:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:43:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:43:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:43:14 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Auditing locally available compute resources for user (node: user) {{(pid=72351) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Aug 09 02:43:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6de75cee-9bf1-4f9f-8fa3-977d9432b805/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:43:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6de75cee-9bf1-4f9f-8fa3-977d9432b805/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:43:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6de75cee-9bf1-4f9f-8fa3-977d9432b805/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:43:15 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6de75cee-9bf1-4f9f-8fa3-977d9432b805/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:43:15 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:43:15 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:43:15 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Hypervisor/Node resource view: name=user free_ram=8680MB free_disk=25.299293518066406GB free_vcpus=11 pci_devices=[{"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72351) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Aug 09 02:43:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:43:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:43:15 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 6de75cee-9bf1-4f9f-8fa3-977d9432b805 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:43:15 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Total usable vcpus: 12, total allocated vcpus: 1 {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Aug 09 02:43:15 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Final resource view: name=user phys_ram=16011MB used_ram=1024MB phys_disk=40GB used_disk=1GB total_vcpus=12 used_vcpus=1 pci_stats=[] {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Aug 09 02:43:15 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:43:15 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:43:15 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Compute_service record updated for user:user {{(pid=72351) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Aug 09 02:43:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.195s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:43:16 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:43:16 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Starting heal instance info cache {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9842}}
Aug 09 02:43:16 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Rebuilding the list of instances to heal {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9846}}
Aug 09 02:43:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "refresh_cache-6de75cee-9bf1-4f9f-8fa3-977d9432b805" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:43:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquired lock "refresh_cache-6de75cee-9bf1-4f9f-8fa3-977d9432b805" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:43:17 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Forcefully refreshing network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1996}}
Aug 09 02:43:17 user nova-compute[72351]: DEBUG nova.objects.instance [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lazy-loading 'info_cache' on Instance uuid 6de75cee-9bf1-4f9f-8fa3-977d9432b805 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:43:17 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Updating instance_info_cache with network_info: [{"id": "bacefb00-adc9-4430-b7e1-83c6200d6144", "address": "fa:16:3e:1a:c1:d6", "network": {"id": "47c43230-a244-4198-84e4-9ed98c005cd9", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1392403106-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.96", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c07a6b254414b41a673468238d3aa62", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbacefb00-ad", "ovs_interfaceid": "bacefb00-adc9-4430-b7e1-83c6200d6144", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:43:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Releasing lock "refresh_cache-6de75cee-9bf1-4f9f-8fa3-977d9432b805" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:43:17 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Updated the network info_cache for instance {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9913}}
Aug 09 02:43:17 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:43:17 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:43:17 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72351) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10461}}
Aug 09 02:43:17 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691548982.9098585, a7f3d076-864b-4b68-9207-214ce2c21444 => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:43:17 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] VM Stopped (Lifecycle Event)
Aug 09 02:43:17 user nova-compute[72351]: DEBUG nova.compute.manager [None req-e4aa6c3f-b516-4dbb-8487-8ab467da2bc3 None None] [instance: a7f3d076-864b-4b68-9207-214ce2c21444] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:43:17 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:43:18 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:43:18 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:43:18 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:43:18 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._cleanup_running_deleted_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:43:20 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:43:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:43:23 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._run_pending_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:43:23 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Cleaning up deleted instances {{(pid=72351) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11129}}
Aug 09 02:43:23 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] There are 0 instances to clean {{(pid=72351) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11138}}
Aug 09 02:43:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:43:32 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:43:33 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._cleanup_incomplete_migrations {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:43:33 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Cleaning up deleted instances with incomplete migration  {{(pid=72351) _cleanup_incomplete_migrations /opt/stack/nova/nova/compute/manager.py:11167}}
Aug 09 02:43:37 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:43:39 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Acquiring lock "c740d131-98a9-4b14-a75a-a33aa7c72b51" by "nova.compute.manager.ComputeManager.unshelve_instance.<locals>.do_unshelve_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:43:39 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "c740d131-98a9-4b14-a75a-a33aa7c72b51" acquired by "nova.compute.manager.ComputeManager.unshelve_instance.<locals>.do_unshelve_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:43:39 user nova-compute[72351]: INFO nova.compute.manager [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Unshelving
Aug 09 02:43:39 user nova-compute[72351]: INFO nova.virt.block_device [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Booting with volume 1bdf7f72-7504-4539-9f73-c9c66d0c2e1d at /dev/vdc
Aug 09 02:43:39 user nova-compute[72351]: DEBUG os_brick.utils [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:43:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:43:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:43:39 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[6e171d4a-215f-477b-b850-a14e71f6d5b7]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:a52269602ff4\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:43:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:43:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:43:39 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[71b17590-798b-4a74-ac71-b9ebc228e2e1]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:43:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:43:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.016s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:43:39 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[968db579-a962-41e0-b47f-9061d60300ed]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:43:39 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[6ec8c999-cf91-49ed-94bc-c01884db0fb2]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:43:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Running cmd (subprocess): nvme version {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:43:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] 'nvme version' failed. Not Retrying. {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:43:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.nvmeof [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] nvme not present on system {{(pid=72351) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Aug 09 02:43:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:43:39 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:43:39 user nova-compute[72351]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Aug 09 02:43:39 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[cb1aa909-39e6-4671-adc2-8a7b6f2a9f90]: (4, '') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:43:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72351) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Aug 09 02:43:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Aug 09 02:43:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] LIGHTOS: no hostnqn found. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Aug 09 02:43:39 user nova-compute[72351]: DEBUG os_brick.utils [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] <== get_connector_properties: return (118ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:a52269602ff4', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:43:39 user nova-compute[72351]: DEBUG nova.virt.block_device [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Updating existing volume attachment record: 7845b787-3a97-47e7-83be-c15dfd7a74a6 {{(pid=72351) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Aug 09 02:43:42 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:43:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:43:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:43:47 user nova-compute[72351]: DEBUG nova.objects.instance [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lazy-loading 'pci_requests' on Instance uuid c740d131-98a9-4b14-a75a-a33aa7c72b51 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:43:47 user nova-compute[72351]: DEBUG nova.objects.instance [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lazy-loading 'numa_topology' on Instance uuid c740d131-98a9-4b14-a75a-a33aa7c72b51 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:43:47 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72351) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Aug 09 02:43:47 user nova-compute[72351]: INFO nova.compute.claims [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Claim successful on node user
Aug 09 02:43:47 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:43:47 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:43:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.256s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:43:47 user nova-compute[72351]: INFO nova.network.neutron [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Updating port ef71c011-3c08-4927-bf2b-763a63fe8c6f with attributes {'binding:host_id': 'user', 'device_owner': 'compute:nova'}
Aug 09 02:43:47 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:43:47 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:43:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Acquiring lock "refresh_cache-c740d131-98a9-4b14-a75a-a33aa7c72b51" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:43:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Acquired lock "refresh_cache-c740d131-98a9-4b14-a75a-a33aa7c72b51" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:43:48 user nova-compute[72351]: DEBUG nova.network.neutron [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:43:48 user nova-compute[72351]: DEBUG nova.compute.manager [req-51e98dc6-2e3b-4cc5-9adc-84537fae1935 req-909c64c8-cc96-46cd-9ddb-a9e90382d7da service nova] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Received event network-changed-ef71c011-3c08-4927-bf2b-763a63fe8c6f {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:43:48 user nova-compute[72351]: DEBUG nova.compute.manager [req-51e98dc6-2e3b-4cc5-9adc-84537fae1935 req-909c64c8-cc96-46cd-9ddb-a9e90382d7da service nova] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Refreshing instance network info cache due to event network-changed-ef71c011-3c08-4927-bf2b-763a63fe8c6f. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:43:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-51e98dc6-2e3b-4cc5-9adc-84537fae1935 req-909c64c8-cc96-46cd-9ddb-a9e90382d7da service nova] Acquiring lock "refresh_cache-c740d131-98a9-4b14-a75a-a33aa7c72b51" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:43:48 user nova-compute[72351]: DEBUG nova.network.neutron [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Updating instance_info_cache with network_info: [{"id": "ef71c011-3c08-4927-bf2b-763a63fe8c6f", "address": "fa:16:3e:73:9f:90", "network": {"id": "da9be96c-ce20-4037-999c-230ca799fb06", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-661780305-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.69", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5ff300d27c9a4dd3b7ecd507b3627d98", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapef71c011-3c", "ovs_interfaceid": "ef71c011-3c08-4927-bf2b-763a63fe8c6f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:43:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Releasing lock "refresh_cache-c740d131-98a9-4b14-a75a-a33aa7c72b51" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:43:48 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Creating instance directory {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4717}}
Aug 09 02:43:48 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Creating image(s)
Aug 09 02:43:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Acquiring lock "/opt/stack/data/nova/instances/c740d131-98a9-4b14-a75a-a33aa7c72b51/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:43:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "/opt/stack/data/nova/instances/c740d131-98a9-4b14-a75a-a33aa7c72b51/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:43:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "/opt/stack/data/nova/instances/c740d131-98a9-4b14-a75a-a33aa7c72b51/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:43:48 user nova-compute[72351]: DEBUG nova.objects.instance [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lazy-loading 'trusted_certs' on Instance uuid c740d131-98a9-4b14-a75a-a33aa7c72b51 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:43:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-51e98dc6-2e3b-4cc5-9adc-84537fae1935 req-909c64c8-cc96-46cd-9ddb-a9e90382d7da service nova] Acquired lock "refresh_cache-c740d131-98a9-4b14-a75a-a33aa7c72b51" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:43:48 user nova-compute[72351]: DEBUG nova.network.neutron [req-51e98dc6-2e3b-4cc5-9adc-84537fae1935 req-909c64c8-cc96-46cd-9ddb-a9e90382d7da service nova] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Refreshing network info cache for port ef71c011-3c08-4927-bf2b-763a63fe8c6f {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:43:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Acquiring lock "6f51e218d942a068c64b64903ff80f9c3e6dee79" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:43:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "6f51e218d942a068c64b64903ff80f9c3e6dee79" acquired by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:43:49 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/6f51e218d942a068c64b64903ff80f9c3e6dee79.part --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:43:49 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/6f51e218d942a068c64b64903ff80f9c3e6dee79.part --force-share --output=json" returned: 0 in 0.139s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:43:49 user nova-compute[72351]: DEBUG nova.virt.images [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] 6195b054-3ba1-4222-8581-110cbefd136a was qcow2, converting to raw {{(pid=72351) fetch_to_raw /opt/stack/nova/nova/virt/images.py:165}}
Aug 09 02:43:49 user nova-compute[72351]: DEBUG nova.privsep.utils [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=72351) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Aug 09 02:43:49 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Running cmd (subprocess): qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/6f51e218d942a068c64b64903ff80f9c3e6dee79.part /opt/stack/data/nova/instances/_base/6f51e218d942a068c64b64903ff80f9c3e6dee79.converted {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:43:50 user nova-compute[72351]: DEBUG nova.network.neutron [req-51e98dc6-2e3b-4cc5-9adc-84537fae1935 req-909c64c8-cc96-46cd-9ddb-a9e90382d7da service nova] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Updated VIF entry in instance network info cache for port ef71c011-3c08-4927-bf2b-763a63fe8c6f. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:43:50 user nova-compute[72351]: DEBUG nova.network.neutron [req-51e98dc6-2e3b-4cc5-9adc-84537fae1935 req-909c64c8-cc96-46cd-9ddb-a9e90382d7da service nova] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Updating instance_info_cache with network_info: [{"id": "ef71c011-3c08-4927-bf2b-763a63fe8c6f", "address": "fa:16:3e:73:9f:90", "network": {"id": "da9be96c-ce20-4037-999c-230ca799fb06", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-661780305-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.69", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5ff300d27c9a4dd3b7ecd507b3627d98", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapef71c011-3c", "ovs_interfaceid": "ef71c011-3c08-4927-bf2b-763a63fe8c6f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:43:50 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-51e98dc6-2e3b-4cc5-9adc-84537fae1935 req-909c64c8-cc96-46cd-9ddb-a9e90382d7da service nova] Releasing lock "refresh_cache-c740d131-98a9-4b14-a75a-a33aa7c72b51" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:43:50 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] CMD "qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/6f51e218d942a068c64b64903ff80f9c3e6dee79.part /opt/stack/data/nova/instances/_base/6f51e218d942a068c64b64903ff80f9c3e6dee79.converted" returned: 0 in 0.433s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:43:50 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/6f51e218d942a068c64b64903ff80f9c3e6dee79.converted --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:43:50 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/6f51e218d942a068c64b64903ff80f9c3e6dee79.converted --force-share --output=json" returned: 0 in 0.135s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:43:50 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "6f51e218d942a068c64b64903ff80f9c3e6dee79" "released" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: held 1.893s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:43:50 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/6f51e218d942a068c64b64903ff80f9c3e6dee79 --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:43:50 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/6f51e218d942a068c64b64903ff80f9c3e6dee79 --force-share --output=json" returned: 0 in 0.130s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:43:50 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Acquiring lock "6f51e218d942a068c64b64903ff80f9c3e6dee79" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:43:50 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "6f51e218d942a068c64b64903ff80f9c3e6dee79" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:43:50 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/6f51e218d942a068c64b64903ff80f9c3e6dee79 --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:43:50 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/6f51e218d942a068c64b64903ff80f9c3e6dee79 --force-share --output=json" returned: 0 in 0.130s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:43:50 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/6f51e218d942a068c64b64903ff80f9c3e6dee79,backing_fmt=raw /opt/stack/data/nova/instances/c740d131-98a9-4b14-a75a-a33aa7c72b51/disk 1073741824 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:43:50 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/6f51e218d942a068c64b64903ff80f9c3e6dee79,backing_fmt=raw /opt/stack/data/nova/instances/c740d131-98a9-4b14-a75a-a33aa7c72b51/disk 1073741824" returned: 0 in 0.047s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:43:50 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "6f51e218d942a068c64b64903ff80f9c3e6dee79" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.184s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:43:50 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/6f51e218d942a068c64b64903ff80f9c3e6dee79 --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:43:51 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/6f51e218d942a068c64b64903ff80f9c3e6dee79 --force-share --output=json" returned: 0 in 0.128s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:43:51 user nova-compute[72351]: DEBUG nova.objects.instance [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lazy-loading 'migration_context' on Instance uuid c740d131-98a9-4b14-a75a-a33aa7c72b51 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:43:51 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Rebasing disk image.
Aug 09 02:43:51 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:43:51 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.132s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:43:51 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Running cmd (subprocess): qemu-img rebase -b /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd -F raw /opt/stack/data/nova/instances/c740d131-98a9-4b14-a75a-a33aa7c72b51/disk {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:43:52 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4997-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] CMD "qemu-img rebase -b /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd -F raw /opt/stack/data/nova/instances/c740d131-98a9-4b14-a75a-a33aa7c72b51/disk" returned: 0 in 2.103s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Created local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4851}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Ensure instance console log exists: /opt/stack/data/nova/instances/c740d131-98a9-4b14-a75a-a33aa7c72b51/console.log {{(pid=72351) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4603}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Start _get_guest_xml network_info=[{"id": "ef71c011-3c08-4927-bf2b-763a63fe8c6f", "address": "fa:16:3e:73:9f:90", "network": {"id": "da9be96c-ce20-4037-999c-230ca799fb06", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-661780305-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.69", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5ff300d27c9a4dd3b7ecd507b3627d98", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapef71c011-3c", "ovs_interfaceid": "ef71c011-3c08-4927-bf2b-763a63fe8c6f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vdc': {'bus': 'virtio', 'dev': 'vdc', 'type': 'disk'}}} image_meta=ImageMeta(checksum='6b33269ffd36dc736262aedde11e2c96',container_format='bare',created_at=2023-08-09T02:41:47Z,direct_url=<?>,disk_format='qcow2',id=6195b054-3ba1-4222-8581-110cbefd136a,min_disk=1,min_ram=0,name='tempest-AttachVolumeShelveTestJSON-server-1075911547-shelved',owner='5ff300d27c9a4dd3b7ecd507b3627d98',properties=ImageMetaProps,protected=<?>,size=52363264,status='active',tags=<?>,updated_at=2023-08-09T02:41:51Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'size': 0, 'device_type': 'disk', 'encryption_secret_uuid': None, 'guest_format': None, 'encryption_options': None, 'disk_bus': 'virtio', 'encrypted': False, 'encryption_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'image_id': '18ccda7f-31d3-4978-ab9d-4ed2f74eff94'}], 'ephemerals': [], 'block_device_mapping': [{'device_type': 'disk', 'guest_format': None, 'disk_bus': 'virtio', 'mount_device': '/dev/vdc', 'boot_index': None, 'attachment_id': '7845b787-3a97-47e7-83be-c15dfd7a74a6', 'delete_on_termination': False, 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqns': ['iqn.2010-10.org.openstack:1bdf7f72-7504-4539-9f73-c9c66d0c2e1d'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'qLeTFgdb', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'attached', 'instance': 'c740d131-98a9-4b14-a75a-a33aa7c72b51', 'attached_at': '', 'detached_at': '', 'volume_id': '1bdf7f72-7504-4539-9f73-c9c66d0c2e1d', 'serial': '1bdf7f72-7504-4539-9f73-c9c66d0c2e1d'}, 'volume_type': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:43:53 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:43:53 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:43:53 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='6b33269ffd36dc736262aedde11e2c96',container_format='bare',created_at=2023-08-09T02:41:47Z,direct_url=<?>,disk_format='qcow2',id=6195b054-3ba1-4222-8581-110cbefd136a,min_disk=1,min_ram=0,name='tempest-AttachVolumeShelveTestJSON-server-1075911547-shelved',owner='5ff300d27c9a4dd3b7ecd507b3627d98',properties=ImageMetaProps,protected=<?>,size=52363264,status='active',tags=<?>,updated_at=2023-08-09T02:41:51Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG nova.objects.instance [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lazy-loading 'vcpu_model' on Instance uuid c740d131-98a9-4b14-a75a-a33aa7c72b51 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:1bdf7f72-7504-4539-9f73-c9c66d0c2e1d'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'qLeTFgdb', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:43:53 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:43:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:43:53 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:43:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:1bdf7f72-7504-4539-9f73-c9c66d0c2e1d" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:1bdf7f72-7504-4539-9f73-c9c66d0c2e1d" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.003s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:43:53 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 02:43:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:1bdf7f72-7504-4539-9f73-c9c66d0c2e1d -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:1bdf7f72-7504-4539-9f73-c9c66d0c2e1d -p 172.16.0.220:3260" returned: 21 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[9eee90c2-9fb2-434f-95c6-69aa67515e0b]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Aug 09 02:43:53 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:1bdf7f72-7504-4539-9f73-c9c66d0c2e1d -p 172.16.0.220:3260 --interface default --op new {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:1bdf7f72-7504-4539-9f73-c9c66d0c2e1d -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[c354be50-3acb-486e-8b96-be88f6b922a5]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:1bdf7f72-7504-4539-9f73-c9c66d0c2e1d] added\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:1bdf7f72-7504-4539-9f73-c9c66d0c2e1d] added
Aug 09 02:43:53 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:1bdf7f72-7504-4539-9f73-c9c66d0c2e1d -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:1bdf7f72-7504-4539-9f73-c9c66d0c2e1d -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[c7cbdf98-b091-4bcb-b57f-0ecd17dca41f]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:1bdf7f72-7504-4539-9f73-c9c66d0c2e1d -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:1bdf7f72-7504-4539-9f73-c9c66d0c2e1d -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.026s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ba9142c4-7e1e-4ac6-9100-08f2daeba82d]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:1bdf7f72-7504-4539-9f73-c9c66d0c2e1d -p 172.16.0.220:3260 --op update -n node.session.auth.username -v qLeTFgdb {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:1bdf7f72-7504-4539-9f73-c9c66d0c2e1d -p 172.16.0.220:3260 --op update -n node.session.auth.username -v qLeTFgdb" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[293d4ce4-97bd-49a8-a8dc-381c1e847d47]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'qLeTFgdb'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:1bdf7f72-7504-4539-9f73-c9c66d0c2e1d -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:1bdf7f72-7504-4539-9f73-c9c66d0c2e1d -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[6e3908a9-31a5-4d75-b8a6-cab40e862b15]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 21 in 0.015s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[8f38216e-2963-49c8-af1a-679a870d2b6e]: (4, ('', 'iscsiadm: No active sessions.\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] iscsiadm ('-m', 'session'): stdout= stderr=iscsiadm: No active sessions.
Aug 09 02:43:53 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] iscsi session list stdout= stderr=iscsiadm: No active sessions.
Aug 09 02:43:53 user nova-compute[72351]:  {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:43:53 user nova-compute[72351]: WARNING os_brick.initiator.connectors.iscsi [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] iscsiadm stderr output when getting sessions: iscsiadm: No active sessions.
Aug 09 02:43:53 user nova-compute[72351]: 
Aug 09 02:43:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:1bdf7f72-7504-4539-9f73-c9c66d0c2e1d -p 172.16.0.220:3260 --login {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:1bdf7f72-7504-4539-9f73-c9c66d0c2e1d -p 172.16.0.220:3260 --login" returned: 0 in 0.047s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[51b4c41a-f2a3-478e-9edd-ac7d9c51a7c4]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:1bdf7f72-7504-4539-9f73-c9c66d0c2e1d, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:1bdf7f72-7504-4539-9f73-c9c66d0c2e1d, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:1bdf7f72-7504-4539-9f73-c9c66d0c2e1d, portal: 172.16.0.220,3260]
Aug 09 02:43:53 user nova-compute[72351]: Login to [iface: default, target: iqn.2010-10.org.openstack:1bdf7f72-7504-4539-9f73-c9c66d0c2e1d, portal: 172.16.0.220,3260] successful.
Aug 09 02:43:53 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:1bdf7f72-7504-4539-9f73-c9c66d0c2e1d -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:1bdf7f72-7504-4539-9f73-c9c66d0c2e1d -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[21f283f7-0864-430a-bb16-ef969fa9e8b1]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ad043e88-a842-4c6e-8ce9-f22e853863fe]: (4, ('tcp: [57] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:1bdf7f72-7504-4539-9f73-c9c66d0c2e1d (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [57] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:1bdf7f72-7504-4539-9f73-c9c66d0c2e1d (non-flash)
Aug 09 02:43:53 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] iscsi session list stdout=tcp: [57] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:1bdf7f72-7504-4539-9f73-c9c66d0c2e1d (non-flash)
Aug 09 02:43:53 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:1bdf7f72-7504-4539-9f73-c9c66d0c2e1d" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.259s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] HCTL ('33', '-', '-', 0) found on session 57 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[eb7f9bba-d9d5-4378-b682-fb7db966a574]: (4, ('- - 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:43:53 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Searching for a device in session 57 and hctl ['33', '*', '*', 0] yield: None {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:43:54 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Searching for a device in session 57 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:43:54 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Connected to sdb using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'qLeTFgdb', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 02:43:54 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.290s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:43:54 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] <== connect_volume: return (1291ms) {'type': 'block', 'scsi_wwn': '23031633036663264', 'path': '/dev/sdb'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:43:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23031633036663264', 'path': '/dev/sdb'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 02:43:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,compute_id=1,config_drive='',created_at=2023-08-09T02:39:51Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-server-1075911547',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumeshelvetestjson-server-1075911547',id=45,image_ref='6195b054-3ba1-4222-8581-110cbefd136a',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name='tempest-keypair-1369286576',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:39:57Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=4,progress=0,project_id='5ff300d27c9a4dd3b7ecd507b3627d98',ramdisk_id='',reservation_id='r-gx6s8m4c',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeShelveTestJSON-1982056968',owner_user_name='tempest-AttachVolumeShelveTestJSON-1982056968-project-member',shelved_at='2023-08-09T02:41:50.820347',shelved_host='user',shelved_image_id='6195b054-3ba1-4222-8581-110cbefd136a'},tags=<?>,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:43:39Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='ad8fc03436a84bdfb13ba6abecd4f27b',uuid=c740d131-98a9-4b14-a75a-a33aa7c72b51,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='shelved_offloaded') vif={"id": "ef71c011-3c08-4927-bf2b-763a63fe8c6f", "address": "fa:16:3e:73:9f:90", "network": {"id": "da9be96c-ce20-4037-999c-230ca799fb06", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-661780305-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.69", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5ff300d27c9a4dd3b7ecd507b3627d98", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapef71c011-3c", "ovs_interfaceid": "ef71c011-3c08-4927-bf2b-763a63fe8c6f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:43:54 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Converting VIF {"id": "ef71c011-3c08-4927-bf2b-763a63fe8c6f", "address": "fa:16:3e:73:9f:90", "network": {"id": "da9be96c-ce20-4037-999c-230ca799fb06", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-661780305-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.69", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5ff300d27c9a4dd3b7ecd507b3627d98", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapef71c011-3c", "ovs_interfaceid": "ef71c011-3c08-4927-bf2b-763a63fe8c6f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:43:54 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:73:9f:90,bridge_name='br-int',has_traffic_filtering=True,id=ef71c011-3c08-4927-bf2b-763a63fe8c6f,network=Network(da9be96c-ce20-4037-999c-230ca799fb06),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapef71c011-3c') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:43:54 user nova-compute[72351]: DEBUG nova.objects.instance [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lazy-loading 'pci_devices' on Instance uuid c740d131-98a9-4b14-a75a-a33aa7c72b51 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:43:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:43:54 user nova-compute[72351]:   <uuid>c740d131-98a9-4b14-a75a-a33aa7c72b51</uuid>
Aug 09 02:43:54 user nova-compute[72351]:   <name>instance-0000002d</name>
Aug 09 02:43:54 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:43:54 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:43:54 user nova-compute[72351]:   <metadata>
Aug 09 02:43:54 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:43:54 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:43:54 user nova-compute[72351]:       <nova:name>tempest-AttachVolumeShelveTestJSON-server-1075911547</nova:name>
Aug 09 02:43:54 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:43:53</nova:creationTime>
Aug 09 02:43:54 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:43:54 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:43:54 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:43:54 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:43:54 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:43:54 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:43:54 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:43:54 user nova-compute[72351]:       <nova:owner>
Aug 09 02:43:54 user nova-compute[72351]:         <nova:user uuid="ad8fc03436a84bdfb13ba6abecd4f27b">tempest-AttachVolumeShelveTestJSON-1982056968-project-member</nova:user>
Aug 09 02:43:54 user nova-compute[72351]:         <nova:project uuid="5ff300d27c9a4dd3b7ecd507b3627d98">tempest-AttachVolumeShelveTestJSON-1982056968</nova:project>
Aug 09 02:43:54 user nova-compute[72351]:       </nova:owner>
Aug 09 02:43:54 user nova-compute[72351]:       <nova:root type="image" uuid="6195b054-3ba1-4222-8581-110cbefd136a"/>
Aug 09 02:43:54 user nova-compute[72351]:       <nova:ports>
Aug 09 02:43:54 user nova-compute[72351]:         <nova:port uuid="ef71c011-3c08-4927-bf2b-763a63fe8c6f">
Aug 09 02:43:54 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.13" ipVersion="4"/>
Aug 09 02:43:54 user nova-compute[72351]:         </nova:port>
Aug 09 02:43:54 user nova-compute[72351]:       </nova:ports>
Aug 09 02:43:54 user nova-compute[72351]:     </nova:instance>
Aug 09 02:43:54 user nova-compute[72351]:   </metadata>
Aug 09 02:43:54 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:43:54 user nova-compute[72351]:     <system>
Aug 09 02:43:54 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:43:54 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:43:54 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:43:54 user nova-compute[72351]:       <entry name="serial">c740d131-98a9-4b14-a75a-a33aa7c72b51</entry>
Aug 09 02:43:54 user nova-compute[72351]:       <entry name="uuid">c740d131-98a9-4b14-a75a-a33aa7c72b51</entry>
Aug 09 02:43:54 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:43:54 user nova-compute[72351]:     </system>
Aug 09 02:43:54 user nova-compute[72351]:   </sysinfo>
Aug 09 02:43:54 user nova-compute[72351]:   <os>
Aug 09 02:43:54 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:43:54 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:43:54 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:43:54 user nova-compute[72351]:   </os>
Aug 09 02:43:54 user nova-compute[72351]:   <features>
Aug 09 02:43:54 user nova-compute[72351]:     <acpi/>
Aug 09 02:43:54 user nova-compute[72351]:     <apic/>
Aug 09 02:43:54 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:43:54 user nova-compute[72351]:   </features>
Aug 09 02:43:54 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:43:54 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:43:54 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:43:54 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:43:54 user nova-compute[72351]:   </clock>
Aug 09 02:43:54 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:43:54 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:43:54 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:43:54 user nova-compute[72351]:   </cpu>
Aug 09 02:43:54 user nova-compute[72351]:   <devices>
Aug 09 02:43:54 user nova-compute[72351]:     <disk type="file" device="disk">
Aug 09 02:43:54 user nova-compute[72351]:       <driver name="qemu" type="qcow2" cache="none"/>
Aug 09 02:43:54 user nova-compute[72351]:       <source file="/opt/stack/data/nova/instances/c740d131-98a9-4b14-a75a-a33aa7c72b51/disk"/>
Aug 09 02:43:54 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:43:54 user nova-compute[72351]:     </disk>
Aug 09 02:43:54 user nova-compute[72351]:     <disk type="block" device="disk">
Aug 09 02:43:54 user nova-compute[72351]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:43:54 user nova-compute[72351]:       <source dev="/dev/sdb"/>
Aug 09 02:43:54 user nova-compute[72351]:       <target dev="vdc" bus="virtio"/>
Aug 09 02:43:54 user nova-compute[72351]:       <serial>1bdf7f72-7504-4539-9f73-c9c66d0c2e1d</serial>
Aug 09 02:43:54 user nova-compute[72351]:     </disk>
Aug 09 02:43:54 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:43:54 user nova-compute[72351]:       <mac address="fa:16:3e:73:9f:90"/>
Aug 09 02:43:54 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:43:54 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:43:54 user nova-compute[72351]:       <target dev="tapef71c011-3c"/>
Aug 09 02:43:54 user nova-compute[72351]:     </interface>
Aug 09 02:43:54 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:43:54 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/c740d131-98a9-4b14-a75a-a33aa7c72b51/console.log" append="off"/>
Aug 09 02:43:54 user nova-compute[72351]:     </serial>
Aug 09 02:43:54 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:43:54 user nova-compute[72351]:     <video>
Aug 09 02:43:54 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:43:54 user nova-compute[72351]:     </video>
Aug 09 02:43:54 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:43:54 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:43:54 user nova-compute[72351]:     </rng>
Aug 09 02:43:54 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:43:54 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:43:54 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:43:54 user nova-compute[72351]:     </memballoon>
Aug 09 02:43:54 user nova-compute[72351]:   </devices>
Aug 09 02:43:54 user nova-compute[72351]: </domain>
Aug 09 02:43:54 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:43:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,compute_id=1,config_drive='',created_at=2023-08-09T02:39:51Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-server-1075911547',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumeshelvetestjson-server-1075911547',id=45,image_ref='6195b054-3ba1-4222-8581-110cbefd136a',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name='tempest-keypair-1369286576',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:39:57Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=4,progress=0,project_id='5ff300d27c9a4dd3b7ecd507b3627d98',ramdisk_id='',reservation_id='r-gx6s8m4c',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeShelveTestJSON-1982056968',owner_user_name='tempest-AttachVolumeShelveTestJSON-1982056968-project-member',shelved_at='2023-08-09T02:41:50.820347',shelved_host='user',shelved_image_id='6195b054-3ba1-4222-8581-110cbefd136a'},tags=<?>,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:43:39Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='ad8fc03436a84bdfb13ba6abecd4f27b',uuid=c740d131-98a9-4b14-a75a-a33aa7c72b51,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='shelved_offloaded') vif={"id": "ef71c011-3c08-4927-bf2b-763a63fe8c6f", "address": "fa:16:3e:73:9f:90", "network": {"id": "da9be96c-ce20-4037-999c-230ca799fb06", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-661780305-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.69", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5ff300d27c9a4dd3b7ecd507b3627d98", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapef71c011-3c", "ovs_interfaceid": "ef71c011-3c08-4927-bf2b-763a63fe8c6f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:43:54 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Converting VIF {"id": "ef71c011-3c08-4927-bf2b-763a63fe8c6f", "address": "fa:16:3e:73:9f:90", "network": {"id": "da9be96c-ce20-4037-999c-230ca799fb06", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-661780305-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.69", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5ff300d27c9a4dd3b7ecd507b3627d98", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapef71c011-3c", "ovs_interfaceid": "ef71c011-3c08-4927-bf2b-763a63fe8c6f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:43:54 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:73:9f:90,bridge_name='br-int',has_traffic_filtering=True,id=ef71c011-3c08-4927-bf2b-763a63fe8c6f,network=Network(da9be96c-ce20-4037-999c-230ca799fb06),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapef71c011-3c') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:43:54 user nova-compute[72351]: DEBUG os_vif [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:73:9f:90,bridge_name='br-int',has_traffic_filtering=True,id=ef71c011-3c08-4927-bf2b-763a63fe8c6f,network=Network(da9be96c-ce20-4037-999c-230ca799fb06),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapef71c011-3c') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:43:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:43:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:43:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:43:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:43:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapef71c011-3c, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:43:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapef71c011-3c, col_values=(('external_ids', {'iface-id': 'ef71c011-3c08-4927-bf2b-763a63fe8c6f', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:73:9f:90', 'vm-uuid': 'c740d131-98a9-4b14-a75a-a33aa7c72b51'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:43:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:43:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:43:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:43:54 user nova-compute[72351]: INFO os_vif [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:73:9f:90,bridge_name='br-int',has_traffic_filtering=True,id=ef71c011-3c08-4927-bf2b-763a63fe8c6f,network=Network(da9be96c-ce20-4037-999c-230ca799fb06),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapef71c011-3c')
Aug 09 02:43:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:43:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] No BDM found with device name vdc, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:43:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] No VIF found with MAC fa:16:3e:73:9f:90, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:43:55 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:43:56 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:43:56 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:43:56 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:43:56 user nova-compute[72351]: DEBUG nova.compute.manager [req-e2d749cc-5b11-45ca-af75-94c77f788f19 req-24f6ca55-36ba-46c2-93ee-c65cb2bffda6 service nova] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Received event network-vif-plugged-ef71c011-3c08-4927-bf2b-763a63fe8c6f {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:43:56 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-e2d749cc-5b11-45ca-af75-94c77f788f19 req-24f6ca55-36ba-46c2-93ee-c65cb2bffda6 service nova] Acquiring lock "c740d131-98a9-4b14-a75a-a33aa7c72b51-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:43:56 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-e2d749cc-5b11-45ca-af75-94c77f788f19 req-24f6ca55-36ba-46c2-93ee-c65cb2bffda6 service nova] Lock "c740d131-98a9-4b14-a75a-a33aa7c72b51-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:43:56 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-e2d749cc-5b11-45ca-af75-94c77f788f19 req-24f6ca55-36ba-46c2-93ee-c65cb2bffda6 service nova] Lock "c740d131-98a9-4b14-a75a-a33aa7c72b51-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:43:56 user nova-compute[72351]: DEBUG nova.compute.manager [req-e2d749cc-5b11-45ca-af75-94c77f788f19 req-24f6ca55-36ba-46c2-93ee-c65cb2bffda6 service nova] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] No waiting events found dispatching network-vif-plugged-ef71c011-3c08-4927-bf2b-763a63fe8c6f {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:43:56 user nova-compute[72351]: WARNING nova.compute.manager [req-e2d749cc-5b11-45ca-af75-94c77f788f19 req-24f6ca55-36ba-46c2-93ee-c65cb2bffda6 service nova] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Received unexpected event network-vif-plugged-ef71c011-3c08-4927-bf2b-763a63fe8c6f for instance with vm_state shelved_offloaded and task_state spawning.
Aug 09 02:43:56 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:43:56 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:43:58 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691549038.2883875, c740d131-98a9-4b14-a75a-a33aa7c72b51 => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:43:58 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] VM Resumed (Lifecycle Event)
Aug 09 02:43:58 user nova-compute[72351]: DEBUG nova.compute.manager [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:43:58 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Guest created on hypervisor {{(pid=72351) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4411}}
Aug 09 02:43:58 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Instance spawned successfully.
Aug 09 02:43:58 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:43:58 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: shelved_offloaded, current task_state: spawning, current DB power_state: 4, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:43:58 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:43:58 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691549038.2898579, c740d131-98a9-4b14-a75a-a33aa7c72b51 => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:43:58 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] VM Started (Lifecycle Event)
Aug 09 02:43:58 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:43:58 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Synchronizing instance power state after lifecycle event "Started"; current vm_state: shelved_offloaded, current task_state: spawning, current DB power_state: 4, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:43:58 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:43:58 user nova-compute[72351]: DEBUG nova.compute.manager [req-7da413af-f3f0-4b7c-8768-bd0acb1790f3 req-8595115b-b581-48e8-ac17-f5a2258be73e service nova] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Received event network-vif-plugged-ef71c011-3c08-4927-bf2b-763a63fe8c6f {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:43:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-7da413af-f3f0-4b7c-8768-bd0acb1790f3 req-8595115b-b581-48e8-ac17-f5a2258be73e service nova] Acquiring lock "c740d131-98a9-4b14-a75a-a33aa7c72b51-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:43:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-7da413af-f3f0-4b7c-8768-bd0acb1790f3 req-8595115b-b581-48e8-ac17-f5a2258be73e service nova] Lock "c740d131-98a9-4b14-a75a-a33aa7c72b51-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:43:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-7da413af-f3f0-4b7c-8768-bd0acb1790f3 req-8595115b-b581-48e8-ac17-f5a2258be73e service nova] Lock "c740d131-98a9-4b14-a75a-a33aa7c72b51-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:43:58 user nova-compute[72351]: DEBUG nova.compute.manager [req-7da413af-f3f0-4b7c-8768-bd0acb1790f3 req-8595115b-b581-48e8-ac17-f5a2258be73e service nova] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] No waiting events found dispatching network-vif-plugged-ef71c011-3c08-4927-bf2b-763a63fe8c6f {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:43:58 user nova-compute[72351]: WARNING nova.compute.manager [req-7da413af-f3f0-4b7c-8768-bd0acb1790f3 req-8595115b-b581-48e8-ac17-f5a2258be73e service nova] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Received unexpected event network-vif-plugged-ef71c011-3c08-4927-bf2b-763a63fe8c6f for instance with vm_state shelved_offloaded and task_state spawning.
Aug 09 02:43:58 user nova-compute[72351]: DEBUG nova.compute.manager [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:43:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-10f3a13b-3049-417d-ba32-d1974a004cdc tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "c740d131-98a9-4b14-a75a-a33aa7c72b51" "released" by "nova.compute.manager.ComputeManager.unshelve_instance.<locals>.do_unshelve_instance" :: held 19.241s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:43:58 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:43:59 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:44:01 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:44:01 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._sync_power_states {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:44:02 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Triggering sync for uuid 6de75cee-9bf1-4f9f-8fa3-977d9432b805 {{(pid=72351) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10252}}
Aug 09 02:44:02 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Triggering sync for uuid c740d131-98a9-4b14-a75a-a33aa7c72b51 {{(pid=72351) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10252}}
Aug 09 02:44:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "6de75cee-9bf1-4f9f-8fa3-977d9432b805" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:44:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "6de75cee-9bf1-4f9f-8fa3-977d9432b805" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:44:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "c740d131-98a9-4b14-a75a-a33aa7c72b51" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:44:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "c740d131-98a9-4b14-a75a-a33aa7c72b51" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:44:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "6de75cee-9bf1-4f9f-8fa3-977d9432b805" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.031s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:44:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "c740d131-98a9-4b14-a75a-a33aa7c72b51" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.033s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:44:03 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-d7a890d1-94c8-4783-be61-edcc77dab7a7 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Acquiring lock "6de75cee-9bf1-4f9f-8fa3-977d9432b805" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:44:03 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-d7a890d1-94c8-4783-be61-edcc77dab7a7 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "6de75cee-9bf1-4f9f-8fa3-977d9432b805" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:44:03 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-d7a890d1-94c8-4783-be61-edcc77dab7a7 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Acquiring lock "6de75cee-9bf1-4f9f-8fa3-977d9432b805-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:44:03 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-d7a890d1-94c8-4783-be61-edcc77dab7a7 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "6de75cee-9bf1-4f9f-8fa3-977d9432b805-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:44:03 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-d7a890d1-94c8-4783-be61-edcc77dab7a7 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "6de75cee-9bf1-4f9f-8fa3-977d9432b805-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:44:03 user nova-compute[72351]: INFO nova.compute.manager [None req-d7a890d1-94c8-4783-be61-edcc77dab7a7 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Terminating instance
Aug 09 02:44:03 user nova-compute[72351]: DEBUG nova.compute.manager [None req-d7a890d1-94c8-4783-be61-edcc77dab7a7 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Start destroying the instance on the hypervisor. {{(pid=72351) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3124}}
Aug 09 02:44:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:44:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:44:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:44:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:44:04 user nova-compute[72351]: DEBUG nova.compute.manager [req-ccdc6817-15ff-4471-8431-b2857496bb20 req-2017fdd4-2563-4dd4-87d8-35f2a7021c60 service nova] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Received event network-vif-unplugged-bacefb00-adc9-4430-b7e1-83c6200d6144 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:44:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ccdc6817-15ff-4471-8431-b2857496bb20 req-2017fdd4-2563-4dd4-87d8-35f2a7021c60 service nova] Acquiring lock "6de75cee-9bf1-4f9f-8fa3-977d9432b805-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:44:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ccdc6817-15ff-4471-8431-b2857496bb20 req-2017fdd4-2563-4dd4-87d8-35f2a7021c60 service nova] Lock "6de75cee-9bf1-4f9f-8fa3-977d9432b805-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:44:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ccdc6817-15ff-4471-8431-b2857496bb20 req-2017fdd4-2563-4dd4-87d8-35f2a7021c60 service nova] Lock "6de75cee-9bf1-4f9f-8fa3-977d9432b805-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:44:04 user nova-compute[72351]: DEBUG nova.compute.manager [req-ccdc6817-15ff-4471-8431-b2857496bb20 req-2017fdd4-2563-4dd4-87d8-35f2a7021c60 service nova] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] No waiting events found dispatching network-vif-unplugged-bacefb00-adc9-4430-b7e1-83c6200d6144 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:44:04 user nova-compute[72351]: DEBUG nova.compute.manager [req-ccdc6817-15ff-4471-8431-b2857496bb20 req-2017fdd4-2563-4dd4-87d8-35f2a7021c60 service nova] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Received event network-vif-unplugged-bacefb00-adc9-4430-b7e1-83c6200d6144 for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:44:04 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Instance destroyed successfully.
Aug 09 02:44:04 user nova-compute[72351]: DEBUG nova.objects.instance [None req-d7a890d1-94c8-4783-be61-edcc77dab7a7 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lazy-loading 'resources' on Instance uuid 6de75cee-9bf1-4f9f-8fa3-977d9432b805 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:44:04 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-d7a890d1-94c8-4783-be61-edcc77dab7a7 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:38:17Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachVolumeNegativeTest-server-328142793',display_name='tempest-AttachVolumeNegativeTest-server-328142793',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumenegativetest-server-328142793',id=44,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJdkoorTUzpV1GANS341gZEFGpd4RrZ1PVNEy4XX7359woZ/ReYCscWlr2MmLYK4hBSO0BqDIVirOq9+Vo0HPFngP7LodWctuluJtAdOKpAvkiD66c/ymOryEHKAKukJuA==',key_name='tempest-keypair-1555678363',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:38:24Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='8c07a6b254414b41a673468238d3aa62',ramdisk_id='',reservation_id='r-ommfsdzj',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeNegativeTest-758771161',owner_user_name='tempest-AttachVolumeNegativeTest-758771161-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:38:24Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='17f7007adf02474e9b019cd8af099ff8',uuid=6de75cee-9bf1-4f9f-8fa3-977d9432b805,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "bacefb00-adc9-4430-b7e1-83c6200d6144", "address": "fa:16:3e:1a:c1:d6", "network": {"id": "47c43230-a244-4198-84e4-9ed98c005cd9", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1392403106-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.96", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c07a6b254414b41a673468238d3aa62", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbacefb00-ad", "ovs_interfaceid": "bacefb00-adc9-4430-b7e1-83c6200d6144", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:44:04 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-d7a890d1-94c8-4783-be61-edcc77dab7a7 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Converting VIF {"id": "bacefb00-adc9-4430-b7e1-83c6200d6144", "address": "fa:16:3e:1a:c1:d6", "network": {"id": "47c43230-a244-4198-84e4-9ed98c005cd9", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1392403106-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.96", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c07a6b254414b41a673468238d3aa62", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbacefb00-ad", "ovs_interfaceid": "bacefb00-adc9-4430-b7e1-83c6200d6144", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:44:04 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-d7a890d1-94c8-4783-be61-edcc77dab7a7 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:1a:c1:d6,bridge_name='br-int',has_traffic_filtering=True,id=bacefb00-adc9-4430-b7e1-83c6200d6144,network=Network(47c43230-a244-4198-84e4-9ed98c005cd9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapbacefb00-ad') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:44:04 user nova-compute[72351]: DEBUG os_vif [None req-d7a890d1-94c8-4783-be61-edcc77dab7a7 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:1a:c1:d6,bridge_name='br-int',has_traffic_filtering=True,id=bacefb00-adc9-4430-b7e1-83c6200d6144,network=Network(47c43230-a244-4198-84e4-9ed98c005cd9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapbacefb00-ad') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:44:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:44:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapbacefb00-ad, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:44:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:44:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:44:04 user nova-compute[72351]: INFO os_vif [None req-d7a890d1-94c8-4783-be61-edcc77dab7a7 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:1a:c1:d6,bridge_name='br-int',has_traffic_filtering=True,id=bacefb00-adc9-4430-b7e1-83c6200d6144,network=Network(47c43230-a244-4198-84e4-9ed98c005cd9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapbacefb00-ad')
Aug 09 02:44:04 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-d7a890d1-94c8-4783-be61-edcc77dab7a7 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Deleting instance files /opt/stack/data/nova/instances/6de75cee-9bf1-4f9f-8fa3-977d9432b805_del
Aug 09 02:44:04 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-d7a890d1-94c8-4783-be61-edcc77dab7a7 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Deletion of /opt/stack/data/nova/instances/6de75cee-9bf1-4f9f-8fa3-977d9432b805_del complete
Aug 09 02:44:04 user nova-compute[72351]: INFO nova.compute.manager [None req-d7a890d1-94c8-4783-be61-edcc77dab7a7 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Took 0.86 seconds to destroy the instance on the hypervisor.
Aug 09 02:44:04 user nova-compute[72351]: DEBUG oslo.service.loopingcall [None req-d7a890d1-94c8-4783-be61-edcc77dab7a7 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72351) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Aug 09 02:44:04 user nova-compute[72351]: DEBUG nova.compute.manager [-] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Deallocating network for instance {{(pid=72351) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2263}}
Aug 09 02:44:04 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] deallocate_for_instance() {{(pid=72351) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1795}}
Aug 09 02:44:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:44:06 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:44:06 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Took 1.52 seconds to deallocate network for instance.
Aug 09 02:44:06 user nova-compute[72351]: DEBUG nova.compute.manager [req-e6360fa7-d4b4-4b71-a33e-b911c33d47eb req-12e5610b-3c54-4bfc-9a46-06615737be78 service nova] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Received event network-vif-plugged-bacefb00-adc9-4430-b7e1-83c6200d6144 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:44:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-e6360fa7-d4b4-4b71-a33e-b911c33d47eb req-12e5610b-3c54-4bfc-9a46-06615737be78 service nova] Acquiring lock "6de75cee-9bf1-4f9f-8fa3-977d9432b805-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:44:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-e6360fa7-d4b4-4b71-a33e-b911c33d47eb req-12e5610b-3c54-4bfc-9a46-06615737be78 service nova] Lock "6de75cee-9bf1-4f9f-8fa3-977d9432b805-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:44:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-e6360fa7-d4b4-4b71-a33e-b911c33d47eb req-12e5610b-3c54-4bfc-9a46-06615737be78 service nova] Lock "6de75cee-9bf1-4f9f-8fa3-977d9432b805-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:44:06 user nova-compute[72351]: DEBUG nova.compute.manager [req-e6360fa7-d4b4-4b71-a33e-b911c33d47eb req-12e5610b-3c54-4bfc-9a46-06615737be78 service nova] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] No waiting events found dispatching network-vif-plugged-bacefb00-adc9-4430-b7e1-83c6200d6144 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:44:06 user nova-compute[72351]: WARNING nova.compute.manager [req-e6360fa7-d4b4-4b71-a33e-b911c33d47eb req-12e5610b-3c54-4bfc-9a46-06615737be78 service nova] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Received unexpected event network-vif-plugged-bacefb00-adc9-4430-b7e1-83c6200d6144 for instance with vm_state active and task_state deleting.
Aug 09 02:44:06 user nova-compute[72351]: DEBUG nova.compute.manager [req-d449bc5d-c33a-4b63-93de-814cd5be9d8e req-5a6737a6-0f37-4ab1-b539-0ad82a6ee0c4 service nova] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Received event network-vif-deleted-bacefb00-adc9-4430-b7e1-83c6200d6144 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:44:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-d7a890d1-94c8-4783-be61-edcc77dab7a7 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:44:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-d7a890d1-94c8-4783-be61-edcc77dab7a7 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:44:06 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-d7a890d1-94c8-4783-be61-edcc77dab7a7 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:44:06 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-d7a890d1-94c8-4783-be61-edcc77dab7a7 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:44:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-d7a890d1-94c8-4783-be61-edcc77dab7a7 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.162s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:44:06 user nova-compute[72351]: INFO nova.scheduler.client.report [None req-d7a890d1-94c8-4783-be61-edcc77dab7a7 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Deleted allocations for instance 6de75cee-9bf1-4f9f-8fa3-977d9432b805
Aug 09 02:44:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-d7a890d1-94c8-4783-be61-edcc77dab7a7 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "6de75cee-9bf1-4f9f-8fa3-977d9432b805" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.754s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:44:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Acquiring lock "c6ae885f-dce2-4dc2-98d5-26139b8fb67e" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:44:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "c6ae885f-dce2-4dc2-98d5-26139b8fb67e" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:44:08 user nova-compute[72351]: DEBUG nova.compute.manager [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Starting instance... {{(pid=72351) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2406}}
Aug 09 02:44:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:44:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:44:08 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72351) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Aug 09 02:44:08 user nova-compute[72351]: INFO nova.compute.claims [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Claim successful on node user
Aug 09 02:44:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:44:09 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:44:09 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:44:09 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.570s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:44:09 user nova-compute[72351]: DEBUG nova.compute.manager [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Start building networks asynchronously for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2803}}
Aug 09 02:44:09 user nova-compute[72351]: DEBUG nova.compute.manager [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Allocating IP information in the background. {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1956}}
Aug 09 02:44:09 user nova-compute[72351]: DEBUG nova.network.neutron [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] allocate_for_instance() {{(pid=72351) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Aug 09 02:44:09 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Aug 09 02:44:09 user nova-compute[72351]: DEBUG nova.compute.manager [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Start building block device mappings for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2838}}
Aug 09 02:44:09 user nova-compute[72351]: DEBUG nova.policy [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'a2bc49dd654f4beca4b8eb5a49cd350d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec9861e0882e4065ab99ad3f74d6b25f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72351) authorize /opt/stack/nova/nova/policy.py:203}}
Aug 09 02:44:09 user nova-compute[72351]: DEBUG nova.compute.manager [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Start spawning the instance on the hypervisor. {{(pid=72351) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2612}}
Aug 09 02:44:09 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Creating instance directory {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4717}}
Aug 09 02:44:09 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Creating image(s)
Aug 09 02:44:09 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Acquiring lock "/opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:44:09 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "/opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:44:09 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "/opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:44:09 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:44:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:44:09 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.130s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:44:09 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Acquiring lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:44:09 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:44:09 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:44:09 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.134s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:44:09 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk 1073741824 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:44:09 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk 1073741824" returned: 0 in 0.048s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:44:09 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.189s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:44:09 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:44:09 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.134s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:44:09 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Checking if we can resize image /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk. size=1073741824 {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Aug 09 02:44:09 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:44:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk --force-share --output=json" returned: 0 in 0.156s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:44:10 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Cannot resize image /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk to a smaller size. {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Aug 09 02:44:10 user nova-compute[72351]: DEBUG nova.objects.instance [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lazy-loading 'migration_context' on Instance uuid c6ae885f-dce2-4dc2-98d5-26139b8fb67e {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:44:10 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Created local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4851}}
Aug 09 02:44:10 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Ensure instance console log exists: /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/console.log {{(pid=72351) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4603}}
Aug 09 02:44:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:44:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:44:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:44:10 user nova-compute[72351]: DEBUG nova.network.neutron [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Successfully created port: 2771d93d-fa0a-4e19-85ac-9d3f5df52782 {{(pid=72351) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Aug 09 02:44:10 user nova-compute[72351]: DEBUG nova.network.neutron [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Successfully updated port: 2771d93d-fa0a-4e19-85ac-9d3f5df52782 {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:44:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Acquiring lock "refresh_cache-c6ae885f-dce2-4dc2-98d5-26139b8fb67e" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:44:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Acquired lock "refresh_cache-c6ae885f-dce2-4dc2-98d5-26139b8fb67e" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:44:10 user nova-compute[72351]: DEBUG nova.network.neutron [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:44:11 user nova-compute[72351]: DEBUG nova.compute.manager [req-f9dde295-e779-4a8c-b395-8e599479086c req-f869863c-35a8-48f4-9971-8e162c37e350 service nova] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Received event network-changed-2771d93d-fa0a-4e19-85ac-9d3f5df52782 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:44:11 user nova-compute[72351]: DEBUG nova.compute.manager [req-f9dde295-e779-4a8c-b395-8e599479086c req-f869863c-35a8-48f4-9971-8e162c37e350 service nova] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Refreshing instance network info cache due to event network-changed-2771d93d-fa0a-4e19-85ac-9d3f5df52782. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:44:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-f9dde295-e779-4a8c-b395-8e599479086c req-f869863c-35a8-48f4-9971-8e162c37e350 service nova] Acquiring lock "refresh_cache-c6ae885f-dce2-4dc2-98d5-26139b8fb67e" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:44:11 user nova-compute[72351]: DEBUG nova.network.neutron [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:44:11 user nova-compute[72351]: DEBUG nova.network.neutron [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Updating instance_info_cache with network_info: [{"id": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "address": "fa:16:3e:dd:49:4d", "network": {"id": "dba57c3d-6efd-4057-8053-1c6a92bb24f8", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-568585773-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ec9861e0882e4065ab99ad3f74d6b25f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2771d93d-fa", "ovs_interfaceid": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:44:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Releasing lock "refresh_cache-c6ae885f-dce2-4dc2-98d5-26139b8fb67e" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:44:11 user nova-compute[72351]: DEBUG nova.compute.manager [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Instance network_info: |[{"id": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "address": "fa:16:3e:dd:49:4d", "network": {"id": "dba57c3d-6efd-4057-8053-1c6a92bb24f8", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-568585773-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ec9861e0882e4065ab99ad3f74d6b25f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2771d93d-fa", "ovs_interfaceid": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1971}}
Aug 09 02:44:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-f9dde295-e779-4a8c-b395-8e599479086c req-f869863c-35a8-48f4-9971-8e162c37e350 service nova] Acquired lock "refresh_cache-c6ae885f-dce2-4dc2-98d5-26139b8fb67e" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:44:11 user nova-compute[72351]: DEBUG nova.network.neutron [req-f9dde295-e779-4a8c-b395-8e599479086c req-f869863c-35a8-48f4-9971-8e162c37e350 service nova] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Refreshing network info cache for port 2771d93d-fa0a-4e19-85ac-9d3f5df52782 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:44:11 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Start _get_guest_xml network_info=[{"id": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "address": "fa:16:3e:dd:49:4d", "network": {"id": "dba57c3d-6efd-4057-8053-1c6a92bb24f8", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-568585773-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ec9861e0882e4065ab99ad3f74d6b25f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2771d93d-fa", "ovs_interfaceid": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'size': 0, 'device_type': 'disk', 'encryption_secret_uuid': None, 'guest_format': None, 'encryption_options': None, 'disk_bus': 'virtio', 'encrypted': False, 'encryption_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'image_id': '18ccda7f-31d3-4978-ab9d-4ed2f74eff94'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:44:11 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:44:11 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:44:11 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:44:11 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:44:11 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:44:11 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:44:11 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:44:11 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:44:11 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:44:11 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:44:11 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:44:11 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:44:11 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:44:11 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:44:11 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:44:11 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:44:11 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:44:11 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:44:11 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:44:08Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachVolumeTestJSON-server-218910074',display_name='tempest-AttachVolumeTestJSON-server-218910074',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumetestjson-server-218910074',id=47,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBIplDVAhYRmCgaFwiRMKOJ3OlhP3n+pFAzSDpMz7BP4WftIaiuBT1r87OWLI3l8JRBwh8sp/09wt0bUhVeTDk2tWONkdagBfHB77ctUB1RkuePwrxnVW0j07TkHZdA5xrw==',key_name='tempest-keypair-304755592',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='ec9861e0882e4065ab99ad3f74d6b25f',ramdisk_id='',reservation_id='r-2vl0h2f4',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeTestJSON-1725368857',owner_user_name='tempest-AttachVolumeTestJSON-1725368857-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:44:09Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='a2bc49dd654f4beca4b8eb5a49cd350d',uuid=c6ae885f-dce2-4dc2-98d5-26139b8fb67e,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "address": "fa:16:3e:dd:49:4d", "network": {"id": "dba57c3d-6efd-4057-8053-1c6a92bb24f8", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-568585773-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ec9861e0882e4065ab99ad3f74d6b25f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2771d93d-fa", "ovs_interfaceid": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:44:11 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Converting VIF {"id": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "address": "fa:16:3e:dd:49:4d", "network": {"id": "dba57c3d-6efd-4057-8053-1c6a92bb24f8", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-568585773-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ec9861e0882e4065ab99ad3f74d6b25f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2771d93d-fa", "ovs_interfaceid": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:44:11 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:dd:49:4d,bridge_name='br-int',has_traffic_filtering=True,id=2771d93d-fa0a-4e19-85ac-9d3f5df52782,network=Network(dba57c3d-6efd-4057-8053-1c6a92bb24f8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2771d93d-fa') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:44:11 user nova-compute[72351]: DEBUG nova.objects.instance [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lazy-loading 'pci_devices' on Instance uuid c6ae885f-dce2-4dc2-98d5-26139b8fb67e {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:44:11 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:44:11 user nova-compute[72351]:   <uuid>c6ae885f-dce2-4dc2-98d5-26139b8fb67e</uuid>
Aug 09 02:44:11 user nova-compute[72351]:   <name>instance-0000002f</name>
Aug 09 02:44:11 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:44:11 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:44:11 user nova-compute[72351]:   <metadata>
Aug 09 02:44:11 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:44:11 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:44:11 user nova-compute[72351]:       <nova:name>tempest-AttachVolumeTestJSON-server-218910074</nova:name>
Aug 09 02:44:11 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:44:11</nova:creationTime>
Aug 09 02:44:11 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:44:11 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:44:11 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:44:11 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:44:11 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:44:11 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:44:11 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:44:11 user nova-compute[72351]:       <nova:owner>
Aug 09 02:44:11 user nova-compute[72351]:         <nova:user uuid="a2bc49dd654f4beca4b8eb5a49cd350d">tempest-AttachVolumeTestJSON-1725368857-project-member</nova:user>
Aug 09 02:44:11 user nova-compute[72351]:         <nova:project uuid="ec9861e0882e4065ab99ad3f74d6b25f">tempest-AttachVolumeTestJSON-1725368857</nova:project>
Aug 09 02:44:11 user nova-compute[72351]:       </nova:owner>
Aug 09 02:44:11 user nova-compute[72351]:       <nova:root type="image" uuid="18ccda7f-31d3-4978-ab9d-4ed2f74eff94"/>
Aug 09 02:44:11 user nova-compute[72351]:       <nova:ports>
Aug 09 02:44:11 user nova-compute[72351]:         <nova:port uuid="2771d93d-fa0a-4e19-85ac-9d3f5df52782">
Aug 09 02:44:11 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.12" ipVersion="4"/>
Aug 09 02:44:11 user nova-compute[72351]:         </nova:port>
Aug 09 02:44:11 user nova-compute[72351]:       </nova:ports>
Aug 09 02:44:11 user nova-compute[72351]:     </nova:instance>
Aug 09 02:44:11 user nova-compute[72351]:   </metadata>
Aug 09 02:44:11 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:44:11 user nova-compute[72351]:     <system>
Aug 09 02:44:11 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:44:11 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:44:11 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:44:11 user nova-compute[72351]:       <entry name="serial">c6ae885f-dce2-4dc2-98d5-26139b8fb67e</entry>
Aug 09 02:44:11 user nova-compute[72351]:       <entry name="uuid">c6ae885f-dce2-4dc2-98d5-26139b8fb67e</entry>
Aug 09 02:44:11 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:44:11 user nova-compute[72351]:     </system>
Aug 09 02:44:11 user nova-compute[72351]:   </sysinfo>
Aug 09 02:44:11 user nova-compute[72351]:   <os>
Aug 09 02:44:11 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:44:11 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:44:11 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:44:11 user nova-compute[72351]:   </os>
Aug 09 02:44:11 user nova-compute[72351]:   <features>
Aug 09 02:44:11 user nova-compute[72351]:     <acpi/>
Aug 09 02:44:11 user nova-compute[72351]:     <apic/>
Aug 09 02:44:11 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:44:11 user nova-compute[72351]:   </features>
Aug 09 02:44:11 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:44:11 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:44:11 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:44:11 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:44:11 user nova-compute[72351]:   </clock>
Aug 09 02:44:11 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:44:11 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:44:11 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:44:11 user nova-compute[72351]:   </cpu>
Aug 09 02:44:11 user nova-compute[72351]:   <devices>
Aug 09 02:44:11 user nova-compute[72351]:     <disk type="file" device="disk">
Aug 09 02:44:11 user nova-compute[72351]:       <driver name="qemu" type="qcow2" cache="none"/>
Aug 09 02:44:11 user nova-compute[72351]:       <source file="/opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk"/>
Aug 09 02:44:11 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:44:11 user nova-compute[72351]:     </disk>
Aug 09 02:44:11 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:44:11 user nova-compute[72351]:       <mac address="fa:16:3e:dd:49:4d"/>
Aug 09 02:44:11 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:44:11 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:44:11 user nova-compute[72351]:       <target dev="tap2771d93d-fa"/>
Aug 09 02:44:11 user nova-compute[72351]:     </interface>
Aug 09 02:44:11 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:44:11 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/console.log" append="off"/>
Aug 09 02:44:11 user nova-compute[72351]:     </serial>
Aug 09 02:44:11 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:44:11 user nova-compute[72351]:     <video>
Aug 09 02:44:11 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:44:11 user nova-compute[72351]:     </video>
Aug 09 02:44:11 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:44:11 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:44:11 user nova-compute[72351]:     </rng>
Aug 09 02:44:11 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:44:11 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:44:11 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:44:11 user nova-compute[72351]:     </memballoon>
Aug 09 02:44:11 user nova-compute[72351]:   </devices>
Aug 09 02:44:11 user nova-compute[72351]: </domain>
Aug 09 02:44:11 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:44:11 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:44:08Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachVolumeTestJSON-server-218910074',display_name='tempest-AttachVolumeTestJSON-server-218910074',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumetestjson-server-218910074',id=47,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBIplDVAhYRmCgaFwiRMKOJ3OlhP3n+pFAzSDpMz7BP4WftIaiuBT1r87OWLI3l8JRBwh8sp/09wt0bUhVeTDk2tWONkdagBfHB77ctUB1RkuePwrxnVW0j07TkHZdA5xrw==',key_name='tempest-keypair-304755592',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='ec9861e0882e4065ab99ad3f74d6b25f',ramdisk_id='',reservation_id='r-2vl0h2f4',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeTestJSON-1725368857',owner_user_name='tempest-AttachVolumeTestJSON-1725368857-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:44:09Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='a2bc49dd654f4beca4b8eb5a49cd350d',uuid=c6ae885f-dce2-4dc2-98d5-26139b8fb67e,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "address": "fa:16:3e:dd:49:4d", "network": {"id": "dba57c3d-6efd-4057-8053-1c6a92bb24f8", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-568585773-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ec9861e0882e4065ab99ad3f74d6b25f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2771d93d-fa", "ovs_interfaceid": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:44:11 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Converting VIF {"id": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "address": "fa:16:3e:dd:49:4d", "network": {"id": "dba57c3d-6efd-4057-8053-1c6a92bb24f8", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-568585773-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ec9861e0882e4065ab99ad3f74d6b25f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2771d93d-fa", "ovs_interfaceid": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:44:11 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:dd:49:4d,bridge_name='br-int',has_traffic_filtering=True,id=2771d93d-fa0a-4e19-85ac-9d3f5df52782,network=Network(dba57c3d-6efd-4057-8053-1c6a92bb24f8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2771d93d-fa') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:44:11 user nova-compute[72351]: DEBUG os_vif [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:dd:49:4d,bridge_name='br-int',has_traffic_filtering=True,id=2771d93d-fa0a-4e19-85ac-9d3f5df52782,network=Network(dba57c3d-6efd-4057-8053-1c6a92bb24f8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2771d93d-fa') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:44:11 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:44:11 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:44:11 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:44:11 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:44:11 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap2771d93d-fa, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:44:11 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap2771d93d-fa, col_values=(('external_ids', {'iface-id': '2771d93d-fa0a-4e19-85ac-9d3f5df52782', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:dd:49:4d', 'vm-uuid': 'c6ae885f-dce2-4dc2-98d5-26139b8fb67e'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:44:11 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:44:11 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:44:11 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:44:11 user nova-compute[72351]: INFO os_vif [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:dd:49:4d,bridge_name='br-int',has_traffic_filtering=True,id=2771d93d-fa0a-4e19-85ac-9d3f5df52782,network=Network(dba57c3d-6efd-4057-8053-1c6a92bb24f8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2771d93d-fa')
Aug 09 02:44:11 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:44:11 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] No VIF found with MAC fa:16:3e:dd:49:4d, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:44:11 user nova-compute[72351]: DEBUG nova.network.neutron [req-f9dde295-e779-4a8c-b395-8e599479086c req-f869863c-35a8-48f4-9971-8e162c37e350 service nova] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Updated VIF entry in instance network info cache for port 2771d93d-fa0a-4e19-85ac-9d3f5df52782. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:44:11 user nova-compute[72351]: DEBUG nova.network.neutron [req-f9dde295-e779-4a8c-b395-8e599479086c req-f869863c-35a8-48f4-9971-8e162c37e350 service nova] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Updating instance_info_cache with network_info: [{"id": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "address": "fa:16:3e:dd:49:4d", "network": {"id": "dba57c3d-6efd-4057-8053-1c6a92bb24f8", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-568585773-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ec9861e0882e4065ab99ad3f74d6b25f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2771d93d-fa", "ovs_interfaceid": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:44:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-f9dde295-e779-4a8c-b395-8e599479086c req-f869863c-35a8-48f4-9971-8e162c37e350 service nova] Releasing lock "refresh_cache-c6ae885f-dce2-4dc2-98d5-26139b8fb67e" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:44:12 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:44:12 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:44:12 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:44:12 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:44:12 user nova-compute[72351]: DEBUG nova.compute.manager [req-37965710-493c-451c-937d-4954e47d64e0 req-7a086aad-e99c-45a9-b251-61e32859516d service nova] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Received event network-vif-plugged-2771d93d-fa0a-4e19-85ac-9d3f5df52782 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:44:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-37965710-493c-451c-937d-4954e47d64e0 req-7a086aad-e99c-45a9-b251-61e32859516d service nova] Acquiring lock "c6ae885f-dce2-4dc2-98d5-26139b8fb67e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:44:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-37965710-493c-451c-937d-4954e47d64e0 req-7a086aad-e99c-45a9-b251-61e32859516d service nova] Lock "c6ae885f-dce2-4dc2-98d5-26139b8fb67e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:44:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-37965710-493c-451c-937d-4954e47d64e0 req-7a086aad-e99c-45a9-b251-61e32859516d service nova] Lock "c6ae885f-dce2-4dc2-98d5-26139b8fb67e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:44:12 user nova-compute[72351]: DEBUG nova.compute.manager [req-37965710-493c-451c-937d-4954e47d64e0 req-7a086aad-e99c-45a9-b251-61e32859516d service nova] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] No waiting events found dispatching network-vif-plugged-2771d93d-fa0a-4e19-85ac-9d3f5df52782 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:44:12 user nova-compute[72351]: WARNING nova.compute.manager [req-37965710-493c-451c-937d-4954e47d64e0 req-7a086aad-e99c-45a9-b251-61e32859516d service nova] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Received unexpected event network-vif-plugged-2771d93d-fa0a-4e19-85ac-9d3f5df52782 for instance with vm_state building and task_state spawning.
Aug 09 02:44:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:44:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:44:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:44:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:44:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:44:13 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:44:13 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:44:14 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691549054.8411767, c6ae885f-dce2-4dc2-98d5-26139b8fb67e => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:44:14 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] VM Resumed (Lifecycle Event)
Aug 09 02:44:14 user nova-compute[72351]: DEBUG nova.compute.manager [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:44:14 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Guest created on hypervisor {{(pid=72351) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4411}}
Aug 09 02:44:14 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Instance spawned successfully.
Aug 09 02:44:14 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:894}}
Aug 09 02:44:14 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:44:14 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:44:14 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Found default for hw_cdrom_bus of ide {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:44:14 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Found default for hw_disk_bus of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:44:14 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Found default for hw_input_bus of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:44:14 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Found default for hw_pointer_model of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:44:14 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Found default for hw_video_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:44:14 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Found default for hw_vif_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:44:14 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:44:14 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691549054.8445635, c6ae885f-dce2-4dc2-98d5-26139b8fb67e => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:44:14 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] VM Started (Lifecycle Event)
Aug 09 02:44:14 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:44:14 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:44:14 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:44:14 user nova-compute[72351]: INFO nova.compute.manager [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Took 5.46 seconds to spawn the instance on the hypervisor.
Aug 09 02:44:14 user nova-compute[72351]: DEBUG nova.compute.manager [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:44:15 user nova-compute[72351]: DEBUG nova.compute.manager [req-bf70fbfe-4116-4cee-815e-ffa41407abcb req-fd8ce6a5-6d86-4227-b4a5-152b338a1f2b service nova] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Received event network-vif-plugged-2771d93d-fa0a-4e19-85ac-9d3f5df52782 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:44:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-bf70fbfe-4116-4cee-815e-ffa41407abcb req-fd8ce6a5-6d86-4227-b4a5-152b338a1f2b service nova] Acquiring lock "c6ae885f-dce2-4dc2-98d5-26139b8fb67e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:44:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-bf70fbfe-4116-4cee-815e-ffa41407abcb req-fd8ce6a5-6d86-4227-b4a5-152b338a1f2b service nova] Lock "c6ae885f-dce2-4dc2-98d5-26139b8fb67e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:44:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-bf70fbfe-4116-4cee-815e-ffa41407abcb req-fd8ce6a5-6d86-4227-b4a5-152b338a1f2b service nova] Lock "c6ae885f-dce2-4dc2-98d5-26139b8fb67e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:44:15 user nova-compute[72351]: DEBUG nova.compute.manager [req-bf70fbfe-4116-4cee-815e-ffa41407abcb req-fd8ce6a5-6d86-4227-b4a5-152b338a1f2b service nova] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] No waiting events found dispatching network-vif-plugged-2771d93d-fa0a-4e19-85ac-9d3f5df52782 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:44:15 user nova-compute[72351]: WARNING nova.compute.manager [req-bf70fbfe-4116-4cee-815e-ffa41407abcb req-fd8ce6a5-6d86-4227-b4a5-152b338a1f2b service nova] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Received unexpected event network-vif-plugged-2771d93d-fa0a-4e19-85ac-9d3f5df52782 for instance with vm_state building and task_state spawning.
Aug 09 02:44:15 user nova-compute[72351]: INFO nova.compute.manager [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Took 6.34 seconds to build instance.
Aug 09 02:44:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6c01497e-1327-44d1-87d5-1b04df71907d tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "c6ae885f-dce2-4dc2-98d5-26139b8fb67e" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 6.443s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:44:15 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:44:15 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72351) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10461}}
Aug 09 02:44:16 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:44:16 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:44:16 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Starting heal instance info cache {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9842}}
Aug 09 02:44:16 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Rebuilding the list of instances to heal {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9846}}
Aug 09 02:44:16 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "refresh_cache-c740d131-98a9-4b14-a75a-a33aa7c72b51" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:44:16 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquired lock "refresh_cache-c740d131-98a9-4b14-a75a-a33aa7c72b51" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:44:16 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Forcefully refreshing network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1996}}
Aug 09 02:44:16 user nova-compute[72351]: DEBUG nova.objects.instance [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lazy-loading 'info_cache' on Instance uuid c740d131-98a9-4b14-a75a-a33aa7c72b51 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:44:17 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Updating instance_info_cache with network_info: [{"id": "ef71c011-3c08-4927-bf2b-763a63fe8c6f", "address": "fa:16:3e:73:9f:90", "network": {"id": "da9be96c-ce20-4037-999c-230ca799fb06", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-661780305-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.69", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5ff300d27c9a4dd3b7ecd507b3627d98", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapef71c011-3c", "ovs_interfaceid": "ef71c011-3c08-4927-bf2b-763a63fe8c6f", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:44:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Releasing lock "refresh_cache-c740d131-98a9-4b14-a75a-a33aa7c72b51" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:44:17 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Updated the network info_cache for instance {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9913}}
Aug 09 02:44:17 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:44:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:44:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:44:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:44:17 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Auditing locally available compute resources for user (node: user) {{(pid=72351) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Aug 09 02:44:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c740d131-98a9-4b14-a75a-a33aa7c72b51/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:44:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c740d131-98a9-4b14-a75a-a33aa7c72b51/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:44:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c740d131-98a9-4b14-a75a-a33aa7c72b51/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:44:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c740d131-98a9-4b14-a75a-a33aa7c72b51/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:44:17 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdb (vdc) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:44:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:44:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:44:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:44:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:44:18 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:44:18 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:44:18 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Hypervisor/Node resource view: name=user free_ram=8592MB free_disk=25.29901123046875GB free_vcpus=10 pci_devices=[{"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72351) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Aug 09 02:44:18 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:44:18 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:44:18 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance c740d131-98a9-4b14-a75a-a33aa7c72b51 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:44:18 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance c6ae885f-dce2-4dc2-98d5-26139b8fb67e actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:44:18 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Total usable vcpus: 12, total allocated vcpus: 2 {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Aug 09 02:44:18 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Final resource view: name=user phys_ram=16011MB used_ram=1536MB phys_disk=40GB used_disk=2GB total_vcpus=12 used_vcpus=2 pci_stats=[] {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Aug 09 02:44:18 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Refreshing inventories for resource provider 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:804}}
Aug 09 02:44:18 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Updating ProviderTree inventory for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 from _refresh_and_get_inventory using data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) _refresh_and_get_inventory /opt/stack/nova/nova/scheduler/client/report.py:768}}
Aug 09 02:44:18 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Updating inventory in ProviderTree for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 with inventory: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:176}}
Aug 09 02:44:18 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Refreshing aggregate associations for resource provider 8eae9b37-1236-4e6d-91a5-b650de27b009, aggregates: None {{(pid=72351) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:813}}
Aug 09 02:44:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:44:18 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Refreshing trait associations for resource provider 8eae9b37-1236-4e6d-91a5-b650de27b009, traits: COMPUTE_VIOMMU_MODEL_INTEL,COMPUTE_STORAGE_BUS_IDE,COMPUTE_GRAPHICS_MODEL_NONE,COMPUTE_NET_VIF_MODEL_E1000E,COMPUTE_IMAGE_TYPE_AKI,COMPUTE_VOLUME_ATTACH_WITH_TAG,COMPUTE_NET_VIF_MODEL_VMXNET3,COMPUTE_SOCKET_PCI_NUMA_AFFINITY,COMPUTE_ACCELERATORS,HW_CPU_X86_SSE41,COMPUTE_NET_ATTACH_INTERFACE_WITH_TAG,COMPUTE_GRAPHICS_MODEL_VGA,COMPUTE_GRAPHICS_MODEL_QXL,COMPUTE_VIOMMU_MODEL_AUTO,COMPUTE_SECURITY_UEFI_SECURE_BOOT,COMPUTE_NET_VIF_MODEL_E1000,COMPUTE_TRUSTED_CERTS,COMPUTE_VOLUME_MULTI_ATTACH,COMPUTE_STORAGE_BUS_FDC,COMPUTE_VOLUME_EXTEND,COMPUTE_GRAPHICS_MODEL_CIRRUS,COMPUTE_GRAPHICS_MODEL_VMVGA,COMPUTE_GRAPHICS_MODEL_BOCHS,COMPUTE_DEVICE_TAGGING,COMPUTE_STORAGE_BUS_SCSI,COMPUTE_STORAGE_BUS_VIRTIO,HW_CPU_X86_SSSE3,HW_CPU_X86_SSE2,COMPUTE_IMAGE_TYPE_AMI,COMPUTE_STORAGE_BUS_USB,COMPUTE_NET_VIF_MODEL_PCNET,COMPUTE_NET_VIF_MODEL_VIRTIO,COMPUTE_IMAGE_TYPE_ISO,HW_CPU_X86_SSE42,HW_CPU_X86_SSE,COMPUTE_IMAGE_TYPE_ARI,COMPUTE_IMAGE_TYPE_RAW,COMPUTE_IMAGE_TYPE_QCOW2,COMPUTE_NET_VIF_MODEL_NE2K_PCI,COMPUTE_NODE,HW_CPU_X86_MMX,COMPUTE_NET_VIF_MODEL_RTL8139,COMPUTE_GRAPHICS_MODEL_VIRTIO,COMPUTE_NET_VIF_MODEL_SPAPR_VLAN,COMPUTE_STORAGE_BUS_SATA,COMPUTE_NET_ATTACH_INTERFACE,COMPUTE_RESCUE_BFV {{(pid=72351) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:825}}
Aug 09 02:44:19 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:44:19 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:44:19 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Compute_service record updated for user:user {{(pid=72351) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Aug 09 02:44:19 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.506s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:44:19 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:44:19 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691549044.527675, 6de75cee-9bf1-4f9f-8fa3-977d9432b805 => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:44:19 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] VM Stopped (Lifecycle Event)
Aug 09 02:44:19 user nova-compute[72351]: DEBUG nova.compute.manager [None req-65782fda-38c7-48ee-9014-07565c168210 None None] [instance: 6de75cee-9bf1-4f9f-8fa3-977d9432b805] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:44:19 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:44:20 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:44:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:44:22 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:44:23 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:44:26 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:44:28 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:44:31 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:44:33 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:44:36 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:44:38 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:44:41 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:44:43 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:44:46 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:44:51 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:44:51 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:44:51 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5003 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 02:44:51 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:44:51 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:44:51 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:44:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:44:56 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:44:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Acquiring lock "e020e9e8-2372-40c9-ab34-6f65a529a995" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:44:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "e020e9e8-2372-40c9-ab34-6f65a529a995" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:44:57 user nova-compute[72351]: DEBUG nova.compute.manager [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Starting instance... {{(pid=72351) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2406}}
Aug 09 02:44:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:44:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:44:57 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72351) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Aug 09 02:44:57 user nova-compute[72351]: INFO nova.compute.claims [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Claim successful on node user
Aug 09 02:44:57 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:44:57 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:44:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.245s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:44:57 user nova-compute[72351]: DEBUG nova.compute.manager [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Start building networks asynchronously for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2803}}
Aug 09 02:44:57 user nova-compute[72351]: DEBUG nova.compute.manager [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Allocating IP information in the background. {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1956}}
Aug 09 02:44:57 user nova-compute[72351]: DEBUG nova.network.neutron [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] allocate_for_instance() {{(pid=72351) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Aug 09 02:44:57 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Aug 09 02:44:57 user nova-compute[72351]: DEBUG nova.compute.manager [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Start building block device mappings for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2838}}
Aug 09 02:44:57 user nova-compute[72351]: DEBUG nova.policy [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '17f7007adf02474e9b019cd8af099ff8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c07a6b254414b41a673468238d3aa62', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72351) authorize /opt/stack/nova/nova/policy.py:203}}
Aug 09 02:44:57 user nova-compute[72351]: DEBUG nova.compute.manager [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Start spawning the instance on the hypervisor. {{(pid=72351) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2612}}
Aug 09 02:44:57 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Creating instance directory {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4717}}
Aug 09 02:44:57 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Creating image(s)
Aug 09 02:44:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Acquiring lock "/opt/stack/data/nova/instances/e020e9e8-2372-40c9-ab34-6f65a529a995/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:44:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "/opt/stack/data/nova/instances/e020e9e8-2372-40c9-ab34-6f65a529a995/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:44:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "/opt/stack/data/nova/instances/e020e9e8-2372-40c9-ab34-6f65a529a995/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:44:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:44:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.143s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:44:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Acquiring lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:44:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:44:57 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:44:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.136s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:44:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/e020e9e8-2372-40c9-ab34-6f65a529a995/disk 1073741824 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:44:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/e020e9e8-2372-40c9-ab34-6f65a529a995/disk 1073741824" returned: 0 in 0.047s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:44:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.190s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:44:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:44:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.140s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:44:58 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Checking if we can resize image /opt/stack/data/nova/instances/e020e9e8-2372-40c9-ab34-6f65a529a995/disk. size=1073741824 {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Aug 09 02:44:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e020e9e8-2372-40c9-ab34-6f65a529a995/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:44:58 user nova-compute[72351]: DEBUG nova.network.neutron [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Successfully created port: 94c7c88f-9e5e-4e25-b7b8-59fb82e25f52 {{(pid=72351) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Aug 09 02:44:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e020e9e8-2372-40c9-ab34-6f65a529a995/disk --force-share --output=json" returned: 0 in 0.166s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:44:58 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Cannot resize image /opt/stack/data/nova/instances/e020e9e8-2372-40c9-ab34-6f65a529a995/disk to a smaller size. {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Aug 09 02:44:58 user nova-compute[72351]: DEBUG nova.objects.instance [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lazy-loading 'migration_context' on Instance uuid e020e9e8-2372-40c9-ab34-6f65a529a995 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:44:58 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Created local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4851}}
Aug 09 02:44:58 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Ensure instance console log exists: /opt/stack/data/nova/instances/e020e9e8-2372-40c9-ab34-6f65a529a995/console.log {{(pid=72351) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4603}}
Aug 09 02:44:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:44:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:44:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:44:59 user nova-compute[72351]: DEBUG nova.network.neutron [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Successfully updated port: 94c7c88f-9e5e-4e25-b7b8-59fb82e25f52 {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:44:59 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Acquiring lock "refresh_cache-e020e9e8-2372-40c9-ab34-6f65a529a995" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:44:59 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Acquired lock "refresh_cache-e020e9e8-2372-40c9-ab34-6f65a529a995" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:44:59 user nova-compute[72351]: DEBUG nova.network.neutron [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:44:59 user nova-compute[72351]: DEBUG nova.compute.manager [req-4e3c0062-8de8-4d82-8352-12f795efde4d req-e4684321-3cf5-4f4d-bec4-8b71d41c80c6 service nova] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Received event network-changed-94c7c88f-9e5e-4e25-b7b8-59fb82e25f52 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:44:59 user nova-compute[72351]: DEBUG nova.compute.manager [req-4e3c0062-8de8-4d82-8352-12f795efde4d req-e4684321-3cf5-4f4d-bec4-8b71d41c80c6 service nova] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Refreshing instance network info cache due to event network-changed-94c7c88f-9e5e-4e25-b7b8-59fb82e25f52. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:44:59 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-4e3c0062-8de8-4d82-8352-12f795efde4d req-e4684321-3cf5-4f4d-bec4-8b71d41c80c6 service nova] Acquiring lock "refresh_cache-e020e9e8-2372-40c9-ab34-6f65a529a995" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:44:59 user nova-compute[72351]: DEBUG nova.network.neutron [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:44:59 user nova-compute[72351]: DEBUG nova.network.neutron [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Updating instance_info_cache with network_info: [{"id": "94c7c88f-9e5e-4e25-b7b8-59fb82e25f52", "address": "fa:16:3e:c1:3e:66", "network": {"id": "47c43230-a244-4198-84e4-9ed98c005cd9", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1392403106-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c07a6b254414b41a673468238d3aa62", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap94c7c88f-9e", "ovs_interfaceid": "94c7c88f-9e5e-4e25-b7b8-59fb82e25f52", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:44:59 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Releasing lock "refresh_cache-e020e9e8-2372-40c9-ab34-6f65a529a995" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:44:59 user nova-compute[72351]: DEBUG nova.compute.manager [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Instance network_info: |[{"id": "94c7c88f-9e5e-4e25-b7b8-59fb82e25f52", "address": "fa:16:3e:c1:3e:66", "network": {"id": "47c43230-a244-4198-84e4-9ed98c005cd9", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1392403106-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c07a6b254414b41a673468238d3aa62", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap94c7c88f-9e", "ovs_interfaceid": "94c7c88f-9e5e-4e25-b7b8-59fb82e25f52", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1971}}
Aug 09 02:44:59 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-4e3c0062-8de8-4d82-8352-12f795efde4d req-e4684321-3cf5-4f4d-bec4-8b71d41c80c6 service nova] Acquired lock "refresh_cache-e020e9e8-2372-40c9-ab34-6f65a529a995" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:44:59 user nova-compute[72351]: DEBUG nova.network.neutron [req-4e3c0062-8de8-4d82-8352-12f795efde4d req-e4684321-3cf5-4f4d-bec4-8b71d41c80c6 service nova] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Refreshing network info cache for port 94c7c88f-9e5e-4e25-b7b8-59fb82e25f52 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:44:59 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Start _get_guest_xml network_info=[{"id": "94c7c88f-9e5e-4e25-b7b8-59fb82e25f52", "address": "fa:16:3e:c1:3e:66", "network": {"id": "47c43230-a244-4198-84e4-9ed98c005cd9", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1392403106-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c07a6b254414b41a673468238d3aa62", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap94c7c88f-9e", "ovs_interfaceid": "94c7c88f-9e5e-4e25-b7b8-59fb82e25f52", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'size': 0, 'device_type': 'disk', 'encryption_secret_uuid': None, 'guest_format': None, 'encryption_options': None, 'disk_bus': 'virtio', 'encrypted': False, 'encryption_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'image_id': '18ccda7f-31d3-4978-ab9d-4ed2f74eff94'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:44:59 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:44:59 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:44:59 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:44:59 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:44:59 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:44:59 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:44:59 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:44:59 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:44:59 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:44:59 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:44:59 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:44:59 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:44:59 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:44:59 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:44:59 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:44:59 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:44:59 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:44:59 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:44:59 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:44:56Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachVolumeNegativeTest-server-683106636',display_name='tempest-AttachVolumeNegativeTest-server-683106636',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumenegativetest-server-683106636',id=48,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHwdQ8uAXuOji59B0ui3/RExpPgI9bhpJBH1YGsboG+umNG6Flhx2vIHoK2OOqHRN++9f4+Urrs/FGjCFSAhCoRLwtfBcHoTxWtjYovYJmURgXhxEEx/g4+2Bc4XnER+PQ==',key_name='tempest-keypair-987216784',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='8c07a6b254414b41a673468238d3aa62',ramdisk_id='',reservation_id='r-iafv192h',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeNegativeTest-758771161',owner_user_name='tempest-AttachVolumeNegativeTest-758771161-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:44:58Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='17f7007adf02474e9b019cd8af099ff8',uuid=e020e9e8-2372-40c9-ab34-6f65a529a995,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "94c7c88f-9e5e-4e25-b7b8-59fb82e25f52", "address": "fa:16:3e:c1:3e:66", "network": {"id": "47c43230-a244-4198-84e4-9ed98c005cd9", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1392403106-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c07a6b254414b41a673468238d3aa62", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap94c7c88f-9e", "ovs_interfaceid": "94c7c88f-9e5e-4e25-b7b8-59fb82e25f52", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:44:59 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Converting VIF {"id": "94c7c88f-9e5e-4e25-b7b8-59fb82e25f52", "address": "fa:16:3e:c1:3e:66", "network": {"id": "47c43230-a244-4198-84e4-9ed98c005cd9", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1392403106-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c07a6b254414b41a673468238d3aa62", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap94c7c88f-9e", "ovs_interfaceid": "94c7c88f-9e5e-4e25-b7b8-59fb82e25f52", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:44:59 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:c1:3e:66,bridge_name='br-int',has_traffic_filtering=True,id=94c7c88f-9e5e-4e25-b7b8-59fb82e25f52,network=Network(47c43230-a244-4198-84e4-9ed98c005cd9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap94c7c88f-9e') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:44:59 user nova-compute[72351]: DEBUG nova.objects.instance [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lazy-loading 'pci_devices' on Instance uuid e020e9e8-2372-40c9-ab34-6f65a529a995 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:44:59 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:44:59 user nova-compute[72351]:   <uuid>e020e9e8-2372-40c9-ab34-6f65a529a995</uuid>
Aug 09 02:44:59 user nova-compute[72351]:   <name>instance-00000030</name>
Aug 09 02:44:59 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:44:59 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:44:59 user nova-compute[72351]:   <metadata>
Aug 09 02:44:59 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:44:59 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:44:59 user nova-compute[72351]:       <nova:name>tempest-AttachVolumeNegativeTest-server-683106636</nova:name>
Aug 09 02:44:59 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:44:59</nova:creationTime>
Aug 09 02:44:59 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:44:59 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:44:59 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:44:59 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:44:59 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:44:59 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:44:59 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:44:59 user nova-compute[72351]:       <nova:owner>
Aug 09 02:44:59 user nova-compute[72351]:         <nova:user uuid="17f7007adf02474e9b019cd8af099ff8">tempest-AttachVolumeNegativeTest-758771161-project-member</nova:user>
Aug 09 02:44:59 user nova-compute[72351]:         <nova:project uuid="8c07a6b254414b41a673468238d3aa62">tempest-AttachVolumeNegativeTest-758771161</nova:project>
Aug 09 02:44:59 user nova-compute[72351]:       </nova:owner>
Aug 09 02:44:59 user nova-compute[72351]:       <nova:root type="image" uuid="18ccda7f-31d3-4978-ab9d-4ed2f74eff94"/>
Aug 09 02:44:59 user nova-compute[72351]:       <nova:ports>
Aug 09 02:44:59 user nova-compute[72351]:         <nova:port uuid="94c7c88f-9e5e-4e25-b7b8-59fb82e25f52">
Aug 09 02:44:59 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.7" ipVersion="4"/>
Aug 09 02:44:59 user nova-compute[72351]:         </nova:port>
Aug 09 02:44:59 user nova-compute[72351]:       </nova:ports>
Aug 09 02:44:59 user nova-compute[72351]:     </nova:instance>
Aug 09 02:44:59 user nova-compute[72351]:   </metadata>
Aug 09 02:44:59 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:44:59 user nova-compute[72351]:     <system>
Aug 09 02:44:59 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:44:59 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:44:59 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:44:59 user nova-compute[72351]:       <entry name="serial">e020e9e8-2372-40c9-ab34-6f65a529a995</entry>
Aug 09 02:44:59 user nova-compute[72351]:       <entry name="uuid">e020e9e8-2372-40c9-ab34-6f65a529a995</entry>
Aug 09 02:44:59 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:44:59 user nova-compute[72351]:     </system>
Aug 09 02:44:59 user nova-compute[72351]:   </sysinfo>
Aug 09 02:44:59 user nova-compute[72351]:   <os>
Aug 09 02:44:59 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:44:59 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:44:59 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:44:59 user nova-compute[72351]:   </os>
Aug 09 02:44:59 user nova-compute[72351]:   <features>
Aug 09 02:44:59 user nova-compute[72351]:     <acpi/>
Aug 09 02:44:59 user nova-compute[72351]:     <apic/>
Aug 09 02:44:59 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:44:59 user nova-compute[72351]:   </features>
Aug 09 02:44:59 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:44:59 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:44:59 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:44:59 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:44:59 user nova-compute[72351]:   </clock>
Aug 09 02:44:59 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:44:59 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:44:59 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:44:59 user nova-compute[72351]:   </cpu>
Aug 09 02:44:59 user nova-compute[72351]:   <devices>
Aug 09 02:44:59 user nova-compute[72351]:     <disk type="file" device="disk">
Aug 09 02:44:59 user nova-compute[72351]:       <driver name="qemu" type="qcow2" cache="none"/>
Aug 09 02:44:59 user nova-compute[72351]:       <source file="/opt/stack/data/nova/instances/e020e9e8-2372-40c9-ab34-6f65a529a995/disk"/>
Aug 09 02:44:59 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:44:59 user nova-compute[72351]:     </disk>
Aug 09 02:44:59 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:44:59 user nova-compute[72351]:       <mac address="fa:16:3e:c1:3e:66"/>
Aug 09 02:44:59 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:44:59 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:44:59 user nova-compute[72351]:       <target dev="tap94c7c88f-9e"/>
Aug 09 02:44:59 user nova-compute[72351]:     </interface>
Aug 09 02:44:59 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:44:59 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/e020e9e8-2372-40c9-ab34-6f65a529a995/console.log" append="off"/>
Aug 09 02:44:59 user nova-compute[72351]:     </serial>
Aug 09 02:44:59 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:44:59 user nova-compute[72351]:     <video>
Aug 09 02:44:59 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:44:59 user nova-compute[72351]:     </video>
Aug 09 02:44:59 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:44:59 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:44:59 user nova-compute[72351]:     </rng>
Aug 09 02:44:59 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:44:59 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:44:59 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:44:59 user nova-compute[72351]:     </memballoon>
Aug 09 02:44:59 user nova-compute[72351]:   </devices>
Aug 09 02:44:59 user nova-compute[72351]: </domain>
Aug 09 02:44:59 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:44:59 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:44:56Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachVolumeNegativeTest-server-683106636',display_name='tempest-AttachVolumeNegativeTest-server-683106636',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumenegativetest-server-683106636',id=48,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHwdQ8uAXuOji59B0ui3/RExpPgI9bhpJBH1YGsboG+umNG6Flhx2vIHoK2OOqHRN++9f4+Urrs/FGjCFSAhCoRLwtfBcHoTxWtjYovYJmURgXhxEEx/g4+2Bc4XnER+PQ==',key_name='tempest-keypair-987216784',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='8c07a6b254414b41a673468238d3aa62',ramdisk_id='',reservation_id='r-iafv192h',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeNegativeTest-758771161',owner_user_name='tempest-AttachVolumeNegativeTest-758771161-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:44:58Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='17f7007adf02474e9b019cd8af099ff8',uuid=e020e9e8-2372-40c9-ab34-6f65a529a995,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "94c7c88f-9e5e-4e25-b7b8-59fb82e25f52", "address": "fa:16:3e:c1:3e:66", "network": {"id": "47c43230-a244-4198-84e4-9ed98c005cd9", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1392403106-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c07a6b254414b41a673468238d3aa62", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap94c7c88f-9e", "ovs_interfaceid": "94c7c88f-9e5e-4e25-b7b8-59fb82e25f52", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:44:59 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Converting VIF {"id": "94c7c88f-9e5e-4e25-b7b8-59fb82e25f52", "address": "fa:16:3e:c1:3e:66", "network": {"id": "47c43230-a244-4198-84e4-9ed98c005cd9", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1392403106-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c07a6b254414b41a673468238d3aa62", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap94c7c88f-9e", "ovs_interfaceid": "94c7c88f-9e5e-4e25-b7b8-59fb82e25f52", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:44:59 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:c1:3e:66,bridge_name='br-int',has_traffic_filtering=True,id=94c7c88f-9e5e-4e25-b7b8-59fb82e25f52,network=Network(47c43230-a244-4198-84e4-9ed98c005cd9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap94c7c88f-9e') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:44:59 user nova-compute[72351]: DEBUG os_vif [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:c1:3e:66,bridge_name='br-int',has_traffic_filtering=True,id=94c7c88f-9e5e-4e25-b7b8-59fb82e25f52,network=Network(47c43230-a244-4198-84e4-9ed98c005cd9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap94c7c88f-9e') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:44:59 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:44:59 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:44:59 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:44:59 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:44:59 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap94c7c88f-9e, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:44:59 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap94c7c88f-9e, col_values=(('external_ids', {'iface-id': '94c7c88f-9e5e-4e25-b7b8-59fb82e25f52', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:c1:3e:66', 'vm-uuid': 'e020e9e8-2372-40c9-ab34-6f65a529a995'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:44:59 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:44:59 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:44:59 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:44:59 user nova-compute[72351]: INFO os_vif [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:c1:3e:66,bridge_name='br-int',has_traffic_filtering=True,id=94c7c88f-9e5e-4e25-b7b8-59fb82e25f52,network=Network(47c43230-a244-4198-84e4-9ed98c005cd9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap94c7c88f-9e')
Aug 09 02:44:59 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:44:59 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] No VIF found with MAC fa:16:3e:c1:3e:66, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:45:00 user nova-compute[72351]: DEBUG nova.network.neutron [req-4e3c0062-8de8-4d82-8352-12f795efde4d req-e4684321-3cf5-4f4d-bec4-8b71d41c80c6 service nova] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Updated VIF entry in instance network info cache for port 94c7c88f-9e5e-4e25-b7b8-59fb82e25f52. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:45:00 user nova-compute[72351]: DEBUG nova.network.neutron [req-4e3c0062-8de8-4d82-8352-12f795efde4d req-e4684321-3cf5-4f4d-bec4-8b71d41c80c6 service nova] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Updating instance_info_cache with network_info: [{"id": "94c7c88f-9e5e-4e25-b7b8-59fb82e25f52", "address": "fa:16:3e:c1:3e:66", "network": {"id": "47c43230-a244-4198-84e4-9ed98c005cd9", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1392403106-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c07a6b254414b41a673468238d3aa62", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap94c7c88f-9e", "ovs_interfaceid": "94c7c88f-9e5e-4e25-b7b8-59fb82e25f52", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:45:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-4e3c0062-8de8-4d82-8352-12f795efde4d req-e4684321-3cf5-4f4d-bec4-8b71d41c80c6 service nova] Releasing lock "refresh_cache-e020e9e8-2372-40c9-ab34-6f65a529a995" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:45:01 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:45:01 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:45:01 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:45:01 user nova-compute[72351]: DEBUG nova.compute.manager [req-a0982778-a944-4994-b02e-dd06bf0fa6eb req-6982da05-3189-40c6-8aca-0eefc1ec0479 service nova] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Received event network-vif-plugged-94c7c88f-9e5e-4e25-b7b8-59fb82e25f52 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:45:01 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a0982778-a944-4994-b02e-dd06bf0fa6eb req-6982da05-3189-40c6-8aca-0eefc1ec0479 service nova] Acquiring lock "e020e9e8-2372-40c9-ab34-6f65a529a995-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:45:01 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a0982778-a944-4994-b02e-dd06bf0fa6eb req-6982da05-3189-40c6-8aca-0eefc1ec0479 service nova] Lock "e020e9e8-2372-40c9-ab34-6f65a529a995-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:45:01 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a0982778-a944-4994-b02e-dd06bf0fa6eb req-6982da05-3189-40c6-8aca-0eefc1ec0479 service nova] Lock "e020e9e8-2372-40c9-ab34-6f65a529a995-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:45:01 user nova-compute[72351]: DEBUG nova.compute.manager [req-a0982778-a944-4994-b02e-dd06bf0fa6eb req-6982da05-3189-40c6-8aca-0eefc1ec0479 service nova] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] No waiting events found dispatching network-vif-plugged-94c7c88f-9e5e-4e25-b7b8-59fb82e25f52 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:45:01 user nova-compute[72351]: WARNING nova.compute.manager [req-a0982778-a944-4994-b02e-dd06bf0fa6eb req-6982da05-3189-40c6-8aca-0eefc1ec0479 service nova] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Received unexpected event network-vif-plugged-94c7c88f-9e5e-4e25-b7b8-59fb82e25f52 for instance with vm_state building and task_state spawning.
Aug 09 02:45:01 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:45:01 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:45:01 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:45:03 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691549103.171588, e020e9e8-2372-40c9-ab34-6f65a529a995 => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:45:03 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] VM Resumed (Lifecycle Event)
Aug 09 02:45:03 user nova-compute[72351]: DEBUG nova.compute.manager [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:45:03 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Guest created on hypervisor {{(pid=72351) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4411}}
Aug 09 02:45:03 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Instance spawned successfully.
Aug 09 02:45:03 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:894}}
Aug 09 02:45:03 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:45:03 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:45:03 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Found default for hw_cdrom_bus of ide {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:45:03 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Found default for hw_disk_bus of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:45:03 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Found default for hw_input_bus of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:45:03 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Found default for hw_pointer_model of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:45:03 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Found default for hw_video_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:45:03 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Found default for hw_vif_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:45:03 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:45:03 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691549103.172851, e020e9e8-2372-40c9-ab34-6f65a529a995 => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:45:03 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] VM Started (Lifecycle Event)
Aug 09 02:45:03 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:45:03 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:45:03 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:45:03 user nova-compute[72351]: INFO nova.compute.manager [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Took 5.45 seconds to spawn the instance on the hypervisor.
Aug 09 02:45:03 user nova-compute[72351]: DEBUG nova.compute.manager [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:45:03 user nova-compute[72351]: INFO nova.compute.manager [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Took 5.99 seconds to build instance.
Aug 09 02:45:03 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c94ebfbc-108c-4076-8782-635eb228077d tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "e020e9e8-2372-40c9-ab34-6f65a529a995" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 6.088s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:45:03 user nova-compute[72351]: DEBUG nova.compute.manager [req-cbe2229e-fa39-47a4-8145-17ed03db8cd5 req-e4ca565c-824e-4205-9c6e-0b4d742030cf service nova] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Received event network-vif-plugged-94c7c88f-9e5e-4e25-b7b8-59fb82e25f52 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:45:03 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-cbe2229e-fa39-47a4-8145-17ed03db8cd5 req-e4ca565c-824e-4205-9c6e-0b4d742030cf service nova] Acquiring lock "e020e9e8-2372-40c9-ab34-6f65a529a995-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:45:03 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-cbe2229e-fa39-47a4-8145-17ed03db8cd5 req-e4ca565c-824e-4205-9c6e-0b4d742030cf service nova] Lock "e020e9e8-2372-40c9-ab34-6f65a529a995-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:45:03 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-cbe2229e-fa39-47a4-8145-17ed03db8cd5 req-e4ca565c-824e-4205-9c6e-0b4d742030cf service nova] Lock "e020e9e8-2372-40c9-ab34-6f65a529a995-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:45:03 user nova-compute[72351]: DEBUG nova.compute.manager [req-cbe2229e-fa39-47a4-8145-17ed03db8cd5 req-e4ca565c-824e-4205-9c6e-0b4d742030cf service nova] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] No waiting events found dispatching network-vif-plugged-94c7c88f-9e5e-4e25-b7b8-59fb82e25f52 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:45:03 user nova-compute[72351]: WARNING nova.compute.manager [req-cbe2229e-fa39-47a4-8145-17ed03db8cd5 req-e4ca565c-824e-4205-9c6e-0b4d742030cf service nova] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Received unexpected event network-vif-plugged-94c7c88f-9e5e-4e25-b7b8-59fb82e25f52 for instance with vm_state active and task_state None.
Aug 09 02:45:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:45:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:45:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:45:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:45:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 02:45:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:45:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:45:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:45:13 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:45:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:45:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:45:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 02:45:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:45:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:45:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:45:14 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:45:15 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:45:15 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72351) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10461}}
Aug 09 02:45:17 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:45:17 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Starting heal instance info cache {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9842}}
Aug 09 02:45:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "refresh_cache-c6ae885f-dce2-4dc2-98d5-26139b8fb67e" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:45:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquired lock "refresh_cache-c6ae885f-dce2-4dc2-98d5-26139b8fb67e" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:45:17 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Forcefully refreshing network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1996}}
Aug 09 02:45:18 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Updating instance_info_cache with network_info: [{"id": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "address": "fa:16:3e:dd:49:4d", "network": {"id": "dba57c3d-6efd-4057-8053-1c6a92bb24f8", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-568585773-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ec9861e0882e4065ab99ad3f74d6b25f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2771d93d-fa", "ovs_interfaceid": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:45:18 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Releasing lock "refresh_cache-c6ae885f-dce2-4dc2-98d5-26139b8fb67e" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:45:18 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Updated the network info_cache for instance {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9913}}
Aug 09 02:45:18 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:45:18 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:45:18 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:45:18 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:45:18 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Auditing locally available compute resources for user (node: user) {{(pid=72351) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Aug 09 02:45:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c740d131-98a9-4b14-a75a-a33aa7c72b51/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:45:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c740d131-98a9-4b14-a75a-a33aa7c72b51/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:45:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c740d131-98a9-4b14-a75a-a33aa7c72b51/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:45:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c740d131-98a9-4b14-a75a-a33aa7c72b51/disk --force-share --output=json" returned: 0 in 0.139s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:45:18 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdb (vdc) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:45:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:45:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:45:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk --force-share --output=json" returned: 0 in 0.130s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:45:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:45:19 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk --force-share --output=json" returned: 0 in 0.148s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:45:19 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e020e9e8-2372-40c9-ab34-6f65a529a995/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:45:19 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e020e9e8-2372-40c9-ab34-6f65a529a995/disk --force-share --output=json" returned: 0 in 0.130s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:45:19 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e020e9e8-2372-40c9-ab34-6f65a529a995/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:45:19 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e020e9e8-2372-40c9-ab34-6f65a529a995/disk --force-share --output=json" returned: 0 in 0.151s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:45:19 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:45:19 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:45:19 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:45:19 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Hypervisor/Node resource view: name=user free_ram=8387MB free_disk=25.24292755126953GB free_vcpus=9 pci_devices=[{"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72351) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Aug 09 02:45:19 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:45:19 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:45:19 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance c740d131-98a9-4b14-a75a-a33aa7c72b51 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:45:19 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance c6ae885f-dce2-4dc2-98d5-26139b8fb67e actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:45:19 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance e020e9e8-2372-40c9-ab34-6f65a529a995 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:45:19 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Total usable vcpus: 12, total allocated vcpus: 3 {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Aug 09 02:45:19 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Final resource view: name=user phys_ram=16011MB used_ram=2048MB phys_disk=40GB used_disk=3GB total_vcpus=12 used_vcpus=3 pci_stats=[] {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Aug 09 02:45:20 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:45:20 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:45:20 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Compute_service record updated for user:user {{(pid=72351) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Aug 09 02:45:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.253s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:45:21 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:45:21 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:45:21 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:45:21 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:45:23 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:45:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:45:24 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:45:28 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:45:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-cc6b35b2-a072-4af8-956c-386c5be4ebec tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Acquiring lock "c740d131-98a9-4b14-a75a-a33aa7c72b51" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:45:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-cc6b35b2-a072-4af8-956c-386c5be4ebec tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "c740d131-98a9-4b14-a75a-a33aa7c72b51" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:45:29 user nova-compute[72351]: INFO nova.compute.manager [None req-cc6b35b2-a072-4af8-956c-386c5be4ebec tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Detaching volume 1bdf7f72-7504-4539-9f73-c9c66d0c2e1d
Aug 09 02:45:29 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:45:29 user nova-compute[72351]: INFO nova.virt.block_device [None req-cc6b35b2-a072-4af8-956c-386c5be4ebec tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Attempting to driver detach volume 1bdf7f72-7504-4539-9f73-c9c66d0c2e1d from mountpoint /dev/vdc
Aug 09 02:45:29 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-cc6b35b2-a072-4af8-956c-386c5be4ebec tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Attempting to detach device vdc from instance c740d131-98a9-4b14-a75a-a33aa7c72b51 from the persistent domain config. {{(pid=72351) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2477}}
Aug 09 02:45:29 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-cc6b35b2-a072-4af8-956c-386c5be4ebec tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 02:45:29 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:45:29 user nova-compute[72351]:   <source dev="/dev/sdb"/>
Aug 09 02:45:29 user nova-compute[72351]:   <target dev="vdc" bus="virtio"/>
Aug 09 02:45:29 user nova-compute[72351]:   <serial>1bdf7f72-7504-4539-9f73-c9c66d0c2e1d</serial>
Aug 09 02:45:29 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x05" function="0x0"/>
Aug 09 02:45:29 user nova-compute[72351]: </disk>
Aug 09 02:45:29 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 02:45:29 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-cc6b35b2-a072-4af8-956c-386c5be4ebec tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Successfully detached device vdc from instance c740d131-98a9-4b14-a75a-a33aa7c72b51 from the persistent domain config.
Aug 09 02:45:29 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-cc6b35b2-a072-4af8-956c-386c5be4ebec tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] (1/8): Attempting to detach device vdc with device alias virtio-disk2 from instance c740d131-98a9-4b14-a75a-a33aa7c72b51 from the live domain config. {{(pid=72351) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2513}}
Aug 09 02:45:29 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-cc6b35b2-a072-4af8-956c-386c5be4ebec tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 02:45:29 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:45:29 user nova-compute[72351]:   <source dev="/dev/sdb"/>
Aug 09 02:45:29 user nova-compute[72351]:   <target dev="vdc" bus="virtio"/>
Aug 09 02:45:29 user nova-compute[72351]:   <serial>1bdf7f72-7504-4539-9f73-c9c66d0c2e1d</serial>
Aug 09 02:45:29 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x05" function="0x0"/>
Aug 09 02:45:29 user nova-compute[72351]: </disk>
Aug 09 02:45:29 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 02:45:30 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Received event <DeviceRemovedEvent: 1691549130.338514, c740d131-98a9-4b14-a75a-a33aa7c72b51 => virtio-disk2> from libvirt while the driver is waiting for it; dispatched. {{(pid=72351) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2360}}
Aug 09 02:45:30 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-cc6b35b2-a072-4af8-956c-386c5be4ebec tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Start waiting for the detach event from libvirt for device vdc with device alias virtio-disk2 for instance c740d131-98a9-4b14-a75a-a33aa7c72b51 {{(pid=72351) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2589}}
Aug 09 02:45:30 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-cc6b35b2-a072-4af8-956c-386c5be4ebec tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Successfully detached device vdc from instance c740d131-98a9-4b14-a75a-a33aa7c72b51 from the live domain config.
Aug 09 02:45:30 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-cc6b35b2-a072-4af8-956c-386c5be4ebec tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 02:45:30 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-cc6b35b2-a072-4af8-956c-386c5be4ebec tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:1bdf7f72-7504-4539-9f73-c9c66d0c2e1d'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'qLeTFgdb', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb'}, None), 'kwargs': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:45:30 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-cc6b35b2-a072-4af8-956c-386c5be4ebec tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:45:30 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-cc6b35b2-a072-4af8-956c-386c5be4ebec tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:45:30 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-cc6b35b2-a072-4af8-956c-386c5be4ebec tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:45:30 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-cc6b35b2-a072-4af8-956c-386c5be4ebec tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:1bdf7f72-7504-4539-9f73-c9c66d0c2e1d', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 02:45:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:45:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:45:30 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[73ab49a3-147f-4116-befe-85cc7a5ad0f4]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:1bdf7f72-7504-4539-9f73-c9c66d0c2e1d\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:45:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:45:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:45:30 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[5ff7e92f-5f8b-4eb3-86db-ccaac505fbed]: (4, ('tcp: [57] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:1bdf7f72-7504-4539-9f73-c9c66d0c2e1d (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:45:30 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-cc6b35b2-a072-4af8-956c-386c5be4ebec tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [57] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:1bdf7f72-7504-4539-9f73-c9c66d0c2e1d (non-flash)
Aug 09 02:45:30 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:45:30 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-cc6b35b2-a072-4af8-956c-386c5be4ebec tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] iscsi session list stdout=tcp: [57] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:1bdf7f72-7504-4539-9f73-c9c66d0c2e1d (non-flash)
Aug 09 02:45:30 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:45:30 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-cc6b35b2-a072-4af8-956c-386c5be4ebec tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fcfbc63fa30>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:1bdf7f72-7504-4539-9f73-c9c66d0c2e1d'): ({'sdb'}, set())}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 02:45:30 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-cc6b35b2-a072-4af8-956c-386c5be4ebec tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Removing single pathed devices sdb {{(pid=72351) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Aug 09 02:45:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:45:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:45:30 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[80b3cba8-f399-4885-afcc-30c95e7b2df7]: (4, ('path checker states:\nup                  1\n\npaths: 0\nbusy: False\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:45:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:45:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:45:30 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[8f9472d5-cd31-4df4-bb12-8907d083e384]: (4, ('ok\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:45:30 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-cc6b35b2-a072-4af8-956c-386c5be4ebec tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Flushing IO for device /dev/sdb {{(pid=72351) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Aug 09 02:45:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:45:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.014s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:45:30 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[4624d586-699c-49dd-a49e-f590d12a4f6e]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:45:30 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-cc6b35b2-a072-4af8-956c-386c5be4ebec tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=72351) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Aug 09 02:45:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:45:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.033s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:45:30 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[62e49f96-4ae4-484e-b598-6011b7561041]: (4, ('1', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:45:30 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-cc6b35b2-a072-4af8-956c-386c5be4ebec tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Aug 09 02:45:30 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-cc6b35b2-a072-4af8-956c-386c5be4ebec tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] SCSI volumes sdb have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Aug 09 02:45:30 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-cc6b35b2-a072-4af8-956c-386c5be4ebec tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:1bdf7f72-7504-4539-9f73-c9c66d0c2e1d')] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 02:45:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:1bdf7f72-7504-4539-9f73-c9c66d0c2e1d -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:45:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:1bdf7f72-7504-4539-9f73-c9c66d0c2e1d -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.024s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:45:30 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[7425af5c-2152-4e24-984f-f3217e06a520]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:45:30 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-cc6b35b2-a072-4af8-956c-386c5be4ebec tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:45:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:1bdf7f72-7504-4539-9f73-c9c66d0c2e1d -p 172.16.0.220:3260 --logout {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:45:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:1bdf7f72-7504-4539-9f73-c9c66d0c2e1d -p 172.16.0.220:3260 --logout" returned: 0 in 0.062s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:45:30 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[9396f1b5-19e1-4b5b-bfcb-fb35a95f4a34]: (4, ('Logging out of session [sid: 57, target: iqn.2010-10.org.openstack:1bdf7f72-7504-4539-9f73-c9c66d0c2e1d, portal: 172.16.0.220,3260]\nLogout of [sid: 57, target: iqn.2010-10.org.openstack:1bdf7f72-7504-4539-9f73-c9c66d0c2e1d, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:45:30 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-cc6b35b2-a072-4af8-956c-386c5be4ebec tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 57, target: iqn.2010-10.org.openstack:1bdf7f72-7504-4539-9f73-c9c66d0c2e1d, portal: 172.16.0.220,3260]
Aug 09 02:45:30 user nova-compute[72351]: Logout of [sid: 57, target: iqn.2010-10.org.openstack:1bdf7f72-7504-4539-9f73-c9c66d0c2e1d, portal: 172.16.0.220,3260] successful.
Aug 09 02:45:30 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:45:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:1bdf7f72-7504-4539-9f73-c9c66d0c2e1d -p 172.16.0.220:3260 --op delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:45:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:1bdf7f72-7504-4539-9f73-c9c66d0c2e1d -p 172.16.0.220:3260 --op delete" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:45:30 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[6c026f5f-d6a3-4a73-957c-2df1bae57e87]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:45:30 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-cc6b35b2-a072-4af8-956c-386c5be4ebec tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:45:30 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-cc6b35b2-a072-4af8-956c-386c5be4ebec tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.278s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:45:30 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-cc6b35b2-a072-4af8-956c-386c5be4ebec tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] <== disconnect_volume: return (279ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:45:30 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-cc6b35b2-a072-4af8-956c-386c5be4ebec tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 02:45:31 user nova-compute[72351]: DEBUG nova.objects.instance [None req-cc6b35b2-a072-4af8-956c-386c5be4ebec tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lazy-loading 'flavor' on Instance uuid c740d131-98a9-4b14-a75a-a33aa7c72b51 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:45:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-cc6b35b2-a072-4af8-956c-386c5be4ebec tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "c740d131-98a9-4b14-a75a-a33aa7c72b51" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.620s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:45:33 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:45:34 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:45:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:45:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:45:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:45:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:45:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5001 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 02:45:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:45:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:45:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:45:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:45:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:45:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5001 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 02:45:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:45:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:45:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:45:59 user nova-compute[72351]: DEBUG nova.compute.manager [req-842fddae-e0f7-4955-958e-c07cce447bf7 req-d2020c03-8d57-431e-9bfa-70a73508fa87 service nova] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Received event network-changed-2771d93d-fa0a-4e19-85ac-9d3f5df52782 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:45:59 user nova-compute[72351]: DEBUG nova.compute.manager [req-842fddae-e0f7-4955-958e-c07cce447bf7 req-d2020c03-8d57-431e-9bfa-70a73508fa87 service nova] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Refreshing instance network info cache due to event network-changed-2771d93d-fa0a-4e19-85ac-9d3f5df52782. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:45:59 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-842fddae-e0f7-4955-958e-c07cce447bf7 req-d2020c03-8d57-431e-9bfa-70a73508fa87 service nova] Acquiring lock "refresh_cache-c6ae885f-dce2-4dc2-98d5-26139b8fb67e" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:45:59 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-842fddae-e0f7-4955-958e-c07cce447bf7 req-d2020c03-8d57-431e-9bfa-70a73508fa87 service nova] Acquired lock "refresh_cache-c6ae885f-dce2-4dc2-98d5-26139b8fb67e" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:45:59 user nova-compute[72351]: DEBUG nova.network.neutron [req-842fddae-e0f7-4955-958e-c07cce447bf7 req-d2020c03-8d57-431e-9bfa-70a73508fa87 service nova] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Refreshing network info cache for port 2771d93d-fa0a-4e19-85ac-9d3f5df52782 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:45:59 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:46:00 user nova-compute[72351]: DEBUG nova.network.neutron [req-842fddae-e0f7-4955-958e-c07cce447bf7 req-d2020c03-8d57-431e-9bfa-70a73508fa87 service nova] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Updated VIF entry in instance network info cache for port 2771d93d-fa0a-4e19-85ac-9d3f5df52782. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:46:00 user nova-compute[72351]: DEBUG nova.network.neutron [req-842fddae-e0f7-4955-958e-c07cce447bf7 req-d2020c03-8d57-431e-9bfa-70a73508fa87 service nova] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Updating instance_info_cache with network_info: [{"id": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "address": "fa:16:3e:dd:49:4d", "network": {"id": "dba57c3d-6efd-4057-8053-1c6a92bb24f8", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-568585773-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.116", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ec9861e0882e4065ab99ad3f74d6b25f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2771d93d-fa", "ovs_interfaceid": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:46:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-842fddae-e0f7-4955-958e-c07cce447bf7 req-d2020c03-8d57-431e-9bfa-70a73508fa87 service nova] Releasing lock "refresh_cache-c6ae885f-dce2-4dc2-98d5-26139b8fb67e" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:46:03 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-bb62d707-9030-44a3-bec2-ea71177d30ab tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Acquiring lock "c6ae885f-dce2-4dc2-98d5-26139b8fb67e" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:46:03 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-bb62d707-9030-44a3-bec2-ea71177d30ab tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "c6ae885f-dce2-4dc2-98d5-26139b8fb67e" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:46:03 user nova-compute[72351]: DEBUG nova.objects.instance [None req-bb62d707-9030-44a3-bec2-ea71177d30ab tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lazy-loading 'flavor' on Instance uuid c6ae885f-dce2-4dc2-98d5-26139b8fb67e {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:46:03 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-bb62d707-9030-44a3-bec2-ea71177d30ab tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "c6ae885f-dce2-4dc2-98d5-26139b8fb67e" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.048s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:46:03 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-bb62d707-9030-44a3-bec2-ea71177d30ab tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Acquiring lock "c6ae885f-dce2-4dc2-98d5-26139b8fb67e" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:46:03 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-bb62d707-9030-44a3-bec2-ea71177d30ab tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "c6ae885f-dce2-4dc2-98d5-26139b8fb67e" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:46:03 user nova-compute[72351]: INFO nova.compute.manager [None req-bb62d707-9030-44a3-bec2-ea71177d30ab tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Attaching volume 008f5fa6-2644-4c22-b8cd-28e233cf8857 to /dev/vdb
Aug 09 02:46:03 user nova-compute[72351]: DEBUG os_brick.utils [None req-bb62d707-9030-44a3-bec2-ea71177d30ab tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:46:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:46:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:46:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.013s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:46:03 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[3c45a52d-2b41-412e-9089-d79e85d9e093]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:a52269602ff4\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:46:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:46:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.016s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:46:03 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[0e37a3a7-4c54-40fa-a281-2d9d3bd9fe84]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:46:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:46:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:46:03 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[324ad37d-5207-43f1-9325-6a8276413f6b]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:46:03 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[0cb66634-780a-4972-b0a1-9668a8480b1a]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:46:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-bb62d707-9030-44a3-bec2-ea71177d30ab tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Running cmd (subprocess): nvme version {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:46:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-bb62d707-9030-44a3-bec2-ea71177d30ab tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] 'nvme version' failed. Not Retrying. {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:46:04 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.nvmeof [None req-bb62d707-9030-44a3-bec2-ea71177d30ab tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] nvme not present on system {{(pid=72351) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Aug 09 02:46:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:46:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:46:04 user nova-compute[72351]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Aug 09 02:46:04 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[55ea3701-0977-4129-a843-13bbc4fef403]: (4, '') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:46:04 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-bb62d707-9030-44a3-bec2-ea71177d30ab tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72351) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Aug 09 02:46:04 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-bb62d707-9030-44a3-bec2-ea71177d30ab tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Aug 09 02:46:04 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-bb62d707-9030-44a3-bec2-ea71177d30ab tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] LIGHTOS: no hostnqn found. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Aug 09 02:46:04 user nova-compute[72351]: DEBUG os_brick.utils [None req-bb62d707-9030-44a3-bec2-ea71177d30ab tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] <== get_connector_properties: return (118ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:a52269602ff4', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:46:04 user nova-compute[72351]: DEBUG nova.virt.block_device [None req-bb62d707-9030-44a3-bec2-ea71177d30ab tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Updating existing volume attachment record: a470ef1b-600e-4db5-8aeb-04002b6747a0 {{(pid=72351) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Aug 09 02:46:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:46:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:46:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:46:11 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-bb62d707-9030-44a3-bec2-ea71177d30ab tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 02:46:11 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-bb62d707-9030-44a3-bec2-ea71177d30ab tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'QgoDJMbr', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:46:11 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-bb62d707-9030-44a3-bec2-ea71177d30ab tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:46:11 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-bb62d707-9030-44a3-bec2-ea71177d30ab tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:46:11 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-bb62d707-9030-44a3-bec2-ea71177d30ab tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:46:11 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-bb62d707-9030-44a3-bec2-ea71177d30ab tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:46:11 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-bb62d707-9030-44a3-bec2-ea71177d30ab tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:46:11 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-bb62d707-9030-44a3-bec2-ea71177d30ab tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:46:11 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-bb62d707-9030-44a3-bec2-ea71177d30ab tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 02:46:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:46:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 -p 172.16.0.220:3260" returned: 21 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:46:11 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[59eced44-266c-479e-9930-7f52485b7841]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:46:11 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-bb62d707-9030-44a3-bec2-ea71177d30ab tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Aug 09 02:46:11 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:46:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 -p 172.16.0.220:3260 --interface default --op new {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:46:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:46:11 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[93b7b02f-c243-4418-bfb4-9a1741b9888a]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857] added\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:46:11 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-bb62d707-9030-44a3-bec2-ea71177d30ab tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857] added
Aug 09 02:46:11 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:46:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:46:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:46:11 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ecf94808-d964-4fa7-a46c-b024d78b66e3]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:46:11 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-bb62d707-9030-44a3-bec2-ea71177d30ab tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:46:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:46:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:46:11 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[af99e6a8-2aeb-496b-b8f5-aaee7182aecf]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:46:11 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-bb62d707-9030-44a3-bec2-ea71177d30ab tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:46:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v QgoDJMbr {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:46:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v QgoDJMbr" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:46:11 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[7f8d4bda-53fa-4044-8213-df63cf3a462a]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:46:11 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-bb62d707-9030-44a3-bec2-ea71177d30ab tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'QgoDJMbr'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:46:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:46:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:46:11 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a0d9b7af-4552-457c-9f4f-b42bd99d528f]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:46:11 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-bb62d707-9030-44a3-bec2-ea71177d30ab tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:46:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:46:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 21 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:46:11 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[98177f47-3c16-49cd-90d4-a4be83ce74ff]: (4, ('', 'iscsiadm: No active sessions.\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:46:11 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-bb62d707-9030-44a3-bec2-ea71177d30ab tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsiadm ('-m', 'session'): stdout= stderr=iscsiadm: No active sessions.
Aug 09 02:46:11 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:46:11 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-bb62d707-9030-44a3-bec2-ea71177d30ab tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsi session list stdout= stderr=iscsiadm: No active sessions.
Aug 09 02:46:11 user nova-compute[72351]:  {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:46:11 user nova-compute[72351]: WARNING os_brick.initiator.connectors.iscsi [None req-bb62d707-9030-44a3-bec2-ea71177d30ab tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsiadm stderr output when getting sessions: iscsiadm: No active sessions.
Aug 09 02:46:11 user nova-compute[72351]: 
Aug 09 02:46:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 -p 172.16.0.220:3260 --login {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:46:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 -p 172.16.0.220:3260 --login" returned: 0 in 0.045s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:46:11 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[5cad312a-38c0-4406-b160-d84962a7f854]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:46:11 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-bb62d707-9030-44a3-bec2-ea71177d30ab tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857, portal: 172.16.0.220,3260]
Aug 09 02:46:11 user nova-compute[72351]: Login to [iface: default, target: iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857, portal: 172.16.0.220,3260] successful.
Aug 09 02:46:11 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:46:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:46:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:46:11 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[63152fdb-07a1-4a9d-8332-91f037c5224f]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:46:11 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-bb62d707-9030-44a3-bec2-ea71177d30ab tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:46:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:46:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:46:11 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[345ea9ce-153f-4b21-b188-37576d3b2f98]: (4, ('tcp: [58] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:46:11 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-bb62d707-9030-44a3-bec2-ea71177d30ab tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [58] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 (non-flash)
Aug 09 02:46:11 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:46:11 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-bb62d707-9030-44a3-bec2-ea71177d30ab tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsi session list stdout=tcp: [58] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 (non-flash)
Aug 09 02:46:11 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:46:11 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-bb62d707-9030-44a3-bec2-ea71177d30ab tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.251s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:46:11 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-bb62d707-9030-44a3-bec2-ea71177d30ab tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 02:46:11 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-bb62d707-9030-44a3-bec2-ea71177d30ab tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] HCTL ('33', '-', '-', 0) found on session 58 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 02:46:11 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-bb62d707-9030-44a3-bec2-ea71177d30ab tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 02:46:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:46:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:46:11 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[627ff381-5b79-41c4-9514-bdf0cfcfe7bd]: (4, ('- - 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:46:11 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-bb62d707-9030-44a3-bec2-ea71177d30ab tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Searching for a device in session 58 and hctl ['33', '*', '*', 0] yield: None {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:46:12 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-bb62d707-9030-44a3-bec2-ea71177d30ab tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Searching for a device in session 58 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:46:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-bb62d707-9030-44a3-bec2-ea71177d30ab tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Connected to sdb using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'QgoDJMbr', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 02:46:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-bb62d707-9030-44a3-bec2-ea71177d30ab tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.280s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:46:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-bb62d707-9030-44a3-bec2-ea71177d30ab tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] <== connect_volume: return (1281ms) {'type': 'block', 'scsi_wwn': '23066323566643038', 'path': '/dev/sdb'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:46:12 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-bb62d707-9030-44a3-bec2-ea71177d30ab tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23066323566643038', 'path': '/dev/sdb'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 02:46:12 user nova-compute[72351]: DEBUG nova.objects.instance [None req-bb62d707-9030-44a3-bec2-ea71177d30ab tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lazy-loading 'flavor' on Instance uuid c6ae885f-dce2-4dc2-98d5-26139b8fb67e {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:46:12 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-bb62d707-9030-44a3-bec2-ea71177d30ab tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] attach device xml: <disk type="block" device="disk">
Aug 09 02:46:12 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:46:12 user nova-compute[72351]:   <source dev="/dev/sdb"/>
Aug 09 02:46:12 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:46:12 user nova-compute[72351]:   <serial>008f5fa6-2644-4c22-b8cd-28e233cf8857</serial>
Aug 09 02:46:12 user nova-compute[72351]: </disk>
Aug 09 02:46:12 user nova-compute[72351]:  {{(pid=72351) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Aug 09 02:46:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-bb62d707-9030-44a3-bec2-ea71177d30ab tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:46:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-bb62d707-9030-44a3-bec2-ea71177d30ab tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:46:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-bb62d707-9030-44a3-bec2-ea71177d30ab tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] No VIF found with MAC fa:16:3e:dd:49:4d, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:46:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-bb62d707-9030-44a3-bec2-ea71177d30ab tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "c6ae885f-dce2-4dc2-98d5-26139b8fb67e" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 9.459s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:46:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-d51fc9fa-ded1-40c8-9d63-2795088f73fb tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Acquiring lock "c6ae885f-dce2-4dc2-98d5-26139b8fb67e" by "nova.compute.manager.ComputeManager.stop_instance.<locals>.do_stop_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:46:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-d51fc9fa-ded1-40c8-9d63-2795088f73fb tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "c6ae885f-dce2-4dc2-98d5-26139b8fb67e" acquired by "nova.compute.manager.ComputeManager.stop_instance.<locals>.do_stop_instance" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:46:13 user nova-compute[72351]: DEBUG nova.compute.manager [None req-d51fc9fa-ded1-40c8-9d63-2795088f73fb tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:46:13 user nova-compute[72351]: DEBUG nova.compute.manager [None req-d51fc9fa-ded1-40c8-9d63-2795088f73fb tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Stopping instance; current vm_state: active, current task_state: powering-off, current DB power_state: 1, current VM power_state: 1 {{(pid=72351) do_stop_instance /opt/stack/nova/nova/compute/manager.py:3342}}
Aug 09 02:46:13 user nova-compute[72351]: DEBUG nova.objects.instance [None req-d51fc9fa-ded1-40c8-9d63-2795088f73fb tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lazy-loading 'flavor' on Instance uuid c6ae885f-dce2-4dc2-98d5-26139b8fb67e {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:46:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:46:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:46:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:46:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:46:13 user nova-compute[72351]: DEBUG nova.compute.manager [req-6c6d40dc-3975-4135-b62d-041d1aadf75a req-dc5842d5-8a43-48d0-9914-94161f0caed0 service nova] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Received event network-vif-unplugged-2771d93d-fa0a-4e19-85ac-9d3f5df52782 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:46:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-6c6d40dc-3975-4135-b62d-041d1aadf75a req-dc5842d5-8a43-48d0-9914-94161f0caed0 service nova] Acquiring lock "c6ae885f-dce2-4dc2-98d5-26139b8fb67e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:46:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-6c6d40dc-3975-4135-b62d-041d1aadf75a req-dc5842d5-8a43-48d0-9914-94161f0caed0 service nova] Lock "c6ae885f-dce2-4dc2-98d5-26139b8fb67e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:46:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-6c6d40dc-3975-4135-b62d-041d1aadf75a req-dc5842d5-8a43-48d0-9914-94161f0caed0 service nova] Lock "c6ae885f-dce2-4dc2-98d5-26139b8fb67e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:46:13 user nova-compute[72351]: DEBUG nova.compute.manager [req-6c6d40dc-3975-4135-b62d-041d1aadf75a req-dc5842d5-8a43-48d0-9914-94161f0caed0 service nova] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] No waiting events found dispatching network-vif-unplugged-2771d93d-fa0a-4e19-85ac-9d3f5df52782 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:46:13 user nova-compute[72351]: WARNING nova.compute.manager [req-6c6d40dc-3975-4135-b62d-041d1aadf75a req-dc5842d5-8a43-48d0-9914-94161f0caed0 service nova] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Received unexpected event network-vif-unplugged-2771d93d-fa0a-4e19-85ac-9d3f5df52782 for instance with vm_state active and task_state powering-off.
Aug 09 02:46:13 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:46:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:46:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:46:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:46:14 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Instance destroyed successfully.
Aug 09 02:46:14 user nova-compute[72351]: DEBUG nova.compute.manager [None req-d51fc9fa-ded1-40c8-9d63-2795088f73fb tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:46:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-d51fc9fa-ded1-40c8-9d63-2795088f73fb tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "c6ae885f-dce2-4dc2-98d5-26139b8fb67e" "released" by "nova.compute.manager.ComputeManager.stop_instance.<locals>.do_stop_instance" :: held 0.844s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:46:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:46:15 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:46:15 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:46:15 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72351) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10461}}
Aug 09 02:46:15 user nova-compute[72351]: DEBUG nova.compute.manager [req-b9203f37-9d76-4f3c-8c08-9e0c8ff97663 req-9e126109-a22e-40ad-bb11-97c23d9cf98f service nova] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Received event network-vif-plugged-2771d93d-fa0a-4e19-85ac-9d3f5df52782 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:46:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-b9203f37-9d76-4f3c-8c08-9e0c8ff97663 req-9e126109-a22e-40ad-bb11-97c23d9cf98f service nova] Acquiring lock "c6ae885f-dce2-4dc2-98d5-26139b8fb67e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:46:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-b9203f37-9d76-4f3c-8c08-9e0c8ff97663 req-9e126109-a22e-40ad-bb11-97c23d9cf98f service nova] Lock "c6ae885f-dce2-4dc2-98d5-26139b8fb67e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:46:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-b9203f37-9d76-4f3c-8c08-9e0c8ff97663 req-9e126109-a22e-40ad-bb11-97c23d9cf98f service nova] Lock "c6ae885f-dce2-4dc2-98d5-26139b8fb67e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:46:15 user nova-compute[72351]: DEBUG nova.compute.manager [req-b9203f37-9d76-4f3c-8c08-9e0c8ff97663 req-9e126109-a22e-40ad-bb11-97c23d9cf98f service nova] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] No waiting events found dispatching network-vif-plugged-2771d93d-fa0a-4e19-85ac-9d3f5df52782 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:46:15 user nova-compute[72351]: WARNING nova.compute.manager [req-b9203f37-9d76-4f3c-8c08-9e0c8ff97663 req-9e126109-a22e-40ad-bb11-97c23d9cf98f service nova] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Received unexpected event network-vif-plugged-2771d93d-fa0a-4e19-85ac-9d3f5df52782 for instance with vm_state stopped and task_state None.
Aug 09 02:46:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:46:19 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:46:19 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:46:19 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Starting heal instance info cache {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9842}}
Aug 09 02:46:19 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Rebuilding the list of instances to heal {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9846}}
Aug 09 02:46:19 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "refresh_cache-c740d131-98a9-4b14-a75a-a33aa7c72b51" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:46:19 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquired lock "refresh_cache-c740d131-98a9-4b14-a75a-a33aa7c72b51" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:46:19 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Forcefully refreshing network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1996}}
Aug 09 02:46:19 user nova-compute[72351]: DEBUG nova.objects.instance [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lazy-loading 'info_cache' on Instance uuid c740d131-98a9-4b14-a75a-a33aa7c72b51 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:46:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-878c4434-bb6a-462a-9c30-5ec198e406ef tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Acquiring lock "c740d131-98a9-4b14-a75a-a33aa7c72b51" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:46:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-878c4434-bb6a-462a-9c30-5ec198e406ef tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "c740d131-98a9-4b14-a75a-a33aa7c72b51" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:46:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-878c4434-bb6a-462a-9c30-5ec198e406ef tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Acquiring lock "c740d131-98a9-4b14-a75a-a33aa7c72b51-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:46:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-878c4434-bb6a-462a-9c30-5ec198e406ef tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "c740d131-98a9-4b14-a75a-a33aa7c72b51-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:46:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-878c4434-bb6a-462a-9c30-5ec198e406ef tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "c740d131-98a9-4b14-a75a-a33aa7c72b51-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:46:20 user nova-compute[72351]: INFO nova.compute.manager [None req-878c4434-bb6a-462a-9c30-5ec198e406ef tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Terminating instance
Aug 09 02:46:20 user nova-compute[72351]: DEBUG nova.compute.manager [None req-878c4434-bb6a-462a-9c30-5ec198e406ef tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Start destroying the instance on the hypervisor. {{(pid=72351) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3124}}
Aug 09 02:46:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:46:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:46:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:46:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:46:20 user nova-compute[72351]: DEBUG nova.compute.manager [req-67ab4b81-e9c5-4d03-85d1-78575b07977d req-3dba1c94-7a47-4fec-a577-1c767fd4d7d7 service nova] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Received event network-vif-unplugged-ef71c011-3c08-4927-bf2b-763a63fe8c6f {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:46:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-67ab4b81-e9c5-4d03-85d1-78575b07977d req-3dba1c94-7a47-4fec-a577-1c767fd4d7d7 service nova] Acquiring lock "c740d131-98a9-4b14-a75a-a33aa7c72b51-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:46:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-67ab4b81-e9c5-4d03-85d1-78575b07977d req-3dba1c94-7a47-4fec-a577-1c767fd4d7d7 service nova] Lock "c740d131-98a9-4b14-a75a-a33aa7c72b51-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:46:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-67ab4b81-e9c5-4d03-85d1-78575b07977d req-3dba1c94-7a47-4fec-a577-1c767fd4d7d7 service nova] Lock "c740d131-98a9-4b14-a75a-a33aa7c72b51-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:46:20 user nova-compute[72351]: DEBUG nova.compute.manager [req-67ab4b81-e9c5-4d03-85d1-78575b07977d req-3dba1c94-7a47-4fec-a577-1c767fd4d7d7 service nova] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] No waiting events found dispatching network-vif-unplugged-ef71c011-3c08-4927-bf2b-763a63fe8c6f {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:46:20 user nova-compute[72351]: DEBUG nova.compute.manager [req-67ab4b81-e9c5-4d03-85d1-78575b07977d req-3dba1c94-7a47-4fec-a577-1c767fd4d7d7 service nova] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Received event network-vif-unplugged-ef71c011-3c08-4927-bf2b-763a63fe8c6f for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:46:20 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Updating instance_info_cache with network_info: [{"id": "ef71c011-3c08-4927-bf2b-763a63fe8c6f", "address": "fa:16:3e:73:9f:90", "network": {"id": "da9be96c-ce20-4037-999c-230ca799fb06", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-661780305-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.69", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5ff300d27c9a4dd3b7ecd507b3627d98", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapef71c011-3c", "ovs_interfaceid": "ef71c011-3c08-4927-bf2b-763a63fe8c6f", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:46:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Releasing lock "refresh_cache-c740d131-98a9-4b14-a75a-a33aa7c72b51" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:46:20 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Updated the network info_cache for instance {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9913}}
Aug 09 02:46:20 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:46:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:46:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:46:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:46:20 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Auditing locally available compute resources for user (node: user) {{(pid=72351) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Aug 09 02:46:20 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Instance destroyed successfully.
Aug 09 02:46:20 user nova-compute[72351]: DEBUG nova.objects.instance [None req-878c4434-bb6a-462a-9c30-5ec198e406ef tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lazy-loading 'resources' on Instance uuid c740d131-98a9-4b14-a75a-a33aa7c72b51 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:46:20 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-878c4434-bb6a-462a-9c30-5ec198e406ef tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,compute_id=1,config_drive='',created_at=2023-08-09T02:39:51Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-server-1075911547',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumeshelvetestjson-server-1075911547',id=45,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBAhm3wxNXc3qjoZ/WjO45nlSmWKHkz9mk2gcB1iSKP8F60j8gzonzjkrypbEG5qNiTKrTFWzLZuC0FgGJv+9yI6H0VXa1JB0Qe7iMGyt2g59NXDdngUdeDSufXPbcrFG6Q==',key_name='tempest-keypair-1369286576',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:43:58Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='5ff300d27c9a4dd3b7ecd507b3627d98',ramdisk_id='',reservation_id='r-gx6s8m4c',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeShelveTestJSON-1982056968',owner_user_name='tempest-AttachVolumeShelveTestJSON-1982056968-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:43:59Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='ad8fc03436a84bdfb13ba6abecd4f27b',uuid=c740d131-98a9-4b14-a75a-a33aa7c72b51,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "ef71c011-3c08-4927-bf2b-763a63fe8c6f", "address": "fa:16:3e:73:9f:90", "network": {"id": "da9be96c-ce20-4037-999c-230ca799fb06", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-661780305-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.69", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5ff300d27c9a4dd3b7ecd507b3627d98", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapef71c011-3c", "ovs_interfaceid": "ef71c011-3c08-4927-bf2b-763a63fe8c6f", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:46:20 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-878c4434-bb6a-462a-9c30-5ec198e406ef tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Converting VIF {"id": "ef71c011-3c08-4927-bf2b-763a63fe8c6f", "address": "fa:16:3e:73:9f:90", "network": {"id": "da9be96c-ce20-4037-999c-230ca799fb06", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-661780305-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.69", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5ff300d27c9a4dd3b7ecd507b3627d98", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapef71c011-3c", "ovs_interfaceid": "ef71c011-3c08-4927-bf2b-763a63fe8c6f", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:46:20 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-878c4434-bb6a-462a-9c30-5ec198e406ef tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:73:9f:90,bridge_name='br-int',has_traffic_filtering=True,id=ef71c011-3c08-4927-bf2b-763a63fe8c6f,network=Network(da9be96c-ce20-4037-999c-230ca799fb06),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapef71c011-3c') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:46:20 user nova-compute[72351]: DEBUG os_vif [None req-878c4434-bb6a-462a-9c30-5ec198e406ef tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:73:9f:90,bridge_name='br-int',has_traffic_filtering=True,id=ef71c011-3c08-4927-bf2b-763a63fe8c6f,network=Network(da9be96c-ce20-4037-999c-230ca799fb06),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapef71c011-3c') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:46:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:46:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapef71c011-3c, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:46:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:46:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:46:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:46:20 user nova-compute[72351]: INFO os_vif [None req-878c4434-bb6a-462a-9c30-5ec198e406ef tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:73:9f:90,bridge_name='br-int',has_traffic_filtering=True,id=ef71c011-3c08-4927-bf2b-763a63fe8c6f,network=Network(da9be96c-ce20-4037-999c-230ca799fb06),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapef71c011-3c')
Aug 09 02:46:20 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-878c4434-bb6a-462a-9c30-5ec198e406ef tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Deleting instance files /opt/stack/data/nova/instances/c740d131-98a9-4b14-a75a-a33aa7c72b51_del
Aug 09 02:46:20 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-878c4434-bb6a-462a-9c30-5ec198e406ef tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Deletion of /opt/stack/data/nova/instances/c740d131-98a9-4b14-a75a-a33aa7c72b51_del complete
Aug 09 02:46:21 user nova-compute[72351]: INFO nova.compute.manager [None req-878c4434-bb6a-462a-9c30-5ec198e406ef tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Took 0.90 seconds to destroy the instance on the hypervisor.
Aug 09 02:46:21 user nova-compute[72351]: DEBUG oslo.service.loopingcall [None req-878c4434-bb6a-462a-9c30-5ec198e406ef tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72351) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Aug 09 02:46:21 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Periodic task is updating the host stats, it is trying to get disk info for instance-0000002d, but the backing disk storage was removed by a concurrent operation such as resize. Error: No disk at /opt/stack/data/nova/instances/c740d131-98a9-4b14-a75a-a33aa7c72b51/disk: nova.exception.DiskNotFound: No disk at /opt/stack/data/nova/instances/c740d131-98a9-4b14-a75a-a33aa7c72b51/disk
Aug 09 02:46:21 user nova-compute[72351]: DEBUG nova.compute.manager [-] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Deallocating network for instance {{(pid=72351) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2263}}
Aug 09 02:46:21 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] deallocate_for_instance() {{(pid=72351) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1795}}
Aug 09 02:46:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:46:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:46:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:46:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:46:21 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdb (vdb) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:46:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e020e9e8-2372-40c9-ab34-6f65a529a995/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:46:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e020e9e8-2372-40c9-ab34-6f65a529a995/disk --force-share --output=json" returned: 0 in 0.129s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:46:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e020e9e8-2372-40c9-ab34-6f65a529a995/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:46:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e020e9e8-2372-40c9-ab34-6f65a529a995/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:46:22 user nova-compute[72351]: DEBUG nova.compute.manager [req-286b10fb-82fa-4653-a8cf-db26186818d2 req-a5a7c801-badc-47cd-a463-044bc4b5a640 service nova] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Received event network-vif-deleted-ef71c011-3c08-4927-bf2b-763a63fe8c6f {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:46:22 user nova-compute[72351]: INFO nova.compute.manager [req-286b10fb-82fa-4653-a8cf-db26186818d2 req-a5a7c801-badc-47cd-a463-044bc4b5a640 service nova] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Neutron deleted interface ef71c011-3c08-4927-bf2b-763a63fe8c6f; detaching it from the instance and deleting it from the info cache
Aug 09 02:46:22 user nova-compute[72351]: DEBUG nova.network.neutron [req-286b10fb-82fa-4653-a8cf-db26186818d2 req-a5a7c801-badc-47cd-a463-044bc4b5a640 service nova] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:46:22 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:46:22 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Took 1.02 seconds to deallocate network for instance.
Aug 09 02:46:22 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:46:22 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:46:22 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Hypervisor/Node resource view: name=user free_ram=8653MB free_disk=25.23220443725586GB free_vcpus=10 pci_devices=[{"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72351) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Aug 09 02:46:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:46:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:46:22 user nova-compute[72351]: DEBUG nova.compute.manager [req-286b10fb-82fa-4653-a8cf-db26186818d2 req-a5a7c801-badc-47cd-a463-044bc4b5a640 service nova] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Detach interface failed, port_id=ef71c011-3c08-4927-bf2b-763a63fe8c6f, reason: Instance c740d131-98a9-4b14-a75a-a33aa7c72b51 could not be found. {{(pid=72351) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10866}}
Aug 09 02:46:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-878c4434-bb6a-462a-9c30-5ec198e406ef tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:46:22 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance c740d131-98a9-4b14-a75a-a33aa7c72b51 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:46:22 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance c6ae885f-dce2-4dc2-98d5-26139b8fb67e actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:46:22 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance e020e9e8-2372-40c9-ab34-6f65a529a995 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:46:22 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Total usable vcpus: 12, total allocated vcpus: 3 {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Aug 09 02:46:22 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Final resource view: name=user phys_ram=16011MB used_ram=2048MB phys_disk=40GB used_disk=3GB total_vcpus=12 used_vcpus=3 pci_stats=[] {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Aug 09 02:46:22 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:46:22 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:46:22 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Compute_service record updated for user:user {{(pid=72351) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Aug 09 02:46:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.240s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:46:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-878c4434-bb6a-462a-9c30-5ec198e406ef tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.198s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:46:22 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-878c4434-bb6a-462a-9c30-5ec198e406ef tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:46:22 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-878c4434-bb6a-462a-9c30-5ec198e406ef tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:46:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-878c4434-bb6a-462a-9c30-5ec198e406ef tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.159s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:46:22 user nova-compute[72351]: INFO nova.scheduler.client.report [None req-878c4434-bb6a-462a-9c30-5ec198e406ef tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Deleted allocations for instance c740d131-98a9-4b14-a75a-a33aa7c72b51
Aug 09 02:46:22 user nova-compute[72351]: DEBUG nova.compute.manager [req-8f7a639a-86bd-46d9-8a62-7198c87642b1 req-e50d2afe-e3f3-4914-84d0-448775f35735 service nova] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Received event network-vif-plugged-ef71c011-3c08-4927-bf2b-763a63fe8c6f {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:46:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-8f7a639a-86bd-46d9-8a62-7198c87642b1 req-e50d2afe-e3f3-4914-84d0-448775f35735 service nova] Acquiring lock "c740d131-98a9-4b14-a75a-a33aa7c72b51-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:46:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-8f7a639a-86bd-46d9-8a62-7198c87642b1 req-e50d2afe-e3f3-4914-84d0-448775f35735 service nova] Lock "c740d131-98a9-4b14-a75a-a33aa7c72b51-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:46:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-8f7a639a-86bd-46d9-8a62-7198c87642b1 req-e50d2afe-e3f3-4914-84d0-448775f35735 service nova] Lock "c740d131-98a9-4b14-a75a-a33aa7c72b51-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:46:22 user nova-compute[72351]: DEBUG nova.compute.manager [req-8f7a639a-86bd-46d9-8a62-7198c87642b1 req-e50d2afe-e3f3-4914-84d0-448775f35735 service nova] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] No waiting events found dispatching network-vif-plugged-ef71c011-3c08-4927-bf2b-763a63fe8c6f {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:46:22 user nova-compute[72351]: WARNING nova.compute.manager [req-8f7a639a-86bd-46d9-8a62-7198c87642b1 req-e50d2afe-e3f3-4914-84d0-448775f35735 service nova] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Received unexpected event network-vif-plugged-ef71c011-3c08-4927-bf2b-763a63fe8c6f for instance with vm_state deleted and task_state None.
Aug 09 02:46:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-878c4434-bb6a-462a-9c30-5ec198e406ef tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "c740d131-98a9-4b14-a75a-a33aa7c72b51" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.470s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:46:22 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:46:22 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:46:22 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:46:25 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:46:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:46:29 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691549174.3875582, c6ae885f-dce2-4dc2-98d5-26139b8fb67e => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:46:29 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] VM Stopped (Lifecycle Event)
Aug 09 02:46:29 user nova-compute[72351]: DEBUG nova.compute.manager [None req-f034c3e6-3c34-47a3-a71a-9116caa15310 None None] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:46:29 user nova-compute[72351]: DEBUG nova.compute.manager [None req-f034c3e6-3c34-47a3-a71a-9116caa15310 None None] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: stopped, current task_state: None, current DB power_state: 4, VM power_state: 4 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:46:30 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:46:30 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:46:30 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5003 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 02:46:30 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:46:30 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:46:30 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:46:33 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:46:35 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691549180.8938053, c740d131-98a9-4b14-a75a-a33aa7c72b51 => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:46:35 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] VM Stopped (Lifecycle Event)
Aug 09 02:46:35 user nova-compute[72351]: DEBUG nova.compute.manager [None req-9b185ac6-ee14-40b6-ae95-d6fe7c7a10d4 None None] [instance: c740d131-98a9-4b14-a75a-a33aa7c72b51] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:46:35 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:46:40 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:46:45 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:46:48 user nova-compute[72351]: DEBUG nova.compute.manager [req-30664cdb-8481-4a46-8ce6-0650600c7b48 req-5f989790-708f-440a-b325-5cab90696804 service nova] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Received event network-changed-94c7c88f-9e5e-4e25-b7b8-59fb82e25f52 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:46:48 user nova-compute[72351]: DEBUG nova.compute.manager [req-30664cdb-8481-4a46-8ce6-0650600c7b48 req-5f989790-708f-440a-b325-5cab90696804 service nova] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Refreshing instance network info cache due to event network-changed-94c7c88f-9e5e-4e25-b7b8-59fb82e25f52. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:46:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-30664cdb-8481-4a46-8ce6-0650600c7b48 req-5f989790-708f-440a-b325-5cab90696804 service nova] Acquiring lock "refresh_cache-e020e9e8-2372-40c9-ab34-6f65a529a995" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:46:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-30664cdb-8481-4a46-8ce6-0650600c7b48 req-5f989790-708f-440a-b325-5cab90696804 service nova] Acquired lock "refresh_cache-e020e9e8-2372-40c9-ab34-6f65a529a995" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:46:48 user nova-compute[72351]: DEBUG nova.network.neutron [req-30664cdb-8481-4a46-8ce6-0650600c7b48 req-5f989790-708f-440a-b325-5cab90696804 service nova] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Refreshing network info cache for port 94c7c88f-9e5e-4e25-b7b8-59fb82e25f52 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:46:48 user nova-compute[72351]: DEBUG nova.network.neutron [req-30664cdb-8481-4a46-8ce6-0650600c7b48 req-5f989790-708f-440a-b325-5cab90696804 service nova] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Updated VIF entry in instance network info cache for port 94c7c88f-9e5e-4e25-b7b8-59fb82e25f52. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:46:48 user nova-compute[72351]: DEBUG nova.network.neutron [req-30664cdb-8481-4a46-8ce6-0650600c7b48 req-5f989790-708f-440a-b325-5cab90696804 service nova] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Updating instance_info_cache with network_info: [{"id": "94c7c88f-9e5e-4e25-b7b8-59fb82e25f52", "address": "fa:16:3e:c1:3e:66", "network": {"id": "47c43230-a244-4198-84e4-9ed98c005cd9", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1392403106-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.151", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c07a6b254414b41a673468238d3aa62", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap94c7c88f-9e", "ovs_interfaceid": "94c7c88f-9e5e-4e25-b7b8-59fb82e25f52", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:46:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-30664cdb-8481-4a46-8ce6-0650600c7b48 req-5f989790-708f-440a-b325-5cab90696804 service nova] Releasing lock "refresh_cache-e020e9e8-2372-40c9-ab34-6f65a529a995" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:46:48 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:46:50 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:46:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-49bc04b0-1687-44ce-ba30-7d3252509440 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Acquiring lock "e020e9e8-2372-40c9-ab34-6f65a529a995" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:46:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-49bc04b0-1687-44ce-ba30-7d3252509440 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "e020e9e8-2372-40c9-ab34-6f65a529a995" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:46:52 user nova-compute[72351]: DEBUG nova.objects.instance [None req-49bc04b0-1687-44ce-ba30-7d3252509440 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lazy-loading 'flavor' on Instance uuid e020e9e8-2372-40c9-ab34-6f65a529a995 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:46:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-49bc04b0-1687-44ce-ba30-7d3252509440 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "e020e9e8-2372-40c9-ab34-6f65a529a995" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.052s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:46:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-49bc04b0-1687-44ce-ba30-7d3252509440 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Acquiring lock "e020e9e8-2372-40c9-ab34-6f65a529a995" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:46:52 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-49bc04b0-1687-44ce-ba30-7d3252509440 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "e020e9e8-2372-40c9-ab34-6f65a529a995" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:46:52 user nova-compute[72351]: INFO nova.compute.manager [None req-49bc04b0-1687-44ce-ba30-7d3252509440 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Attaching volume d03e3f57-d947-486e-ae6b-60513290548b to /dev/vdb
Aug 09 02:46:52 user nova-compute[72351]: DEBUG os_brick.utils [None req-49bc04b0-1687-44ce-ba30-7d3252509440 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:46:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:46:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:46:52 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[96fa9da9-0f0c-47e3-95c7-e69341effbee]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:a52269602ff4\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:46:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:46:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.015s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:46:52 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[9ba50ab9-9257-4d26-a390-92d5e3ea8443]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:46:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:46:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.016s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:46:52 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[2eb557ce-81b9-445e-97aa-277b3f9ddcdd]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:46:52 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[dbe547c9-803a-47c1-9ac9-8e7309429a21]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:46:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-49bc04b0-1687-44ce-ba30-7d3252509440 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Running cmd (subprocess): nvme version {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:46:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-49bc04b0-1687-44ce-ba30-7d3252509440 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] 'nvme version' failed. Not Retrying. {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:46:52 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.nvmeof [None req-49bc04b0-1687-44ce-ba30-7d3252509440 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] nvme not present on system {{(pid=72351) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Aug 09 02:46:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:46:52 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:46:52 user nova-compute[72351]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Aug 09 02:46:52 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[84e76a07-ce52-4837-99db-ec0032740163]: (4, '') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:46:52 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-49bc04b0-1687-44ce-ba30-7d3252509440 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72351) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Aug 09 02:46:52 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-49bc04b0-1687-44ce-ba30-7d3252509440 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Aug 09 02:46:52 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-49bc04b0-1687-44ce-ba30-7d3252509440 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] LIGHTOS: no hostnqn found. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Aug 09 02:46:52 user nova-compute[72351]: DEBUG os_brick.utils [None req-49bc04b0-1687-44ce-ba30-7d3252509440 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] <== get_connector_properties: return (119ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:a52269602ff4', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:46:52 user nova-compute[72351]: DEBUG nova.virt.block_device [None req-49bc04b0-1687-44ce-ba30-7d3252509440 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Updating existing volume attachment record: dc999260-4537-4190-af27-151237f81b86 {{(pid=72351) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Aug 09 02:46:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:46:55 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:46:58 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:47:00 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-49bc04b0-1687-44ce-ba30-7d3252509440 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 02:47:00 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-49bc04b0-1687-44ce-ba30-7d3252509440 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:d03e3f57-d947-486e-ae6b-60513290548b'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'AyJIXKCa', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:47:00 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-49bc04b0-1687-44ce-ba30-7d3252509440 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:47:00 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-49bc04b0-1687-44ce-ba30-7d3252509440 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:47:00 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-49bc04b0-1687-44ce-ba30-7d3252509440 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:47:00 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-49bc04b0-1687-44ce-ba30-7d3252509440 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:47:00 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-49bc04b0-1687-44ce-ba30-7d3252509440 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:d03e3f57-d947-486e-ae6b-60513290548b" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:47:00 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-49bc04b0-1687-44ce-ba30-7d3252509440 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:d03e3f57-d947-486e-ae6b-60513290548b" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:47:00 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-49bc04b0-1687-44ce-ba30-7d3252509440 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 02:47:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d03e3f57-d947-486e-ae6b-60513290548b -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:47:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d03e3f57-d947-486e-ae6b-60513290548b -p 172.16.0.220:3260" returned: 21 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:47:00 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d9ea98ac-2b50-4dc6-8a19-0d674f88a612]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:47:00 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-49bc04b0-1687-44ce-ba30-7d3252509440 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Aug 09 02:47:00 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:47:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d03e3f57-d947-486e-ae6b-60513290548b -p 172.16.0.220:3260 --interface default --op new {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:47:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d03e3f57-d947-486e-ae6b-60513290548b -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:47:00 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[57885b5f-38d9-4dff-9dd1-3d6d83434ba5]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:d03e3f57-d947-486e-ae6b-60513290548b] added\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:47:00 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-49bc04b0-1687-44ce-ba30-7d3252509440 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:d03e3f57-d947-486e-ae6b-60513290548b] added
Aug 09 02:47:00 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:47:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d03e3f57-d947-486e-ae6b-60513290548b -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:47:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d03e3f57-d947-486e-ae6b-60513290548b -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:47:00 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[9d9944ae-6d7a-4226-a470-9e230b7d0adc]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:47:00 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-49bc04b0-1687-44ce-ba30-7d3252509440 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:47:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d03e3f57-d947-486e-ae6b-60513290548b -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:47:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d03e3f57-d947-486e-ae6b-60513290548b -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.016s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:47:00 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[0ce4a52a-8a14-4a68-911b-6f8b3de899da]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:47:00 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-49bc04b0-1687-44ce-ba30-7d3252509440 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:47:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d03e3f57-d947-486e-ae6b-60513290548b -p 172.16.0.220:3260 --op update -n node.session.auth.username -v AyJIXKCa {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:47:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d03e3f57-d947-486e-ae6b-60513290548b -p 172.16.0.220:3260 --op update -n node.session.auth.username -v AyJIXKCa" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:47:00 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[4e8ff38f-19ad-4e04-b31e-683ef124a54e]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:47:00 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-49bc04b0-1687-44ce-ba30-7d3252509440 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'AyJIXKCa'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:47:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d03e3f57-d947-486e-ae6b-60513290548b -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:47:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d03e3f57-d947-486e-ae6b-60513290548b -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:47:00 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e05a8318-68f4-4148-bf79-71c00924d88c]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:47:00 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-49bc04b0-1687-44ce-ba30-7d3252509440 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:47:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:47:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:47:00 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[805b5fbe-c08b-4673-9857-e1f2c8b4fb8f]: (4, ('tcp: [58] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:47:00 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-49bc04b0-1687-44ce-ba30-7d3252509440 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [58] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 (non-flash)
Aug 09 02:47:00 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:47:00 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-49bc04b0-1687-44ce-ba30-7d3252509440 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] iscsi session list stdout=tcp: [58] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 (non-flash)
Aug 09 02:47:00 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:47:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d03e3f57-d947-486e-ae6b-60513290548b -p 172.16.0.220:3260 --login {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:47:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d03e3f57-d947-486e-ae6b-60513290548b -p 172.16.0.220:3260 --login" returned: 0 in 0.056s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:47:00 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[43680644-4cca-4de8-9a8b-9c0d2344bdb8]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:d03e3f57-d947-486e-ae6b-60513290548b, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:d03e3f57-d947-486e-ae6b-60513290548b, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:47:00 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-49bc04b0-1687-44ce-ba30-7d3252509440 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:d03e3f57-d947-486e-ae6b-60513290548b, portal: 172.16.0.220,3260]
Aug 09 02:47:00 user nova-compute[72351]: Login to [iface: default, target: iqn.2010-10.org.openstack:d03e3f57-d947-486e-ae6b-60513290548b, portal: 172.16.0.220,3260] successful.
Aug 09 02:47:00 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:47:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d03e3f57-d947-486e-ae6b-60513290548b -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:47:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d03e3f57-d947-486e-ae6b-60513290548b -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:47:00 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d7c653c7-78d3-4b61-bfab-0946c5a482fe]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:47:00 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-49bc04b0-1687-44ce-ba30-7d3252509440 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:47:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:47:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:47:00 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[1e14c122-c82f-4c86-a407-1e20c7509c18]: (4, ('tcp: [58] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 (non-flash)\ntcp: [59] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d03e3f57-d947-486e-ae6b-60513290548b (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:47:00 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-49bc04b0-1687-44ce-ba30-7d3252509440 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [58] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 (non-flash)
Aug 09 02:47:00 user nova-compute[72351]: tcp: [59] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d03e3f57-d947-486e-ae6b-60513290548b (non-flash)
Aug 09 02:47:00 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:47:00 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-49bc04b0-1687-44ce-ba30-7d3252509440 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] iscsi session list stdout=tcp: [58] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 (non-flash)
Aug 09 02:47:00 user nova-compute[72351]: tcp: [59] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d03e3f57-d947-486e-ae6b-60513290548b (non-flash)
Aug 09 02:47:00 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:47:00 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-49bc04b0-1687-44ce-ba30-7d3252509440 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:d03e3f57-d947-486e-ae6b-60513290548b" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.256s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:47:00 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-49bc04b0-1687-44ce-ba30-7d3252509440 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 02:47:00 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-49bc04b0-1687-44ce-ba30-7d3252509440 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] HCTL ('34', '-', '-', 0) found on session 59 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 02:47:00 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-49bc04b0-1687-44ce-ba30-7d3252509440 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Scanning host 34 c: -, t: -, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 02:47:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:47:00 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:47:00 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[33365428-08b5-47a3-bc1a-13ebbe452c3c]: (4, ('- - 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:47:00 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-49bc04b0-1687-44ce-ba30-7d3252509440 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Searching for a device in session 59 and hctl ['34', '*', '*', 0] yield: None {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:47:00 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:47:01 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-49bc04b0-1687-44ce-ba30-7d3252509440 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Searching for a device in session 59 and hctl ['34', '*', '*', 0] yield: sdc {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:47:01 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-49bc04b0-1687-44ce-ba30-7d3252509440 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Connected to sdc using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'AyJIXKCa', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 02:47:01 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-49bc04b0-1687-44ce-ba30-7d3252509440 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.287s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:47:01 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-49bc04b0-1687-44ce-ba30-7d3252509440 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] <== connect_volume: return (1288ms) {'type': 'block', 'scsi_wwn': '26561356534363739', 'path': '/dev/sdc'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:47:01 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-49bc04b0-1687-44ce-ba30-7d3252509440 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26561356534363739', 'path': '/dev/sdc'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 02:47:01 user nova-compute[72351]: DEBUG nova.objects.instance [None req-49bc04b0-1687-44ce-ba30-7d3252509440 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lazy-loading 'flavor' on Instance uuid e020e9e8-2372-40c9-ab34-6f65a529a995 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:47:01 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-49bc04b0-1687-44ce-ba30-7d3252509440 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] attach device xml: <disk type="block" device="disk">
Aug 09 02:47:01 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:47:01 user nova-compute[72351]:   <source dev="/dev/sdc"/>
Aug 09 02:47:01 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:47:01 user nova-compute[72351]:   <serial>d03e3f57-d947-486e-ae6b-60513290548b</serial>
Aug 09 02:47:01 user nova-compute[72351]: </disk>
Aug 09 02:47:01 user nova-compute[72351]:  {{(pid=72351) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Aug 09 02:47:01 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-49bc04b0-1687-44ce-ba30-7d3252509440 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:47:01 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-49bc04b0-1687-44ce-ba30-7d3252509440 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:47:01 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-49bc04b0-1687-44ce-ba30-7d3252509440 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] No VIF found with MAC fa:16:3e:c1:3e:66, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:47:01 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-49bc04b0-1687-44ce-ba30-7d3252509440 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "e020e9e8-2372-40c9-ab34-6f65a529a995" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 9.740s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:47:03 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-81f727bc-46ed-4343-99d2-96fcbde73a9f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Acquiring lock "e020e9e8-2372-40c9-ab34-6f65a529a995" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:47:03 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-81f727bc-46ed-4343-99d2-96fcbde73a9f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "e020e9e8-2372-40c9-ab34-6f65a529a995" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:47:03 user nova-compute[72351]: INFO nova.compute.manager [None req-81f727bc-46ed-4343-99d2-96fcbde73a9f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Detaching volume d03e3f57-d947-486e-ae6b-60513290548b
Aug 09 02:47:03 user nova-compute[72351]: INFO nova.virt.block_device [None req-81f727bc-46ed-4343-99d2-96fcbde73a9f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Attempting to driver detach volume d03e3f57-d947-486e-ae6b-60513290548b from mountpoint /dev/vdb
Aug 09 02:47:03 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-81f727bc-46ed-4343-99d2-96fcbde73a9f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Attempting to detach device vdb from instance e020e9e8-2372-40c9-ab34-6f65a529a995 from the persistent domain config. {{(pid=72351) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2477}}
Aug 09 02:47:03 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-81f727bc-46ed-4343-99d2-96fcbde73a9f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 02:47:03 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:47:03 user nova-compute[72351]:   <source dev="/dev/sdc"/>
Aug 09 02:47:03 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:47:03 user nova-compute[72351]:   <serial>d03e3f57-d947-486e-ae6b-60513290548b</serial>
Aug 09 02:47:03 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Aug 09 02:47:03 user nova-compute[72351]: </disk>
Aug 09 02:47:03 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 02:47:03 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-81f727bc-46ed-4343-99d2-96fcbde73a9f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Successfully detached device vdb from instance e020e9e8-2372-40c9-ab34-6f65a529a995 from the persistent domain config.
Aug 09 02:47:03 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-81f727bc-46ed-4343-99d2-96fcbde73a9f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance e020e9e8-2372-40c9-ab34-6f65a529a995 from the live domain config. {{(pid=72351) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2513}}
Aug 09 02:47:03 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-81f727bc-46ed-4343-99d2-96fcbde73a9f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 02:47:03 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:47:03 user nova-compute[72351]:   <source dev="/dev/sdc"/>
Aug 09 02:47:03 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:47:03 user nova-compute[72351]:   <serial>d03e3f57-d947-486e-ae6b-60513290548b</serial>
Aug 09 02:47:03 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Aug 09 02:47:03 user nova-compute[72351]: </disk>
Aug 09 02:47:03 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 02:47:03 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Received event <DeviceRemovedEvent: 1691549223.8927605, e020e9e8-2372-40c9-ab34-6f65a529a995 => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=72351) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2360}}
Aug 09 02:47:03 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-81f727bc-46ed-4343-99d2-96fcbde73a9f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance e020e9e8-2372-40c9-ab34-6f65a529a995 {{(pid=72351) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2589}}
Aug 09 02:47:03 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-81f727bc-46ed-4343-99d2-96fcbde73a9f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Successfully detached device vdb from instance e020e9e8-2372-40c9-ab34-6f65a529a995 from the live domain config.
Aug 09 02:47:03 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-81f727bc-46ed-4343-99d2-96fcbde73a9f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 02:47:03 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-81f727bc-46ed-4343-99d2-96fcbde73a9f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:d03e3f57-d947-486e-ae6b-60513290548b'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'AyJIXKCa', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc'}, None), 'kwargs': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:47:03 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-81f727bc-46ed-4343-99d2-96fcbde73a9f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:47:03 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-81f727bc-46ed-4343-99d2-96fcbde73a9f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:47:03 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-81f727bc-46ed-4343-99d2-96fcbde73a9f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:47:03 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-81f727bc-46ed-4343-99d2-96fcbde73a9f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:d03e3f57-d947-486e-ae6b-60513290548b', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 02:47:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:47:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:47:03 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a76b9d2f-dc41-4d7e-b72f-f267f5a44c8a]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:d03e3f57-d947-486e-ae6b-60513290548b\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:47:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:47:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:47:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:47:03 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[9d90b6e3-1b51-4621-bd68-74e6d9ffedc7]: (4, ('tcp: [58] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 (non-flash)\ntcp: [59] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d03e3f57-d947-486e-ae6b-60513290548b (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:47:03 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-81f727bc-46ed-4343-99d2-96fcbde73a9f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [58] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 (non-flash)
Aug 09 02:47:03 user nova-compute[72351]: tcp: [59] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d03e3f57-d947-486e-ae6b-60513290548b (non-flash)
Aug 09 02:47:03 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:47:03 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-81f727bc-46ed-4343-99d2-96fcbde73a9f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] iscsi session list stdout=tcp: [58] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 (non-flash)
Aug 09 02:47:03 user nova-compute[72351]: tcp: [59] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d03e3f57-d947-486e-ae6b-60513290548b (non-flash)
Aug 09 02:47:03 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:47:03 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-81f727bc-46ed-4343-99d2-96fcbde73a9f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fcfbc707760>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:d03e3f57-d947-486e-ae6b-60513290548b'): ({'sdc'}, set())}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 02:47:03 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-81f727bc-46ed-4343-99d2-96fcbde73a9f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Removing single pathed devices sdc {{(pid=72351) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Aug 09 02:47:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:47:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:47:03 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[b61d1cd7-b24f-43f8-9d9e-b9528ff62c71]: (4, ('path checker states:\nup                  2\n\npaths: 0\nbusy: False\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:47:03 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdc {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:47:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdc" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:47:04 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[161cf756-767a-437e-a7a0-132b352ab41f]: (4, ('ok\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:47:04 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-81f727bc-46ed-4343-99d2-96fcbde73a9f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Flushing IO for device /dev/sdc {{(pid=72351) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Aug 09 02:47:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdc {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:47:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdc" returned: 0 in 0.013s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:47:04 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[8baef6d9-fb43-422c-973a-cf0c080b5e25]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:47:04 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-81f727bc-46ed-4343-99d2-96fcbde73a9f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Remove SCSI device /dev/sdc with /sys/block/sdc/device/delete {{(pid=72351) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Aug 09 02:47:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdc/device/delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:47:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdc/device/delete" returned: 0 in 0.057s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:47:04 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[8f441f70-28af-46cd-bbf9-c930d0ae0b86]: (4, ('1', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:47:04 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-81f727bc-46ed-4343-99d2-96fcbde73a9f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Checking to see if SCSI volumes sdc have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Aug 09 02:47:04 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-81f727bc-46ed-4343-99d2-96fcbde73a9f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] SCSI volumes sdc have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Aug 09 02:47:04 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-81f727bc-46ed-4343-99d2-96fcbde73a9f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:d03e3f57-d947-486e-ae6b-60513290548b')] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 02:47:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d03e3f57-d947-486e-ae6b-60513290548b -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:47:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d03e3f57-d947-486e-ae6b-60513290548b -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.025s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:47:04 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[2d943bcf-cec4-4355-8945-3a0b3cb12baf]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:47:04 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-81f727bc-46ed-4343-99d2-96fcbde73a9f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:47:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d03e3f57-d947-486e-ae6b-60513290548b -p 172.16.0.220:3260 --logout {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:47:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d03e3f57-d947-486e-ae6b-60513290548b -p 172.16.0.220:3260 --logout" returned: 0 in 0.059s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:47:04 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[63b00a80-3b54-4b58-af23-671e38af22a8]: (4, ('Logging out of session [sid: 59, target: iqn.2010-10.org.openstack:d03e3f57-d947-486e-ae6b-60513290548b, portal: 172.16.0.220,3260]\nLogout of [sid: 59, target: iqn.2010-10.org.openstack:d03e3f57-d947-486e-ae6b-60513290548b, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:47:04 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-81f727bc-46ed-4343-99d2-96fcbde73a9f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 59, target: iqn.2010-10.org.openstack:d03e3f57-d947-486e-ae6b-60513290548b, portal: 172.16.0.220,3260]
Aug 09 02:47:04 user nova-compute[72351]: Logout of [sid: 59, target: iqn.2010-10.org.openstack:d03e3f57-d947-486e-ae6b-60513290548b, portal: 172.16.0.220,3260] successful.
Aug 09 02:47:04 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:47:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d03e3f57-d947-486e-ae6b-60513290548b -p 172.16.0.220:3260 --op delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:47:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d03e3f57-d947-486e-ae6b-60513290548b -p 172.16.0.220:3260 --op delete" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:47:04 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[381a9bc7-5f04-48f7-8447-ab8fea6e6e94]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:47:04 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-81f727bc-46ed-4343-99d2-96fcbde73a9f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:47:04 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-81f727bc-46ed-4343-99d2-96fcbde73a9f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.287s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:47:04 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-81f727bc-46ed-4343-99d2-96fcbde73a9f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] <== disconnect_volume: return (289ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:47:04 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-81f727bc-46ed-4343-99d2-96fcbde73a9f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 02:47:04 user nova-compute[72351]: DEBUG nova.objects.instance [None req-81f727bc-46ed-4343-99d2-96fcbde73a9f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lazy-loading 'flavor' on Instance uuid e020e9e8-2372-40c9-ab34-6f65a529a995 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:47:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-81f727bc-46ed-4343-99d2-96fcbde73a9f tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "e020e9e8-2372-40c9-ab34-6f65a529a995" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.644s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:47:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:47:10 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:47:11 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:47:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Acquiring lock "150f1b69-aa23-4702-a9f4-24025963befe" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:47:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "150f1b69-aa23-4702-a9f4-24025963befe" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:47:13 user nova-compute[72351]: DEBUG nova.compute.manager [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Starting instance... {{(pid=72351) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2406}}
Aug 09 02:47:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:47:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:47:13 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72351) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Aug 09 02:47:13 user nova-compute[72351]: INFO nova.compute.claims [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Claim successful on node user
Aug 09 02:47:13 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:47:13 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:47:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.228s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:47:13 user nova-compute[72351]: DEBUG nova.compute.manager [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Start building networks asynchronously for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2803}}
Aug 09 02:47:13 user nova-compute[72351]: DEBUG nova.compute.manager [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Allocating IP information in the background. {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1956}}
Aug 09 02:47:13 user nova-compute[72351]: DEBUG nova.network.neutron [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] allocate_for_instance() {{(pid=72351) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Aug 09 02:47:13 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Aug 09 02:47:14 user nova-compute[72351]: DEBUG nova.compute.manager [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Start building block device mappings for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2838}}
Aug 09 02:47:14 user nova-compute[72351]: DEBUG nova.compute.manager [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Start spawning the instance on the hypervisor. {{(pid=72351) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2612}}
Aug 09 02:47:14 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Creating instance directory {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4717}}
Aug 09 02:47:14 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Creating image(s)
Aug 09 02:47:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Acquiring lock "/opt/stack/data/nova/instances/150f1b69-aa23-4702-a9f4-24025963befe/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:47:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "/opt/stack/data/nova/instances/150f1b69-aa23-4702-a9f4-24025963befe/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:47:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "/opt/stack/data/nova/instances/150f1b69-aa23-4702-a9f4-24025963befe/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:47:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:47:14 user nova-compute[72351]: DEBUG nova.policy [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'ad8fc03436a84bdfb13ba6abecd4f27b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ff300d27c9a4dd3b7ecd507b3627d98', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72351) authorize /opt/stack/nova/nova/policy.py:203}}
Aug 09 02:47:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.132s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:47:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Acquiring lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:47:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:47:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:47:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.135s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:47:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/150f1b69-aa23-4702-a9f4-24025963befe/disk 1073741824 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:47:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/150f1b69-aa23-4702-a9f4-24025963befe/disk 1073741824" returned: 0 in 0.057s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:47:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.199s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:47:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:47:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.130s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:47:14 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Checking if we can resize image /opt/stack/data/nova/instances/150f1b69-aa23-4702-a9f4-24025963befe/disk. size=1073741824 {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Aug 09 02:47:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/150f1b69-aa23-4702-a9f4-24025963befe/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:47:14 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/150f1b69-aa23-4702-a9f4-24025963befe/disk --force-share --output=json" returned: 0 in 0.129s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:47:14 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Cannot resize image /opt/stack/data/nova/instances/150f1b69-aa23-4702-a9f4-24025963befe/disk to a smaller size. {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Aug 09 02:47:14 user nova-compute[72351]: DEBUG nova.objects.instance [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lazy-loading 'migration_context' on Instance uuid 150f1b69-aa23-4702-a9f4-24025963befe {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:47:14 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Created local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4851}}
Aug 09 02:47:14 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Ensure instance console log exists: /opt/stack/data/nova/instances/150f1b69-aa23-4702-a9f4-24025963befe/console.log {{(pid=72351) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4603}}
Aug 09 02:47:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:47:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:47:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:47:15 user nova-compute[72351]: DEBUG nova.network.neutron [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Successfully created port: 0983efc8-e53e-40bd-a721-2924155c20f3 {{(pid=72351) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Aug 09 02:47:15 user nova-compute[72351]: DEBUG nova.network.neutron [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Successfully updated port: 0983efc8-e53e-40bd-a721-2924155c20f3 {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:47:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Acquiring lock "refresh_cache-150f1b69-aa23-4702-a9f4-24025963befe" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:47:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Acquired lock "refresh_cache-150f1b69-aa23-4702-a9f4-24025963befe" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:47:15 user nova-compute[72351]: DEBUG nova.network.neutron [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:47:15 user nova-compute[72351]: DEBUG nova.compute.manager [req-dd83ca1d-aa06-4a81-aaf4-7249d113f8df req-0b3ee67e-fe40-4105-9ed7-9721881bd21c service nova] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Received event network-changed-0983efc8-e53e-40bd-a721-2924155c20f3 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:47:15 user nova-compute[72351]: DEBUG nova.compute.manager [req-dd83ca1d-aa06-4a81-aaf4-7249d113f8df req-0b3ee67e-fe40-4105-9ed7-9721881bd21c service nova] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Refreshing instance network info cache due to event network-changed-0983efc8-e53e-40bd-a721-2924155c20f3. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:47:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-dd83ca1d-aa06-4a81-aaf4-7249d113f8df req-0b3ee67e-fe40-4105-9ed7-9721881bd21c service nova] Acquiring lock "refresh_cache-150f1b69-aa23-4702-a9f4-24025963befe" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:47:15 user nova-compute[72351]: DEBUG nova.network.neutron [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:47:15 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:47:15 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:47:16 user nova-compute[72351]: DEBUG nova.network.neutron [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Updating instance_info_cache with network_info: [{"id": "0983efc8-e53e-40bd-a721-2924155c20f3", "address": "fa:16:3e:d7:24:ef", "network": {"id": "da9be96c-ce20-4037-999c-230ca799fb06", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-661780305-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5ff300d27c9a4dd3b7ecd507b3627d98", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0983efc8-e5", "ovs_interfaceid": "0983efc8-e53e-40bd-a721-2924155c20f3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:47:16 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Releasing lock "refresh_cache-150f1b69-aa23-4702-a9f4-24025963befe" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:47:16 user nova-compute[72351]: DEBUG nova.compute.manager [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Instance network_info: |[{"id": "0983efc8-e53e-40bd-a721-2924155c20f3", "address": "fa:16:3e:d7:24:ef", "network": {"id": "da9be96c-ce20-4037-999c-230ca799fb06", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-661780305-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5ff300d27c9a4dd3b7ecd507b3627d98", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0983efc8-e5", "ovs_interfaceid": "0983efc8-e53e-40bd-a721-2924155c20f3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1971}}
Aug 09 02:47:16 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-dd83ca1d-aa06-4a81-aaf4-7249d113f8df req-0b3ee67e-fe40-4105-9ed7-9721881bd21c service nova] Acquired lock "refresh_cache-150f1b69-aa23-4702-a9f4-24025963befe" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:47:16 user nova-compute[72351]: DEBUG nova.network.neutron [req-dd83ca1d-aa06-4a81-aaf4-7249d113f8df req-0b3ee67e-fe40-4105-9ed7-9721881bd21c service nova] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Refreshing network info cache for port 0983efc8-e53e-40bd-a721-2924155c20f3 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:47:16 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Start _get_guest_xml network_info=[{"id": "0983efc8-e53e-40bd-a721-2924155c20f3", "address": "fa:16:3e:d7:24:ef", "network": {"id": "da9be96c-ce20-4037-999c-230ca799fb06", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-661780305-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5ff300d27c9a4dd3b7ecd507b3627d98", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0983efc8-e5", "ovs_interfaceid": "0983efc8-e53e-40bd-a721-2924155c20f3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'size': 0, 'device_type': 'disk', 'encryption_secret_uuid': None, 'guest_format': None, 'encryption_options': None, 'disk_bus': 'virtio', 'encrypted': False, 'encryption_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'image_id': '18ccda7f-31d3-4978-ab9d-4ed2f74eff94'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:47:16 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:47:16 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:47:16 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:47:16 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:47:16 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:47:16 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:47:16 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:47:16 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:47:16 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:47:16 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:47:16 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:47:16 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:47:16 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:47:16 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:47:16 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:47:16 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:47:16 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:47:16 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:47:16 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:47:13Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-server-547567417',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumeshelvetestjson-server-547567417',id=49,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBC6Kt1VBrwPlXoj775IMgThdXYe/hd3G1m2PhSYFaCtMLpu/4nO1MCXo8m5UYekLtMKzV6viB1JRAwTbgfITOhrGcZO/1RTBqsi/z0vUwAuglB5045WiXDwqXuvJuthb5A==',key_name='tempest-keypair-647885492',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='5ff300d27c9a4dd3b7ecd507b3627d98',ramdisk_id='',reservation_id='r-x20xmo7b',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeShelveTestJSON-1982056968',owner_user_name='tempest-AttachVolumeShelveTestJSON-1982056968-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:47:14Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='ad8fc03436a84bdfb13ba6abecd4f27b',uuid=150f1b69-aa23-4702-a9f4-24025963befe,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "0983efc8-e53e-40bd-a721-2924155c20f3", "address": "fa:16:3e:d7:24:ef", "network": {"id": "da9be96c-ce20-4037-999c-230ca799fb06", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-661780305-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5ff300d27c9a4dd3b7ecd507b3627d98", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0983efc8-e5", "ovs_interfaceid": "0983efc8-e53e-40bd-a721-2924155c20f3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:47:16 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Converting VIF {"id": "0983efc8-e53e-40bd-a721-2924155c20f3", "address": "fa:16:3e:d7:24:ef", "network": {"id": "da9be96c-ce20-4037-999c-230ca799fb06", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-661780305-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5ff300d27c9a4dd3b7ecd507b3627d98", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0983efc8-e5", "ovs_interfaceid": "0983efc8-e53e-40bd-a721-2924155c20f3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:47:16 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:d7:24:ef,bridge_name='br-int',has_traffic_filtering=True,id=0983efc8-e53e-40bd-a721-2924155c20f3,network=Network(da9be96c-ce20-4037-999c-230ca799fb06),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap0983efc8-e5') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:47:16 user nova-compute[72351]: DEBUG nova.objects.instance [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lazy-loading 'pci_devices' on Instance uuid 150f1b69-aa23-4702-a9f4-24025963befe {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:47:16 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:47:16 user nova-compute[72351]:   <uuid>150f1b69-aa23-4702-a9f4-24025963befe</uuid>
Aug 09 02:47:16 user nova-compute[72351]:   <name>instance-00000031</name>
Aug 09 02:47:16 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:47:16 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:47:16 user nova-compute[72351]:   <metadata>
Aug 09 02:47:16 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:47:16 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:47:16 user nova-compute[72351]:       <nova:name>tempest-AttachVolumeShelveTestJSON-server-547567417</nova:name>
Aug 09 02:47:16 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:47:16</nova:creationTime>
Aug 09 02:47:16 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:47:16 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:47:16 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:47:16 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:47:16 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:47:16 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:47:16 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:47:16 user nova-compute[72351]:       <nova:owner>
Aug 09 02:47:16 user nova-compute[72351]:         <nova:user uuid="ad8fc03436a84bdfb13ba6abecd4f27b">tempest-AttachVolumeShelveTestJSON-1982056968-project-member</nova:user>
Aug 09 02:47:16 user nova-compute[72351]:         <nova:project uuid="5ff300d27c9a4dd3b7ecd507b3627d98">tempest-AttachVolumeShelveTestJSON-1982056968</nova:project>
Aug 09 02:47:16 user nova-compute[72351]:       </nova:owner>
Aug 09 02:47:16 user nova-compute[72351]:       <nova:root type="image" uuid="18ccda7f-31d3-4978-ab9d-4ed2f74eff94"/>
Aug 09 02:47:16 user nova-compute[72351]:       <nova:ports>
Aug 09 02:47:16 user nova-compute[72351]:         <nova:port uuid="0983efc8-e53e-40bd-a721-2924155c20f3">
Aug 09 02:47:16 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.7" ipVersion="4"/>
Aug 09 02:47:16 user nova-compute[72351]:         </nova:port>
Aug 09 02:47:16 user nova-compute[72351]:       </nova:ports>
Aug 09 02:47:16 user nova-compute[72351]:     </nova:instance>
Aug 09 02:47:16 user nova-compute[72351]:   </metadata>
Aug 09 02:47:16 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:47:16 user nova-compute[72351]:     <system>
Aug 09 02:47:16 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:47:16 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:47:16 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:47:16 user nova-compute[72351]:       <entry name="serial">150f1b69-aa23-4702-a9f4-24025963befe</entry>
Aug 09 02:47:16 user nova-compute[72351]:       <entry name="uuid">150f1b69-aa23-4702-a9f4-24025963befe</entry>
Aug 09 02:47:16 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:47:16 user nova-compute[72351]:     </system>
Aug 09 02:47:16 user nova-compute[72351]:   </sysinfo>
Aug 09 02:47:16 user nova-compute[72351]:   <os>
Aug 09 02:47:16 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:47:16 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:47:16 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:47:16 user nova-compute[72351]:   </os>
Aug 09 02:47:16 user nova-compute[72351]:   <features>
Aug 09 02:47:16 user nova-compute[72351]:     <acpi/>
Aug 09 02:47:16 user nova-compute[72351]:     <apic/>
Aug 09 02:47:16 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:47:16 user nova-compute[72351]:   </features>
Aug 09 02:47:16 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:47:16 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:47:16 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:47:16 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:47:16 user nova-compute[72351]:   </clock>
Aug 09 02:47:16 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:47:16 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:47:16 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:47:16 user nova-compute[72351]:   </cpu>
Aug 09 02:47:16 user nova-compute[72351]:   <devices>
Aug 09 02:47:16 user nova-compute[72351]:     <disk type="file" device="disk">
Aug 09 02:47:16 user nova-compute[72351]:       <driver name="qemu" type="qcow2" cache="none"/>
Aug 09 02:47:16 user nova-compute[72351]:       <source file="/opt/stack/data/nova/instances/150f1b69-aa23-4702-a9f4-24025963befe/disk"/>
Aug 09 02:47:16 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:47:16 user nova-compute[72351]:     </disk>
Aug 09 02:47:16 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:47:16 user nova-compute[72351]:       <mac address="fa:16:3e:d7:24:ef"/>
Aug 09 02:47:16 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:47:16 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:47:16 user nova-compute[72351]:       <target dev="tap0983efc8-e5"/>
Aug 09 02:47:16 user nova-compute[72351]:     </interface>
Aug 09 02:47:16 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:47:16 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/150f1b69-aa23-4702-a9f4-24025963befe/console.log" append="off"/>
Aug 09 02:47:16 user nova-compute[72351]:     </serial>
Aug 09 02:47:16 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:47:16 user nova-compute[72351]:     <video>
Aug 09 02:47:16 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:47:16 user nova-compute[72351]:     </video>
Aug 09 02:47:16 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:47:16 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:47:16 user nova-compute[72351]:     </rng>
Aug 09 02:47:16 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:47:16 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:47:16 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:47:16 user nova-compute[72351]:     </memballoon>
Aug 09 02:47:16 user nova-compute[72351]:   </devices>
Aug 09 02:47:16 user nova-compute[72351]: </domain>
Aug 09 02:47:16 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:47:16 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:47:13Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-server-547567417',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumeshelvetestjson-server-547567417',id=49,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBC6Kt1VBrwPlXoj775IMgThdXYe/hd3G1m2PhSYFaCtMLpu/4nO1MCXo8m5UYekLtMKzV6viB1JRAwTbgfITOhrGcZO/1RTBqsi/z0vUwAuglB5045WiXDwqXuvJuthb5A==',key_name='tempest-keypair-647885492',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='5ff300d27c9a4dd3b7ecd507b3627d98',ramdisk_id='',reservation_id='r-x20xmo7b',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeShelveTestJSON-1982056968',owner_user_name='tempest-AttachVolumeShelveTestJSON-1982056968-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:47:14Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='ad8fc03436a84bdfb13ba6abecd4f27b',uuid=150f1b69-aa23-4702-a9f4-24025963befe,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "0983efc8-e53e-40bd-a721-2924155c20f3", "address": "fa:16:3e:d7:24:ef", "network": {"id": "da9be96c-ce20-4037-999c-230ca799fb06", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-661780305-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5ff300d27c9a4dd3b7ecd507b3627d98", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0983efc8-e5", "ovs_interfaceid": "0983efc8-e53e-40bd-a721-2924155c20f3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:47:16 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Converting VIF {"id": "0983efc8-e53e-40bd-a721-2924155c20f3", "address": "fa:16:3e:d7:24:ef", "network": {"id": "da9be96c-ce20-4037-999c-230ca799fb06", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-661780305-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5ff300d27c9a4dd3b7ecd507b3627d98", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0983efc8-e5", "ovs_interfaceid": "0983efc8-e53e-40bd-a721-2924155c20f3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:47:16 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:d7:24:ef,bridge_name='br-int',has_traffic_filtering=True,id=0983efc8-e53e-40bd-a721-2924155c20f3,network=Network(da9be96c-ce20-4037-999c-230ca799fb06),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap0983efc8-e5') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:47:16 user nova-compute[72351]: DEBUG os_vif [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:d7:24:ef,bridge_name='br-int',has_traffic_filtering=True,id=0983efc8-e53e-40bd-a721-2924155c20f3,network=Network(da9be96c-ce20-4037-999c-230ca799fb06),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap0983efc8-e5') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:47:16 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:47:16 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:47:16 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:47:16 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:47:16 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap0983efc8-e5, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:47:16 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap0983efc8-e5, col_values=(('external_ids', {'iface-id': '0983efc8-e53e-40bd-a721-2924155c20f3', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:d7:24:ef', 'vm-uuid': '150f1b69-aa23-4702-a9f4-24025963befe'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:47:16 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:47:16 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:47:16 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:47:16 user nova-compute[72351]: INFO os_vif [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:d7:24:ef,bridge_name='br-int',has_traffic_filtering=True,id=0983efc8-e53e-40bd-a721-2924155c20f3,network=Network(da9be96c-ce20-4037-999c-230ca799fb06),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap0983efc8-e5')
Aug 09 02:47:16 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:47:16 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] No VIF found with MAC fa:16:3e:d7:24:ef, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:47:16 user nova-compute[72351]: DEBUG nova.network.neutron [req-dd83ca1d-aa06-4a81-aaf4-7249d113f8df req-0b3ee67e-fe40-4105-9ed7-9721881bd21c service nova] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Updated VIF entry in instance network info cache for port 0983efc8-e53e-40bd-a721-2924155c20f3. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:47:16 user nova-compute[72351]: DEBUG nova.network.neutron [req-dd83ca1d-aa06-4a81-aaf4-7249d113f8df req-0b3ee67e-fe40-4105-9ed7-9721881bd21c service nova] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Updating instance_info_cache with network_info: [{"id": "0983efc8-e53e-40bd-a721-2924155c20f3", "address": "fa:16:3e:d7:24:ef", "network": {"id": "da9be96c-ce20-4037-999c-230ca799fb06", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-661780305-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5ff300d27c9a4dd3b7ecd507b3627d98", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0983efc8-e5", "ovs_interfaceid": "0983efc8-e53e-40bd-a721-2924155c20f3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:47:16 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-dd83ca1d-aa06-4a81-aaf4-7249d113f8df req-0b3ee67e-fe40-4105-9ed7-9721881bd21c service nova] Releasing lock "refresh_cache-150f1b69-aa23-4702-a9f4-24025963befe" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:47:16 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:47:16 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:47:16 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72351) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10461}}
Aug 09 02:47:17 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:47:17 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:47:17 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:47:17 user nova-compute[72351]: DEBUG nova.compute.manager [req-eae5be3f-355e-4cac-9351-b344801db0de req-536267e9-e1b3-45e5-96a1-cd0d94b757bb service nova] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Received event network-vif-plugged-0983efc8-e53e-40bd-a721-2924155c20f3 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:47:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-eae5be3f-355e-4cac-9351-b344801db0de req-536267e9-e1b3-45e5-96a1-cd0d94b757bb service nova] Acquiring lock "150f1b69-aa23-4702-a9f4-24025963befe-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:47:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-eae5be3f-355e-4cac-9351-b344801db0de req-536267e9-e1b3-45e5-96a1-cd0d94b757bb service nova] Lock "150f1b69-aa23-4702-a9f4-24025963befe-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:47:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-eae5be3f-355e-4cac-9351-b344801db0de req-536267e9-e1b3-45e5-96a1-cd0d94b757bb service nova] Lock "150f1b69-aa23-4702-a9f4-24025963befe-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:47:17 user nova-compute[72351]: DEBUG nova.compute.manager [req-eae5be3f-355e-4cac-9351-b344801db0de req-536267e9-e1b3-45e5-96a1-cd0d94b757bb service nova] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] No waiting events found dispatching network-vif-plugged-0983efc8-e53e-40bd-a721-2924155c20f3 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:47:17 user nova-compute[72351]: WARNING nova.compute.manager [req-eae5be3f-355e-4cac-9351-b344801db0de req-536267e9-e1b3-45e5-96a1-cd0d94b757bb service nova] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Received unexpected event network-vif-plugged-0983efc8-e53e-40bd-a721-2924155c20f3 for instance with vm_state building and task_state spawning.
Aug 09 02:47:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:47:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:47:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:47:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:47:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:47:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:47:19 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691549239.5525358, 150f1b69-aa23-4702-a9f4-24025963befe => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:47:19 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] VM Resumed (Lifecycle Event)
Aug 09 02:47:19 user nova-compute[72351]: DEBUG nova.compute.manager [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:47:19 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Guest created on hypervisor {{(pid=72351) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4411}}
Aug 09 02:47:19 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Instance spawned successfully.
Aug 09 02:47:19 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:894}}
Aug 09 02:47:19 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:47:19 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:47:19 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Found default for hw_cdrom_bus of ide {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:47:19 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Found default for hw_disk_bus of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:47:19 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Found default for hw_input_bus of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:47:19 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Found default for hw_pointer_model of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:47:19 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Found default for hw_video_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:47:19 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Found default for hw_vif_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:47:19 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:47:19 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691549239.5546381, 150f1b69-aa23-4702-a9f4-24025963befe => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:47:19 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] VM Started (Lifecycle Event)
Aug 09 02:47:19 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:47:19 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:47:19 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:47:19 user nova-compute[72351]: INFO nova.compute.manager [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Took 5.54 seconds to spawn the instance on the hypervisor.
Aug 09 02:47:19 user nova-compute[72351]: DEBUG nova.compute.manager [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:47:19 user nova-compute[72351]: INFO nova.compute.manager [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Took 6.08 seconds to build instance.
Aug 09 02:47:19 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6ae8ebcb-b46e-4e5d-ab87-505a0d46427b tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "150f1b69-aa23-4702-a9f4-24025963befe" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 6.169s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:47:19 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:47:19 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Starting heal instance info cache {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9842}}
Aug 09 02:47:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "refresh_cache-c6ae885f-dce2-4dc2-98d5-26139b8fb67e" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:47:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquired lock "refresh_cache-c6ae885f-dce2-4dc2-98d5-26139b8fb67e" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:47:20 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Forcefully refreshing network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1996}}
Aug 09 02:47:20 user nova-compute[72351]: DEBUG nova.compute.manager [req-f2e99e68-24b8-49d8-ba7b-45af8df78664 req-973d4d8a-3e52-4c12-b92d-bf06ff6ec4c9 service nova] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Received event network-vif-plugged-0983efc8-e53e-40bd-a721-2924155c20f3 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:47:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-f2e99e68-24b8-49d8-ba7b-45af8df78664 req-973d4d8a-3e52-4c12-b92d-bf06ff6ec4c9 service nova] Acquiring lock "150f1b69-aa23-4702-a9f4-24025963befe-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:47:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-f2e99e68-24b8-49d8-ba7b-45af8df78664 req-973d4d8a-3e52-4c12-b92d-bf06ff6ec4c9 service nova] Lock "150f1b69-aa23-4702-a9f4-24025963befe-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:47:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-f2e99e68-24b8-49d8-ba7b-45af8df78664 req-973d4d8a-3e52-4c12-b92d-bf06ff6ec4c9 service nova] Lock "150f1b69-aa23-4702-a9f4-24025963befe-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:47:20 user nova-compute[72351]: DEBUG nova.compute.manager [req-f2e99e68-24b8-49d8-ba7b-45af8df78664 req-973d4d8a-3e52-4c12-b92d-bf06ff6ec4c9 service nova] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] No waiting events found dispatching network-vif-plugged-0983efc8-e53e-40bd-a721-2924155c20f3 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:47:20 user nova-compute[72351]: WARNING nova.compute.manager [req-f2e99e68-24b8-49d8-ba7b-45af8df78664 req-973d4d8a-3e52-4c12-b92d-bf06ff6ec4c9 service nova] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Received unexpected event network-vif-plugged-0983efc8-e53e-40bd-a721-2924155c20f3 for instance with vm_state active and task_state None.
Aug 09 02:47:20 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Updating instance_info_cache with network_info: [{"id": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "address": "fa:16:3e:dd:49:4d", "network": {"id": "dba57c3d-6efd-4057-8053-1c6a92bb24f8", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-568585773-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.116", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ec9861e0882e4065ab99ad3f74d6b25f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2771d93d-fa", "ovs_interfaceid": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:47:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Releasing lock "refresh_cache-c6ae885f-dce2-4dc2-98d5-26139b8fb67e" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:47:20 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Updated the network info_cache for instance {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9913}}
Aug 09 02:47:20 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:47:20 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:47:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:47:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:47:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:47:20 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Auditing locally available compute resources for user (node: user) {{(pid=72351) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Aug 09 02:47:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:47:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:47:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:47:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:47:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk --force-share --output=json" returned: 0 in 0.143s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:47:21 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdb (vdb) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:47:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e020e9e8-2372-40c9-ab34-6f65a529a995/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:47:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e020e9e8-2372-40c9-ab34-6f65a529a995/disk --force-share --output=json" returned: 0 in 0.139s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:47:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e020e9e8-2372-40c9-ab34-6f65a529a995/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:47:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e020e9e8-2372-40c9-ab34-6f65a529a995/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:47:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/150f1b69-aa23-4702-a9f4-24025963befe/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:47:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/150f1b69-aa23-4702-a9f4-24025963befe/disk --force-share --output=json" returned: 0 in 0.132s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:47:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/150f1b69-aa23-4702-a9f4-24025963befe/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:47:21 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/150f1b69-aa23-4702-a9f4-24025963befe/disk --force-share --output=json" returned: 0 in 0.133s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:47:22 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:47:22 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:47:22 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Hypervisor/Node resource view: name=user free_ram=8620MB free_disk=25.257732391357422GB free_vcpus=10 pci_devices=[{"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72351) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Aug 09 02:47:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:47:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:47:22 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance c6ae885f-dce2-4dc2-98d5-26139b8fb67e actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:47:22 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance e020e9e8-2372-40c9-ab34-6f65a529a995 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:47:22 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 150f1b69-aa23-4702-a9f4-24025963befe actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:47:22 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Total usable vcpus: 12, total allocated vcpus: 3 {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Aug 09 02:47:22 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Final resource view: name=user phys_ram=16011MB used_ram=2048MB phys_disk=40GB used_disk=3GB total_vcpus=12 used_vcpus=3 pci_stats=[] {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Aug 09 02:47:22 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:47:22 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:47:22 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Compute_service record updated for user:user {{(pid=72351) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Aug 09 02:47:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.257s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:47:23 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:47:24 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:47:24 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:47:24 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:47:25 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:47:26 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:47:31 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:47:31 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:47:31 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 02:47:31 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:47:31 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:47:31 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:47:33 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:47:36 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:47:38 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:47:41 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:47:43 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:47:46 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:47:48 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:47:51 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:47:53 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ae0843bb-2784-469f-b4bc-700c2a8a95a6 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Acquiring lock "e020e9e8-2372-40c9-ab34-6f65a529a995" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:47:53 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ae0843bb-2784-469f-b4bc-700c2a8a95a6 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "e020e9e8-2372-40c9-ab34-6f65a529a995" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:47:53 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ae0843bb-2784-469f-b4bc-700c2a8a95a6 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Acquiring lock "e020e9e8-2372-40c9-ab34-6f65a529a995-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:47:53 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ae0843bb-2784-469f-b4bc-700c2a8a95a6 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "e020e9e8-2372-40c9-ab34-6f65a529a995-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:47:53 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ae0843bb-2784-469f-b4bc-700c2a8a95a6 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "e020e9e8-2372-40c9-ab34-6f65a529a995-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:47:53 user nova-compute[72351]: INFO nova.compute.manager [None req-ae0843bb-2784-469f-b4bc-700c2a8a95a6 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Terminating instance
Aug 09 02:47:53 user nova-compute[72351]: DEBUG nova.compute.manager [None req-ae0843bb-2784-469f-b4bc-700c2a8a95a6 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Start destroying the instance on the hypervisor. {{(pid=72351) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3124}}
Aug 09 02:47:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:47:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:47:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:47:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:47:54 user nova-compute[72351]: DEBUG nova.compute.manager [req-00180e6a-208f-4278-abf6-96d0e2447a26 req-1432e73f-51bf-4ead-950b-19491edf62f7 service nova] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Received event network-vif-unplugged-94c7c88f-9e5e-4e25-b7b8-59fb82e25f52 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:47:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-00180e6a-208f-4278-abf6-96d0e2447a26 req-1432e73f-51bf-4ead-950b-19491edf62f7 service nova] Acquiring lock "e020e9e8-2372-40c9-ab34-6f65a529a995-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:47:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-00180e6a-208f-4278-abf6-96d0e2447a26 req-1432e73f-51bf-4ead-950b-19491edf62f7 service nova] Lock "e020e9e8-2372-40c9-ab34-6f65a529a995-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:47:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-00180e6a-208f-4278-abf6-96d0e2447a26 req-1432e73f-51bf-4ead-950b-19491edf62f7 service nova] Lock "e020e9e8-2372-40c9-ab34-6f65a529a995-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:47:54 user nova-compute[72351]: DEBUG nova.compute.manager [req-00180e6a-208f-4278-abf6-96d0e2447a26 req-1432e73f-51bf-4ead-950b-19491edf62f7 service nova] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] No waiting events found dispatching network-vif-unplugged-94c7c88f-9e5e-4e25-b7b8-59fb82e25f52 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:47:54 user nova-compute[72351]: DEBUG nova.compute.manager [req-00180e6a-208f-4278-abf6-96d0e2447a26 req-1432e73f-51bf-4ead-950b-19491edf62f7 service nova] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Received event network-vif-unplugged-94c7c88f-9e5e-4e25-b7b8-59fb82e25f52 for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:47:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:47:54 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Instance destroyed successfully.
Aug 09 02:47:54 user nova-compute[72351]: DEBUG nova.objects.instance [None req-ae0843bb-2784-469f-b4bc-700c2a8a95a6 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lazy-loading 'resources' on Instance uuid e020e9e8-2372-40c9-ab34-6f65a529a995 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:47:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-ae0843bb-2784-469f-b4bc-700c2a8a95a6 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:44:56Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachVolumeNegativeTest-server-683106636',display_name='tempest-AttachVolumeNegativeTest-server-683106636',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumenegativetest-server-683106636',id=48,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHwdQ8uAXuOji59B0ui3/RExpPgI9bhpJBH1YGsboG+umNG6Flhx2vIHoK2OOqHRN++9f4+Urrs/FGjCFSAhCoRLwtfBcHoTxWtjYovYJmURgXhxEEx/g4+2Bc4XnER+PQ==',key_name='tempest-keypair-987216784',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:45:03Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='8c07a6b254414b41a673468238d3aa62',ramdisk_id='',reservation_id='r-iafv192h',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeNegativeTest-758771161',owner_user_name='tempest-AttachVolumeNegativeTest-758771161-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:45:03Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='17f7007adf02474e9b019cd8af099ff8',uuid=e020e9e8-2372-40c9-ab34-6f65a529a995,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "94c7c88f-9e5e-4e25-b7b8-59fb82e25f52", "address": "fa:16:3e:c1:3e:66", "network": {"id": "47c43230-a244-4198-84e4-9ed98c005cd9", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1392403106-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.151", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c07a6b254414b41a673468238d3aa62", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap94c7c88f-9e", "ovs_interfaceid": "94c7c88f-9e5e-4e25-b7b8-59fb82e25f52", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:47:54 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-ae0843bb-2784-469f-b4bc-700c2a8a95a6 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Converting VIF {"id": "94c7c88f-9e5e-4e25-b7b8-59fb82e25f52", "address": "fa:16:3e:c1:3e:66", "network": {"id": "47c43230-a244-4198-84e4-9ed98c005cd9", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1392403106-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.151", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c07a6b254414b41a673468238d3aa62", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap94c7c88f-9e", "ovs_interfaceid": "94c7c88f-9e5e-4e25-b7b8-59fb82e25f52", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:47:54 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-ae0843bb-2784-469f-b4bc-700c2a8a95a6 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:c1:3e:66,bridge_name='br-int',has_traffic_filtering=True,id=94c7c88f-9e5e-4e25-b7b8-59fb82e25f52,network=Network(47c43230-a244-4198-84e4-9ed98c005cd9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap94c7c88f-9e') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:47:54 user nova-compute[72351]: DEBUG os_vif [None req-ae0843bb-2784-469f-b4bc-700c2a8a95a6 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:c1:3e:66,bridge_name='br-int',has_traffic_filtering=True,id=94c7c88f-9e5e-4e25-b7b8-59fb82e25f52,network=Network(47c43230-a244-4198-84e4-9ed98c005cd9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap94c7c88f-9e') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:47:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:47:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap94c7c88f-9e, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:47:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:47:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:47:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:47:54 user nova-compute[72351]: INFO os_vif [None req-ae0843bb-2784-469f-b4bc-700c2a8a95a6 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:c1:3e:66,bridge_name='br-int',has_traffic_filtering=True,id=94c7c88f-9e5e-4e25-b7b8-59fb82e25f52,network=Network(47c43230-a244-4198-84e4-9ed98c005cd9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap94c7c88f-9e')
Aug 09 02:47:54 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-ae0843bb-2784-469f-b4bc-700c2a8a95a6 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Deleting instance files /opt/stack/data/nova/instances/e020e9e8-2372-40c9-ab34-6f65a529a995_del
Aug 09 02:47:54 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-ae0843bb-2784-469f-b4bc-700c2a8a95a6 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Deletion of /opt/stack/data/nova/instances/e020e9e8-2372-40c9-ab34-6f65a529a995_del complete
Aug 09 02:47:54 user nova-compute[72351]: INFO nova.compute.manager [None req-ae0843bb-2784-469f-b4bc-700c2a8a95a6 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Took 0.87 seconds to destroy the instance on the hypervisor.
Aug 09 02:47:54 user nova-compute[72351]: DEBUG oslo.service.loopingcall [None req-ae0843bb-2784-469f-b4bc-700c2a8a95a6 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72351) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Aug 09 02:47:54 user nova-compute[72351]: DEBUG nova.compute.manager [-] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Deallocating network for instance {{(pid=72351) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2263}}
Aug 09 02:47:54 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] deallocate_for_instance() {{(pid=72351) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1795}}
Aug 09 02:47:55 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:47:55 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Took 0.92 seconds to deallocate network for instance.
Aug 09 02:47:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ae0843bb-2784-469f-b4bc-700c2a8a95a6 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:47:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ae0843bb-2784-469f-b4bc-700c2a8a95a6 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:47:55 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-ae0843bb-2784-469f-b4bc-700c2a8a95a6 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:47:55 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-ae0843bb-2784-469f-b4bc-700c2a8a95a6 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:47:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ae0843bb-2784-469f-b4bc-700c2a8a95a6 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.148s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:47:55 user nova-compute[72351]: INFO nova.scheduler.client.report [None req-ae0843bb-2784-469f-b4bc-700c2a8a95a6 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Deleted allocations for instance e020e9e8-2372-40c9-ab34-6f65a529a995
Aug 09 02:47:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-ae0843bb-2784-469f-b4bc-700c2a8a95a6 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "e020e9e8-2372-40c9-ab34-6f65a529a995" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.132s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:47:56 user nova-compute[72351]: DEBUG nova.compute.manager [req-2bee0827-c581-4435-bf7a-c5ed647f53aa req-3ac4f590-69c7-4a99-b307-85d52bd137cf service nova] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Received event network-vif-plugged-94c7c88f-9e5e-4e25-b7b8-59fb82e25f52 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:47:56 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-2bee0827-c581-4435-bf7a-c5ed647f53aa req-3ac4f590-69c7-4a99-b307-85d52bd137cf service nova] Acquiring lock "e020e9e8-2372-40c9-ab34-6f65a529a995-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:47:56 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-2bee0827-c581-4435-bf7a-c5ed647f53aa req-3ac4f590-69c7-4a99-b307-85d52bd137cf service nova] Lock "e020e9e8-2372-40c9-ab34-6f65a529a995-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:47:56 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-2bee0827-c581-4435-bf7a-c5ed647f53aa req-3ac4f590-69c7-4a99-b307-85d52bd137cf service nova] Lock "e020e9e8-2372-40c9-ab34-6f65a529a995-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:47:56 user nova-compute[72351]: DEBUG nova.compute.manager [req-2bee0827-c581-4435-bf7a-c5ed647f53aa req-3ac4f590-69c7-4a99-b307-85d52bd137cf service nova] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] No waiting events found dispatching network-vif-plugged-94c7c88f-9e5e-4e25-b7b8-59fb82e25f52 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:47:56 user nova-compute[72351]: WARNING nova.compute.manager [req-2bee0827-c581-4435-bf7a-c5ed647f53aa req-3ac4f590-69c7-4a99-b307-85d52bd137cf service nova] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Received unexpected event network-vif-plugged-94c7c88f-9e5e-4e25-b7b8-59fb82e25f52 for instance with vm_state deleted and task_state None.
Aug 09 02:47:56 user nova-compute[72351]: DEBUG nova.compute.manager [req-2bee0827-c581-4435-bf7a-c5ed647f53aa req-3ac4f590-69c7-4a99-b307-85d52bd137cf service nova] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Received event network-vif-deleted-94c7c88f-9e5e-4e25-b7b8-59fb82e25f52 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:47:59 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:48:04 user nova-compute[72351]: DEBUG nova.objects.instance [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lazy-loading 'flavor' on Instance uuid c6ae885f-dce2-4dc2-98d5-26139b8fb67e {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:48:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Acquiring lock "refresh_cache-c6ae885f-dce2-4dc2-98d5-26139b8fb67e" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:48:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Acquired lock "refresh_cache-c6ae885f-dce2-4dc2-98d5-26139b8fb67e" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:48:04 user nova-compute[72351]: DEBUG nova.network.neutron [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:48:04 user nova-compute[72351]: DEBUG nova.objects.instance [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lazy-loading 'info_cache' on Instance uuid c6ae885f-dce2-4dc2-98d5-26139b8fb67e {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:48:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:48:04 user nova-compute[72351]: DEBUG nova.network.neutron [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Updating instance_info_cache with network_info: [{"id": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "address": "fa:16:3e:dd:49:4d", "network": {"id": "dba57c3d-6efd-4057-8053-1c6a92bb24f8", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-568585773-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.116", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ec9861e0882e4065ab99ad3f74d6b25f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2771d93d-fa", "ovs_interfaceid": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:48:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Releasing lock "refresh_cache-c6ae885f-dce2-4dc2-98d5-26139b8fb67e" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:48:04 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Instance destroyed successfully.
Aug 09 02:48:04 user nova-compute[72351]: DEBUG nova.objects.instance [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lazy-loading 'resources' on Instance uuid c6ae885f-dce2-4dc2-98d5-26139b8fb67e {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:48:04 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:44:08Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachVolumeTestJSON-server-218910074',display_name='tempest-AttachVolumeTestJSON-server-218910074',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumetestjson-server-218910074',id=47,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBIplDVAhYRmCgaFwiRMKOJ3OlhP3n+pFAzSDpMz7BP4WftIaiuBT1r87OWLI3l8JRBwh8sp/09wt0bUhVeTDk2tWONkdagBfHB77ctUB1RkuePwrxnVW0j07TkHZdA5xrw==',key_name='tempest-keypair-304755592',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:44:14Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=<?>,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=4,progress=0,project_id='ec9861e0882e4065ab99ad3f74d6b25f',ramdisk_id='',reservation_id='r-2vl0h2f4',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=<?>,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeTestJSON-1725368857',owner_user_name='tempest-AttachVolumeTestJSON-1725368857-project-member'},tags=<?>,task_state='powering-on',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:46:14Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='a2bc49dd654f4beca4b8eb5a49cd350d',uuid=c6ae885f-dce2-4dc2-98d5-26139b8fb67e,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='stopped') vif={"id": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "address": "fa:16:3e:dd:49:4d", "network": {"id": "dba57c3d-6efd-4057-8053-1c6a92bb24f8", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-568585773-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.116", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ec9861e0882e4065ab99ad3f74d6b25f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2771d93d-fa", "ovs_interfaceid": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:48:04 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Converting VIF {"id": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "address": "fa:16:3e:dd:49:4d", "network": {"id": "dba57c3d-6efd-4057-8053-1c6a92bb24f8", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-568585773-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.116", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ec9861e0882e4065ab99ad3f74d6b25f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2771d93d-fa", "ovs_interfaceid": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:48:04 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:dd:49:4d,bridge_name='br-int',has_traffic_filtering=True,id=2771d93d-fa0a-4e19-85ac-9d3f5df52782,network=Network(dba57c3d-6efd-4057-8053-1c6a92bb24f8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2771d93d-fa') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:48:04 user nova-compute[72351]: DEBUG os_vif [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:dd:49:4d,bridge_name='br-int',has_traffic_filtering=True,id=2771d93d-fa0a-4e19-85ac-9d3f5df52782,network=Network(dba57c3d-6efd-4057-8053-1c6a92bb24f8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2771d93d-fa') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:48:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:48:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap2771d93d-fa, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:48:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:48:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:48:04 user nova-compute[72351]: INFO os_vif [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:dd:49:4d,bridge_name='br-int',has_traffic_filtering=True,id=2771d93d-fa0a-4e19-85ac-9d3f5df52782,network=Network(dba57c3d-6efd-4057-8053-1c6a92bb24f8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2771d93d-fa')
Aug 09 02:48:04 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 02:48:04 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'QgoDJMbr', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb'}, None), 'kwargs': True}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:48:04 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:48:04 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[f1c07418-7b4a-4657-b6f8-b6949c64b221]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[20f880e7-32cf-4787-bbcc-f8b58cc8bcd8]: (4, ('tcp: [58] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [58] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 (non-flash)
Aug 09 02:48:05 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsi session list stdout=tcp: [58] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 (non-flash)
Aug 09 02:48:05 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fcffc276200>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857'): ({'sdb'}, set())}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Removing single pathed devices sdb {{(pid=72351) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[3bd6af5e-9d60-408c-90be-0171696f1330]: (4, ('path checker states:\nup                  1\n\npaths: 0\nbusy: False\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[3d1b71cb-b62a-4ec3-a30e-a2c42691029a]: (4, ('ok\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Flushing IO for device /dev/sdb {{(pid=72351) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.013s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[9fe6681a-9b4f-479f-9dd5-1d4b169ce170]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=72351) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.045s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[cfb235c0-77dd-4da2-85a3-5187d5ec04d8]: (4, ('1', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] SCSI volumes sdb have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857')] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[6d9c932f-e43f-4f47-a7dc-eb26dd4e6ae7]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 -p 172.16.0.220:3260 --logout {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 -p 172.16.0.220:3260 --logout" returned: 0 in 0.054s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[3e9d763c-a554-4c41-9fc0-cd7b22093765]: (4, ('Logging out of session [sid: 58, target: iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857, portal: 172.16.0.220,3260]\nLogout of [sid: 58, target: iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 58, target: iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857, portal: 172.16.0.220,3260]
Aug 09 02:48:05 user nova-compute[72351]: Logout of [sid: 58, target: iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857, portal: 172.16.0.220,3260] successful.
Aug 09 02:48:05 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 -p 172.16.0.220:3260 --op delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[27636442-c3c8-4972-98f4-a20466325403]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.276s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] <== disconnect_volume: return (277ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Start _get_guest_xml network_info=[{"id": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "address": "fa:16:3e:dd:49:4d", "network": {"id": "dba57c3d-6efd-4057-8053-1c6a92bb24f8", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-568585773-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.116", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ec9861e0882e4065ab99ad3f74d6b25f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2771d93d-fa", "ovs_interfaceid": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vdb': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} image_meta=ImageMeta(checksum=<?>,container_format='bare',created_at=<?>,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=1,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=<?>,status=<?>,tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'size': 0, 'device_type': 'disk', 'encryption_secret_uuid': None, 'guest_format': None, 'encryption_options': None, 'disk_bus': 'virtio', 'encrypted': False, 'encryption_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'image_id': '18ccda7f-31d3-4978-ab9d-4ed2f74eff94'}], 'ephemerals': [], 'block_device_mapping': [{'device_type': 'disk', 'guest_format': None, 'disk_bus': 'virtio', 'mount_device': '/dev/vdb', 'boot_index': None, 'attachment_id': 'a470ef1b-600e-4db5-8aeb-04002b6747a0', 'delete_on_termination': False, 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqns': ['iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'QgoDJMbr', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb'}, 'status': 'reserved', 'instance': 'c6ae885f-dce2-4dc2-98d5-26139b8fb67e', 'attached_at': '', 'detached_at': '', 'volume_id': '008f5fa6-2644-4c22-b8cd-28e233cf8857', 'serial': '008f5fa6-2644-4c22-b8cd-28e233cf8857'}, 'volume_type': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG nova.objects.instance [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lazy-loading 'numa_topology' on Instance uuid c6ae885f-dce2-4dc2-98d5-26139b8fb67e {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:48:05 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:48:05 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:48:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format='bare',created_at=<?>,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=1,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=<?>,status=<?>,tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG nova.objects.instance [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lazy-loading 'vcpu_model' on Instance uuid c6ae885f-dce2-4dc2-98d5-26139b8fb67e {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'QgoDJMbr', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path'}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:48:05 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:48:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:48:05 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:48:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:48:05 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 02:48:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 -p 172.16.0.220:3260" returned: 21 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[71b359cf-19c5-4a5b-a253-39856712437a]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Aug 09 02:48:05 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 -p 172.16.0.220:3260 --interface default --op new {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d9bfc5f7-93fd-4dc5-9f85-c18a8432487f]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857] added\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857] added
Aug 09 02:48:05 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.032s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[04db7d10-3769-4a4d-85c3-d10d0aa4d92d]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[80bc315b-b265-4971-8df4-f06b5a96bf3f]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v QgoDJMbr {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v QgoDJMbr" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[fad60d6e-9911-4751-aa48-34b696a1be8d]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'QgoDJMbr'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[4c32a370-1c7b-46fd-b15f-c31a5ba3c4fb]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 21 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[baa66c7d-aa75-425c-919f-68d0cce410a8]: (4, ('', 'iscsiadm: No active sessions.\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsiadm ('-m', 'session'): stdout= stderr=iscsiadm: No active sessions.
Aug 09 02:48:05 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsi session list stdout= stderr=iscsiadm: No active sessions.
Aug 09 02:48:05 user nova-compute[72351]:  {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:48:05 user nova-compute[72351]: WARNING os_brick.initiator.connectors.iscsi [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsiadm stderr output when getting sessions: iscsiadm: No active sessions.
Aug 09 02:48:05 user nova-compute[72351]: 
Aug 09 02:48:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 -p 172.16.0.220:3260 --login {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 -p 172.16.0.220:3260 --login" returned: 0 in 0.050s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[800be4f7-917a-4851-a24d-7ae7190db835]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857, portal: 172.16.0.220,3260]
Aug 09 02:48:05 user nova-compute[72351]: Login to [iface: default, target: iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857, portal: 172.16.0.220,3260] successful.
Aug 09 02:48:05 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[2006810d-e1e6-4407-aa9c-1f708239cb5f]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[5c151ad5-148f-413d-8ebb-94bd9e4dbac1]: (4, ('tcp: [60] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [60] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 (non-flash)
Aug 09 02:48:05 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsi session list stdout=tcp: [60] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 (non-flash)
Aug 09 02:48:05 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.278s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] HCTL ('33', '-', '-', 0) found on session 60 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[fe3b5d9d-23cf-446c-ac4e-24c7d300d619]: (4, ('- - 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Searching for a device in session 60 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:48:05 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Connected to sdb using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'QgoDJMbr', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb', 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 02:48:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.307s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:48:06 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] <== connect_volume: return (1308ms) {'type': 'block', 'scsi_wwn': '23066323566643038', 'path': '/dev/sdb'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:48:06 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23066323566643038', 'path': '/dev/sdb'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 02:48:06 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:44:08Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachVolumeTestJSON-server-218910074',display_name='tempest-AttachVolumeTestJSON-server-218910074',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumetestjson-server-218910074',id=47,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBIplDVAhYRmCgaFwiRMKOJ3OlhP3n+pFAzSDpMz7BP4WftIaiuBT1r87OWLI3l8JRBwh8sp/09wt0bUhVeTDk2tWONkdagBfHB77ctUB1RkuePwrxnVW0j07TkHZdA5xrw==',key_name='tempest-keypair-304755592',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:44:14Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=4,progress=0,project_id='ec9861e0882e4065ab99ad3f74d6b25f',ramdisk_id='',reservation_id='r-2vl0h2f4',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=<?>,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeTestJSON-1725368857',owner_user_name='tempest-AttachVolumeTestJSON-1725368857-project-member'},tags=<?>,task_state='powering-on',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:46:14Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='a2bc49dd654f4beca4b8eb5a49cd350d',uuid=c6ae885f-dce2-4dc2-98d5-26139b8fb67e,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='stopped') vif={"id": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "address": "fa:16:3e:dd:49:4d", "network": {"id": "dba57c3d-6efd-4057-8053-1c6a92bb24f8", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-568585773-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.116", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ec9861e0882e4065ab99ad3f74d6b25f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2771d93d-fa", "ovs_interfaceid": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:48:06 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Converting VIF {"id": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "address": "fa:16:3e:dd:49:4d", "network": {"id": "dba57c3d-6efd-4057-8053-1c6a92bb24f8", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-568585773-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.116", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ec9861e0882e4065ab99ad3f74d6b25f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2771d93d-fa", "ovs_interfaceid": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:48:06 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:dd:49:4d,bridge_name='br-int',has_traffic_filtering=True,id=2771d93d-fa0a-4e19-85ac-9d3f5df52782,network=Network(dba57c3d-6efd-4057-8053-1c6a92bb24f8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2771d93d-fa') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:48:06 user nova-compute[72351]: DEBUG nova.objects.instance [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lazy-loading 'pci_devices' on Instance uuid c6ae885f-dce2-4dc2-98d5-26139b8fb67e {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:48:06 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:48:06 user nova-compute[72351]:   <uuid>c6ae885f-dce2-4dc2-98d5-26139b8fb67e</uuid>
Aug 09 02:48:06 user nova-compute[72351]:   <name>instance-0000002f</name>
Aug 09 02:48:06 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:48:06 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:48:06 user nova-compute[72351]:   <metadata>
Aug 09 02:48:06 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:48:06 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:48:06 user nova-compute[72351]:       <nova:name>tempest-AttachVolumeTestJSON-server-218910074</nova:name>
Aug 09 02:48:06 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:48:05</nova:creationTime>
Aug 09 02:48:06 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:48:06 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:48:06 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:48:06 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:48:06 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:48:06 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:48:06 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:48:06 user nova-compute[72351]:       <nova:owner>
Aug 09 02:48:06 user nova-compute[72351]:         <nova:user uuid="a2bc49dd654f4beca4b8eb5a49cd350d">tempest-AttachVolumeTestJSON-1725368857-project-member</nova:user>
Aug 09 02:48:06 user nova-compute[72351]:         <nova:project uuid="ec9861e0882e4065ab99ad3f74d6b25f">tempest-AttachVolumeTestJSON-1725368857</nova:project>
Aug 09 02:48:06 user nova-compute[72351]:       </nova:owner>
Aug 09 02:48:06 user nova-compute[72351]:       <nova:root type="image" uuid="18ccda7f-31d3-4978-ab9d-4ed2f74eff94"/>
Aug 09 02:48:06 user nova-compute[72351]:       <nova:ports>
Aug 09 02:48:06 user nova-compute[72351]:         <nova:port uuid="2771d93d-fa0a-4e19-85ac-9d3f5df52782">
Aug 09 02:48:06 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.12" ipVersion="4"/>
Aug 09 02:48:06 user nova-compute[72351]:         </nova:port>
Aug 09 02:48:06 user nova-compute[72351]:       </nova:ports>
Aug 09 02:48:06 user nova-compute[72351]:     </nova:instance>
Aug 09 02:48:06 user nova-compute[72351]:   </metadata>
Aug 09 02:48:06 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:48:06 user nova-compute[72351]:     <system>
Aug 09 02:48:06 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:48:06 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:48:06 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:48:06 user nova-compute[72351]:       <entry name="serial">c6ae885f-dce2-4dc2-98d5-26139b8fb67e</entry>
Aug 09 02:48:06 user nova-compute[72351]:       <entry name="uuid">c6ae885f-dce2-4dc2-98d5-26139b8fb67e</entry>
Aug 09 02:48:06 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:48:06 user nova-compute[72351]:     </system>
Aug 09 02:48:06 user nova-compute[72351]:   </sysinfo>
Aug 09 02:48:06 user nova-compute[72351]:   <os>
Aug 09 02:48:06 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:48:06 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:48:06 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:48:06 user nova-compute[72351]:   </os>
Aug 09 02:48:06 user nova-compute[72351]:   <features>
Aug 09 02:48:06 user nova-compute[72351]:     <acpi/>
Aug 09 02:48:06 user nova-compute[72351]:     <apic/>
Aug 09 02:48:06 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:48:06 user nova-compute[72351]:   </features>
Aug 09 02:48:06 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:48:06 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:48:06 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:48:06 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:48:06 user nova-compute[72351]:   </clock>
Aug 09 02:48:06 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:48:06 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:48:06 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:48:06 user nova-compute[72351]:   </cpu>
Aug 09 02:48:06 user nova-compute[72351]:   <devices>
Aug 09 02:48:06 user nova-compute[72351]:     <disk type="file" device="disk">
Aug 09 02:48:06 user nova-compute[72351]:       <driver name="qemu" type="qcow2" cache="none"/>
Aug 09 02:48:06 user nova-compute[72351]:       <source file="/opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk"/>
Aug 09 02:48:06 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:48:06 user nova-compute[72351]:     </disk>
Aug 09 02:48:06 user nova-compute[72351]:     <disk type="block" device="disk">
Aug 09 02:48:06 user nova-compute[72351]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:48:06 user nova-compute[72351]:       <source dev="/dev/sdb"/>
Aug 09 02:48:06 user nova-compute[72351]:       <target dev="vdb" bus="virtio"/>
Aug 09 02:48:06 user nova-compute[72351]:       <serial>008f5fa6-2644-4c22-b8cd-28e233cf8857</serial>
Aug 09 02:48:06 user nova-compute[72351]:     </disk>
Aug 09 02:48:06 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:48:06 user nova-compute[72351]:       <mac address="fa:16:3e:dd:49:4d"/>
Aug 09 02:48:06 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:48:06 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:48:06 user nova-compute[72351]:       <target dev="tap2771d93d-fa"/>
Aug 09 02:48:06 user nova-compute[72351]:     </interface>
Aug 09 02:48:06 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:48:06 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/console.log" append="off"/>
Aug 09 02:48:06 user nova-compute[72351]:     </serial>
Aug 09 02:48:06 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:48:06 user nova-compute[72351]:     <video>
Aug 09 02:48:06 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:48:06 user nova-compute[72351]:     </video>
Aug 09 02:48:06 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:48:06 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:48:06 user nova-compute[72351]:     </rng>
Aug 09 02:48:06 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:48:06 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:48:06 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:48:06 user nova-compute[72351]:     </memballoon>
Aug 09 02:48:06 user nova-compute[72351]:   </devices>
Aug 09 02:48:06 user nova-compute[72351]: </domain>
Aug 09 02:48:06 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:48:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:48:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:48:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:48:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk --force-share --output=json" returned: 0 in 0.154s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:48:07 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] skipping disk /dev/sdb (vdb) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:48:07 user nova-compute[72351]: DEBUG nova.objects.instance [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lazy-loading 'trusted_certs' on Instance uuid c6ae885f-dce2-4dc2-98d5-26139b8fb67e {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:48:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:48:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.130s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:48:07 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Checking if we can resize image /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk. size=1073741824 {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Aug 09 02:48:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:48:07 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:48:07 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Cannot resize image /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk to a smaller size. {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Aug 09 02:48:07 user nova-compute[72351]: DEBUG nova.objects.instance [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lazy-loading 'migration_context' on Instance uuid c6ae885f-dce2-4dc2-98d5-26139b8fb67e {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:48:07 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:44:08Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachVolumeTestJSON-server-218910074',display_name='tempest-AttachVolumeTestJSON-server-218910074',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumetestjson-server-218910074',id=47,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBIplDVAhYRmCgaFwiRMKOJ3OlhP3n+pFAzSDpMz7BP4WftIaiuBT1r87OWLI3l8JRBwh8sp/09wt0bUhVeTDk2tWONkdagBfHB77ctUB1RkuePwrxnVW0j07TkHZdA5xrw==',key_name='tempest-keypair-304755592',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:44:14Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=<?>,power_state=4,progress=0,project_id='ec9861e0882e4065ab99ad3f74d6b25f',ramdisk_id='',reservation_id='r-2vl0h2f4',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=<?>,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeTestJSON-1725368857',owner_user_name='tempest-AttachVolumeTestJSON-1725368857-project-member'},tags=<?>,task_state='powering-on',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:46:14Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='a2bc49dd654f4beca4b8eb5a49cd350d',uuid=c6ae885f-dce2-4dc2-98d5-26139b8fb67e,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='stopped') vif={"id": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "address": "fa:16:3e:dd:49:4d", "network": {"id": "dba57c3d-6efd-4057-8053-1c6a92bb24f8", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-568585773-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.116", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ec9861e0882e4065ab99ad3f74d6b25f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2771d93d-fa", "ovs_interfaceid": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:48:07 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Converting VIF {"id": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "address": "fa:16:3e:dd:49:4d", "network": {"id": "dba57c3d-6efd-4057-8053-1c6a92bb24f8", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-568585773-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.116", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ec9861e0882e4065ab99ad3f74d6b25f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2771d93d-fa", "ovs_interfaceid": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:48:07 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:dd:49:4d,bridge_name='br-int',has_traffic_filtering=True,id=2771d93d-fa0a-4e19-85ac-9d3f5df52782,network=Network(dba57c3d-6efd-4057-8053-1c6a92bb24f8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2771d93d-fa') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:48:07 user nova-compute[72351]: DEBUG os_vif [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:dd:49:4d,bridge_name='br-int',has_traffic_filtering=True,id=2771d93d-fa0a-4e19-85ac-9d3f5df52782,network=Network(dba57c3d-6efd-4057-8053-1c6a92bb24f8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2771d93d-fa') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:48:07 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:48:07 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:48:07 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:48:07 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:48:07 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap2771d93d-fa, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:48:07 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap2771d93d-fa, col_values=(('external_ids', {'iface-id': '2771d93d-fa0a-4e19-85ac-9d3f5df52782', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:dd:49:4d', 'vm-uuid': 'c6ae885f-dce2-4dc2-98d5-26139b8fb67e'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:48:07 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:48:07 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:48:07 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:48:07 user nova-compute[72351]: INFO os_vif [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:dd:49:4d,bridge_name='br-int',has_traffic_filtering=True,id=2771d93d-fa0a-4e19-85ac-9d3f5df52782,network=Network(dba57c3d-6efd-4057-8053-1c6a92bb24f8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2771d93d-fa')
Aug 09 02:48:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:48:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:48:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:48:08 user nova-compute[72351]: DEBUG nova.compute.manager [req-d9d33b1a-5c3e-4060-a407-2825b27f46ee req-bb494eb1-4749-475d-b827-04d3c77ee182 service nova] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Received event network-vif-plugged-2771d93d-fa0a-4e19-85ac-9d3f5df52782 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:48:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d9d33b1a-5c3e-4060-a407-2825b27f46ee req-bb494eb1-4749-475d-b827-04d3c77ee182 service nova] Acquiring lock "c6ae885f-dce2-4dc2-98d5-26139b8fb67e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:48:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d9d33b1a-5c3e-4060-a407-2825b27f46ee req-bb494eb1-4749-475d-b827-04d3c77ee182 service nova] Lock "c6ae885f-dce2-4dc2-98d5-26139b8fb67e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:48:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-d9d33b1a-5c3e-4060-a407-2825b27f46ee req-bb494eb1-4749-475d-b827-04d3c77ee182 service nova] Lock "c6ae885f-dce2-4dc2-98d5-26139b8fb67e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:48:08 user nova-compute[72351]: DEBUG nova.compute.manager [req-d9d33b1a-5c3e-4060-a407-2825b27f46ee req-bb494eb1-4749-475d-b827-04d3c77ee182 service nova] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] No waiting events found dispatching network-vif-plugged-2771d93d-fa0a-4e19-85ac-9d3f5df52782 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:48:08 user nova-compute[72351]: WARNING nova.compute.manager [req-d9d33b1a-5c3e-4060-a407-2825b27f46ee req-bb494eb1-4749-475d-b827-04d3c77ee182 service nova] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Received unexpected event network-vif-plugged-2771d93d-fa0a-4e19-85ac-9d3f5df52782 for instance with vm_state stopped and task_state powering-on.
Aug 09 02:48:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:48:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:48:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:48:09 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691549274.510538, e020e9e8-2372-40c9-ab34-6f65a529a995 => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:48:09 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] VM Stopped (Lifecycle Event)
Aug 09 02:48:09 user nova-compute[72351]: DEBUG nova.compute.manager [None req-cc220e6f-b469-4c98-a0e1-1c90628dd825 None None] [instance: e020e9e8-2372-40c9-ab34-6f65a529a995] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:48:10 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691549290.7468822, c6ae885f-dce2-4dc2-98d5-26139b8fb67e => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:48:10 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] VM Resumed (Lifecycle Event)
Aug 09 02:48:10 user nova-compute[72351]: DEBUG nova.compute.manager [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:48:10 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Instance rebooted successfully.
Aug 09 02:48:10 user nova-compute[72351]: DEBUG nova.compute.manager [None req-4a8cb50a-427e-4e91-8fbb-82aaabda6c55 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:48:10 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:48:10 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: stopped, current task_state: powering-on, current DB power_state: 4, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:48:10 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] During sync_power_state the instance has a pending task (powering-on). Skip.
Aug 09 02:48:10 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691549290.7479784, c6ae885f-dce2-4dc2-98d5-26139b8fb67e => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:48:10 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] VM Started (Lifecycle Event)
Aug 09 02:48:10 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:48:10 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Synchronizing instance power state after lifecycle event "Started"; current vm_state: stopped, current task_state: powering-on, current DB power_state: 4, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:48:10 user nova-compute[72351]: DEBUG nova.compute.manager [req-c6059865-0a11-4cb2-8a8d-1b8f8cedc3cf req-95350508-cd3b-47e4-bc81-7c6314c0ae18 service nova] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Received event network-vif-plugged-2771d93d-fa0a-4e19-85ac-9d3f5df52782 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:48:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c6059865-0a11-4cb2-8a8d-1b8f8cedc3cf req-95350508-cd3b-47e4-bc81-7c6314c0ae18 service nova] Acquiring lock "c6ae885f-dce2-4dc2-98d5-26139b8fb67e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:48:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c6059865-0a11-4cb2-8a8d-1b8f8cedc3cf req-95350508-cd3b-47e4-bc81-7c6314c0ae18 service nova] Lock "c6ae885f-dce2-4dc2-98d5-26139b8fb67e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:48:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c6059865-0a11-4cb2-8a8d-1b8f8cedc3cf req-95350508-cd3b-47e4-bc81-7c6314c0ae18 service nova] Lock "c6ae885f-dce2-4dc2-98d5-26139b8fb67e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:48:10 user nova-compute[72351]: DEBUG nova.compute.manager [req-c6059865-0a11-4cb2-8a8d-1b8f8cedc3cf req-95350508-cd3b-47e4-bc81-7c6314c0ae18 service nova] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] No waiting events found dispatching network-vif-plugged-2771d93d-fa0a-4e19-85ac-9d3f5df52782 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:48:10 user nova-compute[72351]: WARNING nova.compute.manager [req-c6059865-0a11-4cb2-8a8d-1b8f8cedc3cf req-95350508-cd3b-47e4-bc81-7c6314c0ae18 service nova] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Received unexpected event network-vif-plugged-2771d93d-fa0a-4e19-85ac-9d3f5df52782 for instance with vm_state active and task_state None.
Aug 09 02:48:12 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:48:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:48:16 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:48:17 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:48:17 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:48:17 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:48:17 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72351) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10461}}
Aug 09 02:48:19 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:48:19 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Starting heal instance info cache {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9842}}
Aug 09 02:48:19 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Didn't find any instances for network info cache update. {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9928}}
Aug 09 02:48:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:48:22 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:48:22 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:48:22 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:48:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:48:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:48:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:48:22 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Auditing locally available compute resources for user (node: user) {{(pid=72351) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Aug 09 02:48:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:48:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk --force-share --output=json" returned: 0 in 0.132s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:48:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:48:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk --force-share --output=json" returned: 0 in 0.130s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:48:23 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdb (vdb) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:48:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/150f1b69-aa23-4702-a9f4-24025963befe/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:48:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/150f1b69-aa23-4702-a9f4-24025963befe/disk --force-share --output=json" returned: 0 in 0.129s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:48:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/150f1b69-aa23-4702-a9f4-24025963befe/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:48:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/150f1b69-aa23-4702-a9f4-24025963befe/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:48:24 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:48:24 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:48:24 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Hypervisor/Node resource view: name=user free_ram=8545MB free_disk=25.25632095336914GB free_vcpus=10 pci_devices=[{"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72351) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Aug 09 02:48:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:48:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:48:24 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance c6ae885f-dce2-4dc2-98d5-26139b8fb67e actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:48:24 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 150f1b69-aa23-4702-a9f4-24025963befe actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:48:24 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Total usable vcpus: 12, total allocated vcpus: 2 {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Aug 09 02:48:24 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Final resource view: name=user phys_ram=16011MB used_ram=1536MB phys_disk=40GB used_disk=2GB total_vcpus=12 used_vcpus=2 pci_stats=[] {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Aug 09 02:48:24 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:48:24 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:48:24 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Compute_service record updated for user:user {{(pid=72351) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Aug 09 02:48:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.237s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:48:24 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._cleanup_expired_console_auth_tokens {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:48:26 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:48:26 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:48:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:48:28 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:48:32 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:48:32 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._run_pending_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:48:32 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Cleaning up deleted instances {{(pid=72351) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11129}}
Aug 09 02:48:32 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] There are 0 instances to clean {{(pid=72351) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11138}}
Aug 09 02:48:33 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:48:37 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:48:38 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:48:42 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:48:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:48:45 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._cleanup_incomplete_migrations {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:48:45 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Cleaning up deleted instances with incomplete migration  {{(pid=72351) _cleanup_incomplete_migrations /opt/stack/nova/nova/compute/manager.py:11167}}
Aug 09 02:48:47 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:48:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Acquiring lock "0d4c65d2-c641-4985-8115-24dcbc15555f" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:48:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "0d4c65d2-c641-4985-8115-24dcbc15555f" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:48:47 user nova-compute[72351]: DEBUG nova.compute.manager [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Starting instance... {{(pid=72351) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2406}}
Aug 09 02:48:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:48:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:48:47 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72351) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Aug 09 02:48:47 user nova-compute[72351]: INFO nova.compute.claims [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Claim successful on node user
Aug 09 02:48:47 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:48:47 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:48:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.254s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:48:47 user nova-compute[72351]: DEBUG nova.compute.manager [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Start building networks asynchronously for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2803}}
Aug 09 02:48:47 user nova-compute[72351]: DEBUG nova.compute.manager [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Allocating IP information in the background. {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1956}}
Aug 09 02:48:47 user nova-compute[72351]: DEBUG nova.network.neutron [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] allocate_for_instance() {{(pid=72351) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Aug 09 02:48:48 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Aug 09 02:48:48 user nova-compute[72351]: DEBUG nova.compute.manager [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Start building block device mappings for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2838}}
Aug 09 02:48:48 user nova-compute[72351]: DEBUG nova.policy [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '17f7007adf02474e9b019cd8af099ff8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c07a6b254414b41a673468238d3aa62', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72351) authorize /opt/stack/nova/nova/policy.py:203}}
Aug 09 02:48:48 user nova-compute[72351]: DEBUG nova.compute.manager [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Start spawning the instance on the hypervisor. {{(pid=72351) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2612}}
Aug 09 02:48:48 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Creating instance directory {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4717}}
Aug 09 02:48:48 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Creating image(s)
Aug 09 02:48:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Acquiring lock "/opt/stack/data/nova/instances/0d4c65d2-c641-4985-8115-24dcbc15555f/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:48:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "/opt/stack/data/nova/instances/0d4c65d2-c641-4985-8115-24dcbc15555f/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:48:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "/opt/stack/data/nova/instances/0d4c65d2-c641-4985-8115-24dcbc15555f/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:48:48 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:48:48 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.130s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:48:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Acquiring lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:48:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:48:48 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:48:48 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.127s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:48:48 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/0d4c65d2-c641-4985-8115-24dcbc15555f/disk 1073741824 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:48:48 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/0d4c65d2-c641-4985-8115-24dcbc15555f/disk 1073741824" returned: 0 in 0.048s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:48:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.183s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:48:48 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:48:48 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.159s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:48:48 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Checking if we can resize image /opt/stack/data/nova/instances/0d4c65d2-c641-4985-8115-24dcbc15555f/disk. size=1073741824 {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Aug 09 02:48:48 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/0d4c65d2-c641-4985-8115-24dcbc15555f/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:48:48 user nova-compute[72351]: DEBUG nova.network.neutron [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Successfully created port: c0c9db92-f7f2-4a6d-90d0-bfe48762cc6e {{(pid=72351) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Aug 09 02:48:48 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/0d4c65d2-c641-4985-8115-24dcbc15555f/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:48:48 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Cannot resize image /opt/stack/data/nova/instances/0d4c65d2-c641-4985-8115-24dcbc15555f/disk to a smaller size. {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Aug 09 02:48:48 user nova-compute[72351]: DEBUG nova.objects.instance [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lazy-loading 'migration_context' on Instance uuid 0d4c65d2-c641-4985-8115-24dcbc15555f {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:48:48 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Created local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4851}}
Aug 09 02:48:48 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Ensure instance console log exists: /opt/stack/data/nova/instances/0d4c65d2-c641-4985-8115-24dcbc15555f/console.log {{(pid=72351) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4603}}
Aug 09 02:48:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:48:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:48:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:48:49 user nova-compute[72351]: DEBUG nova.network.neutron [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Successfully updated port: c0c9db92-f7f2-4a6d-90d0-bfe48762cc6e {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:48:49 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Acquiring lock "refresh_cache-0d4c65d2-c641-4985-8115-24dcbc15555f" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:48:49 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Acquired lock "refresh_cache-0d4c65d2-c641-4985-8115-24dcbc15555f" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:48:49 user nova-compute[72351]: DEBUG nova.network.neutron [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:48:49 user nova-compute[72351]: DEBUG nova.network.neutron [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:48:49 user nova-compute[72351]: DEBUG nova.compute.manager [req-dfccdc1a-0c1e-4fc2-b020-90ae9946be3d req-e02eb557-9841-4820-aa07-273a0c227582 service nova] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Received event network-changed-c0c9db92-f7f2-4a6d-90d0-bfe48762cc6e {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:48:49 user nova-compute[72351]: DEBUG nova.compute.manager [req-dfccdc1a-0c1e-4fc2-b020-90ae9946be3d req-e02eb557-9841-4820-aa07-273a0c227582 service nova] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Refreshing instance network info cache due to event network-changed-c0c9db92-f7f2-4a6d-90d0-bfe48762cc6e. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:48:49 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-dfccdc1a-0c1e-4fc2-b020-90ae9946be3d req-e02eb557-9841-4820-aa07-273a0c227582 service nova] Acquiring lock "refresh_cache-0d4c65d2-c641-4985-8115-24dcbc15555f" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:48:49 user nova-compute[72351]: DEBUG nova.network.neutron [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Updating instance_info_cache with network_info: [{"id": "c0c9db92-f7f2-4a6d-90d0-bfe48762cc6e", "address": "fa:16:3e:93:37:be", "network": {"id": "47c43230-a244-4198-84e4-9ed98c005cd9", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1392403106-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c07a6b254414b41a673468238d3aa62", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc0c9db92-f7", "ovs_interfaceid": "c0c9db92-f7f2-4a6d-90d0-bfe48762cc6e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:48:49 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Releasing lock "refresh_cache-0d4c65d2-c641-4985-8115-24dcbc15555f" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:48:49 user nova-compute[72351]: DEBUG nova.compute.manager [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Instance network_info: |[{"id": "c0c9db92-f7f2-4a6d-90d0-bfe48762cc6e", "address": "fa:16:3e:93:37:be", "network": {"id": "47c43230-a244-4198-84e4-9ed98c005cd9", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1392403106-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c07a6b254414b41a673468238d3aa62", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc0c9db92-f7", "ovs_interfaceid": "c0c9db92-f7f2-4a6d-90d0-bfe48762cc6e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1971}}
Aug 09 02:48:49 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-dfccdc1a-0c1e-4fc2-b020-90ae9946be3d req-e02eb557-9841-4820-aa07-273a0c227582 service nova] Acquired lock "refresh_cache-0d4c65d2-c641-4985-8115-24dcbc15555f" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:48:49 user nova-compute[72351]: DEBUG nova.network.neutron [req-dfccdc1a-0c1e-4fc2-b020-90ae9946be3d req-e02eb557-9841-4820-aa07-273a0c227582 service nova] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Refreshing network info cache for port c0c9db92-f7f2-4a6d-90d0-bfe48762cc6e {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:48:49 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Start _get_guest_xml network_info=[{"id": "c0c9db92-f7f2-4a6d-90d0-bfe48762cc6e", "address": "fa:16:3e:93:37:be", "network": {"id": "47c43230-a244-4198-84e4-9ed98c005cd9", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1392403106-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c07a6b254414b41a673468238d3aa62", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc0c9db92-f7", "ovs_interfaceid": "c0c9db92-f7f2-4a6d-90d0-bfe48762cc6e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'size': 0, 'device_type': 'disk', 'encryption_secret_uuid': None, 'guest_format': None, 'encryption_options': None, 'disk_bus': 'virtio', 'encrypted': False, 'encryption_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'image_id': '18ccda7f-31d3-4978-ab9d-4ed2f74eff94'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:48:49 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:48:49 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:48:49 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:48:49 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:48:49 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:48:49 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:48:49 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:48:49 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:48:49 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:48:49 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:48:49 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:48:49 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:48:49 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:48:49 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:48:49 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:48:49 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:48:49 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:48:49 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:48:49 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:48:47Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachVolumeNegativeTest-server-469412905',display_name='tempest-AttachVolumeNegativeTest-server-469412905',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumenegativetest-server-469412905',id=50,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHgU3jYUGV9FAA8guL1Tj+BZlg1d7wi47Hb565DUMcyiQvfNEhdjrF/FS2zNWPjiwHt3nj4XcSDefgraL9MvGTKIwkeBYQhe+INpMsQvhbKox5N3Z6a2GDe0PbyTrwSXjw==',key_name='tempest-keypair-27411291',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='8c07a6b254414b41a673468238d3aa62',ramdisk_id='',reservation_id='r-7d8pn0x5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeNegativeTest-758771161',owner_user_name='tempest-AttachVolumeNegativeTest-758771161-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:48:48Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='17f7007adf02474e9b019cd8af099ff8',uuid=0d4c65d2-c641-4985-8115-24dcbc15555f,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "c0c9db92-f7f2-4a6d-90d0-bfe48762cc6e", "address": "fa:16:3e:93:37:be", "network": {"id": "47c43230-a244-4198-84e4-9ed98c005cd9", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1392403106-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c07a6b254414b41a673468238d3aa62", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc0c9db92-f7", "ovs_interfaceid": "c0c9db92-f7f2-4a6d-90d0-bfe48762cc6e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:48:49 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Converting VIF {"id": "c0c9db92-f7f2-4a6d-90d0-bfe48762cc6e", "address": "fa:16:3e:93:37:be", "network": {"id": "47c43230-a244-4198-84e4-9ed98c005cd9", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1392403106-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c07a6b254414b41a673468238d3aa62", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc0c9db92-f7", "ovs_interfaceid": "c0c9db92-f7f2-4a6d-90d0-bfe48762cc6e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:48:49 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:93:37:be,bridge_name='br-int',has_traffic_filtering=True,id=c0c9db92-f7f2-4a6d-90d0-bfe48762cc6e,network=Network(47c43230-a244-4198-84e4-9ed98c005cd9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc0c9db92-f7') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:48:49 user nova-compute[72351]: DEBUG nova.objects.instance [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lazy-loading 'pci_devices' on Instance uuid 0d4c65d2-c641-4985-8115-24dcbc15555f {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:48:49 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:48:49 user nova-compute[72351]:   <uuid>0d4c65d2-c641-4985-8115-24dcbc15555f</uuid>
Aug 09 02:48:49 user nova-compute[72351]:   <name>instance-00000032</name>
Aug 09 02:48:49 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:48:49 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:48:49 user nova-compute[72351]:   <metadata>
Aug 09 02:48:49 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:48:49 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:48:49 user nova-compute[72351]:       <nova:name>tempest-AttachVolumeNegativeTest-server-469412905</nova:name>
Aug 09 02:48:49 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:48:49</nova:creationTime>
Aug 09 02:48:49 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:48:49 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:48:49 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:48:49 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:48:49 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:48:49 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:48:49 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:48:49 user nova-compute[72351]:       <nova:owner>
Aug 09 02:48:49 user nova-compute[72351]:         <nova:user uuid="17f7007adf02474e9b019cd8af099ff8">tempest-AttachVolumeNegativeTest-758771161-project-member</nova:user>
Aug 09 02:48:49 user nova-compute[72351]:         <nova:project uuid="8c07a6b254414b41a673468238d3aa62">tempest-AttachVolumeNegativeTest-758771161</nova:project>
Aug 09 02:48:49 user nova-compute[72351]:       </nova:owner>
Aug 09 02:48:49 user nova-compute[72351]:       <nova:root type="image" uuid="18ccda7f-31d3-4978-ab9d-4ed2f74eff94"/>
Aug 09 02:48:49 user nova-compute[72351]:       <nova:ports>
Aug 09 02:48:49 user nova-compute[72351]:         <nova:port uuid="c0c9db92-f7f2-4a6d-90d0-bfe48762cc6e">
Aug 09 02:48:49 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.5" ipVersion="4"/>
Aug 09 02:48:49 user nova-compute[72351]:         </nova:port>
Aug 09 02:48:49 user nova-compute[72351]:       </nova:ports>
Aug 09 02:48:49 user nova-compute[72351]:     </nova:instance>
Aug 09 02:48:49 user nova-compute[72351]:   </metadata>
Aug 09 02:48:49 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:48:49 user nova-compute[72351]:     <system>
Aug 09 02:48:49 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:48:49 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:48:49 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:48:49 user nova-compute[72351]:       <entry name="serial">0d4c65d2-c641-4985-8115-24dcbc15555f</entry>
Aug 09 02:48:49 user nova-compute[72351]:       <entry name="uuid">0d4c65d2-c641-4985-8115-24dcbc15555f</entry>
Aug 09 02:48:49 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:48:49 user nova-compute[72351]:     </system>
Aug 09 02:48:49 user nova-compute[72351]:   </sysinfo>
Aug 09 02:48:49 user nova-compute[72351]:   <os>
Aug 09 02:48:49 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:48:49 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:48:49 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:48:49 user nova-compute[72351]:   </os>
Aug 09 02:48:49 user nova-compute[72351]:   <features>
Aug 09 02:48:49 user nova-compute[72351]:     <acpi/>
Aug 09 02:48:49 user nova-compute[72351]:     <apic/>
Aug 09 02:48:49 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:48:49 user nova-compute[72351]:   </features>
Aug 09 02:48:49 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:48:49 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:48:49 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:48:49 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:48:49 user nova-compute[72351]:   </clock>
Aug 09 02:48:49 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:48:49 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:48:49 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:48:49 user nova-compute[72351]:   </cpu>
Aug 09 02:48:49 user nova-compute[72351]:   <devices>
Aug 09 02:48:49 user nova-compute[72351]:     <disk type="file" device="disk">
Aug 09 02:48:49 user nova-compute[72351]:       <driver name="qemu" type="qcow2" cache="none"/>
Aug 09 02:48:49 user nova-compute[72351]:       <source file="/opt/stack/data/nova/instances/0d4c65d2-c641-4985-8115-24dcbc15555f/disk"/>
Aug 09 02:48:49 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:48:49 user nova-compute[72351]:     </disk>
Aug 09 02:48:49 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:48:49 user nova-compute[72351]:       <mac address="fa:16:3e:93:37:be"/>
Aug 09 02:48:49 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:48:49 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:48:49 user nova-compute[72351]:       <target dev="tapc0c9db92-f7"/>
Aug 09 02:48:49 user nova-compute[72351]:     </interface>
Aug 09 02:48:49 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:48:49 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/0d4c65d2-c641-4985-8115-24dcbc15555f/console.log" append="off"/>
Aug 09 02:48:49 user nova-compute[72351]:     </serial>
Aug 09 02:48:49 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:48:49 user nova-compute[72351]:     <video>
Aug 09 02:48:49 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:48:49 user nova-compute[72351]:     </video>
Aug 09 02:48:49 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:48:49 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:48:49 user nova-compute[72351]:     </rng>
Aug 09 02:48:49 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:48:49 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:48:49 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:48:49 user nova-compute[72351]:     </memballoon>
Aug 09 02:48:49 user nova-compute[72351]:   </devices>
Aug 09 02:48:49 user nova-compute[72351]: </domain>
Aug 09 02:48:49 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:48:49 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:48:47Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachVolumeNegativeTest-server-469412905',display_name='tempest-AttachVolumeNegativeTest-server-469412905',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumenegativetest-server-469412905',id=50,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHgU3jYUGV9FAA8guL1Tj+BZlg1d7wi47Hb565DUMcyiQvfNEhdjrF/FS2zNWPjiwHt3nj4XcSDefgraL9MvGTKIwkeBYQhe+INpMsQvhbKox5N3Z6a2GDe0PbyTrwSXjw==',key_name='tempest-keypair-27411291',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='8c07a6b254414b41a673468238d3aa62',ramdisk_id='',reservation_id='r-7d8pn0x5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeNegativeTest-758771161',owner_user_name='tempest-AttachVolumeNegativeTest-758771161-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:48:48Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='17f7007adf02474e9b019cd8af099ff8',uuid=0d4c65d2-c641-4985-8115-24dcbc15555f,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "c0c9db92-f7f2-4a6d-90d0-bfe48762cc6e", "address": "fa:16:3e:93:37:be", "network": {"id": "47c43230-a244-4198-84e4-9ed98c005cd9", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1392403106-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c07a6b254414b41a673468238d3aa62", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc0c9db92-f7", "ovs_interfaceid": "c0c9db92-f7f2-4a6d-90d0-bfe48762cc6e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:48:49 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Converting VIF {"id": "c0c9db92-f7f2-4a6d-90d0-bfe48762cc6e", "address": "fa:16:3e:93:37:be", "network": {"id": "47c43230-a244-4198-84e4-9ed98c005cd9", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1392403106-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c07a6b254414b41a673468238d3aa62", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc0c9db92-f7", "ovs_interfaceid": "c0c9db92-f7f2-4a6d-90d0-bfe48762cc6e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:48:49 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:93:37:be,bridge_name='br-int',has_traffic_filtering=True,id=c0c9db92-f7f2-4a6d-90d0-bfe48762cc6e,network=Network(47c43230-a244-4198-84e4-9ed98c005cd9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc0c9db92-f7') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:48:49 user nova-compute[72351]: DEBUG os_vif [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:93:37:be,bridge_name='br-int',has_traffic_filtering=True,id=c0c9db92-f7f2-4a6d-90d0-bfe48762cc6e,network=Network(47c43230-a244-4198-84e4-9ed98c005cd9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc0c9db92-f7') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:48:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:48:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:48:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:48:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:48:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapc0c9db92-f7, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:48:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapc0c9db92-f7, col_values=(('external_ids', {'iface-id': 'c0c9db92-f7f2-4a6d-90d0-bfe48762cc6e', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:93:37:be', 'vm-uuid': '0d4c65d2-c641-4985-8115-24dcbc15555f'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:48:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:48:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:48:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:48:49 user nova-compute[72351]: INFO os_vif [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:93:37:be,bridge_name='br-int',has_traffic_filtering=True,id=c0c9db92-f7f2-4a6d-90d0-bfe48762cc6e,network=Network(47c43230-a244-4198-84e4-9ed98c005cd9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc0c9db92-f7')
Aug 09 02:48:50 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:48:50 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] No VIF found with MAC fa:16:3e:93:37:be, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:48:50 user nova-compute[72351]: DEBUG nova.network.neutron [req-dfccdc1a-0c1e-4fc2-b020-90ae9946be3d req-e02eb557-9841-4820-aa07-273a0c227582 service nova] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Updated VIF entry in instance network info cache for port c0c9db92-f7f2-4a6d-90d0-bfe48762cc6e. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:48:50 user nova-compute[72351]: DEBUG nova.network.neutron [req-dfccdc1a-0c1e-4fc2-b020-90ae9946be3d req-e02eb557-9841-4820-aa07-273a0c227582 service nova] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Updating instance_info_cache with network_info: [{"id": "c0c9db92-f7f2-4a6d-90d0-bfe48762cc6e", "address": "fa:16:3e:93:37:be", "network": {"id": "47c43230-a244-4198-84e4-9ed98c005cd9", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1392403106-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c07a6b254414b41a673468238d3aa62", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc0c9db92-f7", "ovs_interfaceid": "c0c9db92-f7f2-4a6d-90d0-bfe48762cc6e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:48:50 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-dfccdc1a-0c1e-4fc2-b020-90ae9946be3d req-e02eb557-9841-4820-aa07-273a0c227582 service nova] Releasing lock "refresh_cache-0d4c65d2-c641-4985-8115-24dcbc15555f" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:48:51 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:48:51 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:48:51 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:48:51 user nova-compute[72351]: DEBUG nova.compute.manager [req-469546f9-93d6-4cd5-9477-c6014bcb0809 req-aa4e773a-2d65-439f-a319-404b28ab68de service nova] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Received event network-vif-plugged-c0c9db92-f7f2-4a6d-90d0-bfe48762cc6e {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:48:51 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-469546f9-93d6-4cd5-9477-c6014bcb0809 req-aa4e773a-2d65-439f-a319-404b28ab68de service nova] Acquiring lock "0d4c65d2-c641-4985-8115-24dcbc15555f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:48:51 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-469546f9-93d6-4cd5-9477-c6014bcb0809 req-aa4e773a-2d65-439f-a319-404b28ab68de service nova] Lock "0d4c65d2-c641-4985-8115-24dcbc15555f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:48:51 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-469546f9-93d6-4cd5-9477-c6014bcb0809 req-aa4e773a-2d65-439f-a319-404b28ab68de service nova] Lock "0d4c65d2-c641-4985-8115-24dcbc15555f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:48:51 user nova-compute[72351]: DEBUG nova.compute.manager [req-469546f9-93d6-4cd5-9477-c6014bcb0809 req-aa4e773a-2d65-439f-a319-404b28ab68de service nova] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] No waiting events found dispatching network-vif-plugged-c0c9db92-f7f2-4a6d-90d0-bfe48762cc6e {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:48:51 user nova-compute[72351]: WARNING nova.compute.manager [req-469546f9-93d6-4cd5-9477-c6014bcb0809 req-aa4e773a-2d65-439f-a319-404b28ab68de service nova] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Received unexpected event network-vif-plugged-c0c9db92-f7f2-4a6d-90d0-bfe48762cc6e for instance with vm_state building and task_state spawning.
Aug 09 02:48:51 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:48:51 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:48:51 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:48:51 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:48:53 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691549333.4141142, 0d4c65d2-c641-4985-8115-24dcbc15555f => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:48:53 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] VM Resumed (Lifecycle Event)
Aug 09 02:48:53 user nova-compute[72351]: DEBUG nova.compute.manager [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:48:53 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Guest created on hypervisor {{(pid=72351) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4411}}
Aug 09 02:48:53 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Instance spawned successfully.
Aug 09 02:48:53 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:894}}
Aug 09 02:48:53 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:48:53 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:48:53 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Found default for hw_cdrom_bus of ide {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:48:53 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Found default for hw_disk_bus of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:48:53 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Found default for hw_input_bus of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:48:53 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Found default for hw_pointer_model of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:48:53 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Found default for hw_video_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:48:53 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Found default for hw_vif_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:48:53 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:48:53 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691549333.4182236, 0d4c65d2-c641-4985-8115-24dcbc15555f => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:48:53 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] VM Started (Lifecycle Event)
Aug 09 02:48:53 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:48:53 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:48:53 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:48:53 user nova-compute[72351]: INFO nova.compute.manager [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Took 5.38 seconds to spawn the instance on the hypervisor.
Aug 09 02:48:53 user nova-compute[72351]: DEBUG nova.compute.manager [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:48:53 user nova-compute[72351]: INFO nova.compute.manager [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Took 5.96 seconds to build instance.
Aug 09 02:48:53 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-c616af98-2d92-4871-a38c-5dfa94ee5bf5 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "0d4c65d2-c641-4985-8115-24dcbc15555f" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 6.071s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:48:53 user nova-compute[72351]: DEBUG nova.compute.manager [req-a8219341-0507-4383-86dd-218fc91daacf req-44f3f1b6-0cf6-42d2-8a87-58bb31c18f8a service nova] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Received event network-vif-plugged-c0c9db92-f7f2-4a6d-90d0-bfe48762cc6e {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:48:53 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a8219341-0507-4383-86dd-218fc91daacf req-44f3f1b6-0cf6-42d2-8a87-58bb31c18f8a service nova] Acquiring lock "0d4c65d2-c641-4985-8115-24dcbc15555f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:48:53 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a8219341-0507-4383-86dd-218fc91daacf req-44f3f1b6-0cf6-42d2-8a87-58bb31c18f8a service nova] Lock "0d4c65d2-c641-4985-8115-24dcbc15555f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:48:53 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a8219341-0507-4383-86dd-218fc91daacf req-44f3f1b6-0cf6-42d2-8a87-58bb31c18f8a service nova] Lock "0d4c65d2-c641-4985-8115-24dcbc15555f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:48:53 user nova-compute[72351]: DEBUG nova.compute.manager [req-a8219341-0507-4383-86dd-218fc91daacf req-44f3f1b6-0cf6-42d2-8a87-58bb31c18f8a service nova] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] No waiting events found dispatching network-vif-plugged-c0c9db92-f7f2-4a6d-90d0-bfe48762cc6e {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:48:53 user nova-compute[72351]: WARNING nova.compute.manager [req-a8219341-0507-4383-86dd-218fc91daacf req-44f3f1b6-0cf6-42d2-8a87-58bb31c18f8a service nova] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Received unexpected event network-vif-plugged-c0c9db92-f7f2-4a6d-90d0-bfe48762cc6e for instance with vm_state active and task_state None.
Aug 09 02:48:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:48:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:48:58 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:48:59 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:49:04 user nova-compute[72351]: DEBUG nova.compute.manager [req-2355b7ab-aac0-41cf-a6b3-0b0c54d32c9a req-533a9960-8369-49f3-b1de-350073b67927 service nova] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Received event network-changed-0983efc8-e53e-40bd-a721-2924155c20f3 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:49:04 user nova-compute[72351]: DEBUG nova.compute.manager [req-2355b7ab-aac0-41cf-a6b3-0b0c54d32c9a req-533a9960-8369-49f3-b1de-350073b67927 service nova] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Refreshing instance network info cache due to event network-changed-0983efc8-e53e-40bd-a721-2924155c20f3. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:49:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-2355b7ab-aac0-41cf-a6b3-0b0c54d32c9a req-533a9960-8369-49f3-b1de-350073b67927 service nova] Acquiring lock "refresh_cache-150f1b69-aa23-4702-a9f4-24025963befe" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:49:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-2355b7ab-aac0-41cf-a6b3-0b0c54d32c9a req-533a9960-8369-49f3-b1de-350073b67927 service nova] Acquired lock "refresh_cache-150f1b69-aa23-4702-a9f4-24025963befe" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:49:04 user nova-compute[72351]: DEBUG nova.network.neutron [req-2355b7ab-aac0-41cf-a6b3-0b0c54d32c9a req-533a9960-8369-49f3-b1de-350073b67927 service nova] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Refreshing network info cache for port 0983efc8-e53e-40bd-a721-2924155c20f3 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:49:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:49:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:49:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5001 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 02:49:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:49:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:49:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:49:05 user nova-compute[72351]: DEBUG nova.network.neutron [req-2355b7ab-aac0-41cf-a6b3-0b0c54d32c9a req-533a9960-8369-49f3-b1de-350073b67927 service nova] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Updated VIF entry in instance network info cache for port 0983efc8-e53e-40bd-a721-2924155c20f3. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:49:05 user nova-compute[72351]: DEBUG nova.network.neutron [req-2355b7ab-aac0-41cf-a6b3-0b0c54d32c9a req-533a9960-8369-49f3-b1de-350073b67927 service nova] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Updating instance_info_cache with network_info: [{"id": "0983efc8-e53e-40bd-a721-2924155c20f3", "address": "fa:16:3e:d7:24:ef", "network": {"id": "da9be96c-ce20-4037-999c-230ca799fb06", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-661780305-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.240", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5ff300d27c9a4dd3b7ecd507b3627d98", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0983efc8-e5", "ovs_interfaceid": "0983efc8-e53e-40bd-a721-2924155c20f3", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:49:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-2355b7ab-aac0-41cf-a6b3-0b0c54d32c9a req-533a9960-8369-49f3-b1de-350073b67927 service nova] Releasing lock "refresh_cache-150f1b69-aa23-4702-a9f4-24025963befe" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:49:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:49:09 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-fe901d4d-3d6b-4188-9de6-70642d22f6d3 tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Acquiring lock "150f1b69-aa23-4702-a9f4-24025963befe" by "nova.compute.manager.ComputeManager.shelve_instance.<locals>.do_shelve_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:49:09 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-fe901d4d-3d6b-4188-9de6-70642d22f6d3 tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "150f1b69-aa23-4702-a9f4-24025963befe" acquired by "nova.compute.manager.ComputeManager.shelve_instance.<locals>.do_shelve_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:49:09 user nova-compute[72351]: INFO nova.compute.manager [None req-fe901d4d-3d6b-4188-9de6-70642d22f6d3 tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Shelving
Aug 09 02:49:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:49:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:49:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:49:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:49:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:49:10 user nova-compute[72351]: DEBUG nova.compute.manager [req-0e63d1a8-9e91-4f66-8d23-10fa125eae36 req-9a4dc5f8-e3d7-4450-bca5-105b5247c404 service nova] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Received event network-vif-unplugged-0983efc8-e53e-40bd-a721-2924155c20f3 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:49:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-0e63d1a8-9e91-4f66-8d23-10fa125eae36 req-9a4dc5f8-e3d7-4450-bca5-105b5247c404 service nova] Acquiring lock "150f1b69-aa23-4702-a9f4-24025963befe-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:49:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-0e63d1a8-9e91-4f66-8d23-10fa125eae36 req-9a4dc5f8-e3d7-4450-bca5-105b5247c404 service nova] Lock "150f1b69-aa23-4702-a9f4-24025963befe-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:49:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-0e63d1a8-9e91-4f66-8d23-10fa125eae36 req-9a4dc5f8-e3d7-4450-bca5-105b5247c404 service nova] Lock "150f1b69-aa23-4702-a9f4-24025963befe-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:49:10 user nova-compute[72351]: DEBUG nova.compute.manager [req-0e63d1a8-9e91-4f66-8d23-10fa125eae36 req-9a4dc5f8-e3d7-4450-bca5-105b5247c404 service nova] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] No waiting events found dispatching network-vif-unplugged-0983efc8-e53e-40bd-a721-2924155c20f3 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:49:10 user nova-compute[72351]: WARNING nova.compute.manager [req-0e63d1a8-9e91-4f66-8d23-10fa125eae36 req-9a4dc5f8-e3d7-4450-bca5-105b5247c404 service nova] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Received unexpected event network-vif-unplugged-0983efc8-e53e-40bd-a721-2924155c20f3 for instance with vm_state active and task_state shelving.
Aug 09 02:49:10 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Instance destroyed successfully.
Aug 09 02:49:10 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-fe901d4d-3d6b-4188-9de6-70642d22f6d3 tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Beginning cold snapshot process
Aug 09 02:49:10 user nova-compute[72351]: DEBUG nova.privsep.utils [None req-fe901d4d-3d6b-4188-9de6-70642d22f6d3 tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=72351) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Aug 09 02:49:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-fe901d4d-3d6b-4188-9de6-70642d22f6d3 tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Running cmd (subprocess): qemu-img convert -t none -O qcow2 -f qcow2 /opt/stack/data/nova/instances/150f1b69-aa23-4702-a9f4-24025963befe/disk /opt/stack/data/nova/instances/snapshots/tmpuny4qwzj/dce34fce607e4601a300de3e3ddc39c7 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:49:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-fe901d4d-3d6b-4188-9de6-70642d22f6d3 tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] CMD "qemu-img convert -t none -O qcow2 -f qcow2 /opt/stack/data/nova/instances/150f1b69-aa23-4702-a9f4-24025963befe/disk /opt/stack/data/nova/instances/snapshots/tmpuny4qwzj/dce34fce607e4601a300de3e3ddc39c7" returned: 0 in 0.468s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:49:11 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-fe901d4d-3d6b-4188-9de6-70642d22f6d3 tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Snapshot extracted, beginning image upload
Aug 09 02:49:12 user nova-compute[72351]: DEBUG nova.compute.manager [req-c9a41019-e2a3-4294-ad05-bdab80c41953 req-c9e086f0-e1b7-4760-82da-fcef579aac69 service nova] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Received event network-vif-plugged-0983efc8-e53e-40bd-a721-2924155c20f3 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:49:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c9a41019-e2a3-4294-ad05-bdab80c41953 req-c9e086f0-e1b7-4760-82da-fcef579aac69 service nova] Acquiring lock "150f1b69-aa23-4702-a9f4-24025963befe-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:49:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c9a41019-e2a3-4294-ad05-bdab80c41953 req-c9e086f0-e1b7-4760-82da-fcef579aac69 service nova] Lock "150f1b69-aa23-4702-a9f4-24025963befe-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:49:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c9a41019-e2a3-4294-ad05-bdab80c41953 req-c9e086f0-e1b7-4760-82da-fcef579aac69 service nova] Lock "150f1b69-aa23-4702-a9f4-24025963befe-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:49:12 user nova-compute[72351]: DEBUG nova.compute.manager [req-c9a41019-e2a3-4294-ad05-bdab80c41953 req-c9e086f0-e1b7-4760-82da-fcef579aac69 service nova] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] No waiting events found dispatching network-vif-plugged-0983efc8-e53e-40bd-a721-2924155c20f3 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:49:12 user nova-compute[72351]: WARNING nova.compute.manager [req-c9a41019-e2a3-4294-ad05-bdab80c41953 req-c9e086f0-e1b7-4760-82da-fcef579aac69 service nova] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Received unexpected event network-vif-plugged-0983efc8-e53e-40bd-a721-2924155c20f3 for instance with vm_state active and task_state shelving_image_uploading.
Aug 09 02:49:13 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-fe901d4d-3d6b-4188-9de6-70642d22f6d3 tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Snapshot image upload complete
Aug 09 02:49:13 user nova-compute[72351]: DEBUG nova.compute.manager [None req-fe901d4d-3d6b-4188-9de6-70642d22f6d3 tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:49:13 user nova-compute[72351]: INFO nova.compute.manager [None req-fe901d4d-3d6b-4188-9de6-70642d22f6d3 tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Shelve offloading
Aug 09 02:49:13 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Instance destroyed successfully.
Aug 09 02:49:13 user nova-compute[72351]: DEBUG nova.compute.manager [None req-fe901d4d-3d6b-4188-9de6-70642d22f6d3 tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:49:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-fe901d4d-3d6b-4188-9de6-70642d22f6d3 tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Acquiring lock "refresh_cache-150f1b69-aa23-4702-a9f4-24025963befe" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:49:13 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-fe901d4d-3d6b-4188-9de6-70642d22f6d3 tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Acquired lock "refresh_cache-150f1b69-aa23-4702-a9f4-24025963befe" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:49:13 user nova-compute[72351]: DEBUG nova.network.neutron [None req-fe901d4d-3d6b-4188-9de6-70642d22f6d3 tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:49:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:49:14 user nova-compute[72351]: DEBUG nova.network.neutron [None req-fe901d4d-3d6b-4188-9de6-70642d22f6d3 tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Updating instance_info_cache with network_info: [{"id": "0983efc8-e53e-40bd-a721-2924155c20f3", "address": "fa:16:3e:d7:24:ef", "network": {"id": "da9be96c-ce20-4037-999c-230ca799fb06", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-661780305-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.240", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5ff300d27c9a4dd3b7ecd507b3627d98", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0983efc8-e5", "ovs_interfaceid": "0983efc8-e53e-40bd-a721-2924155c20f3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:49:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-fe901d4d-3d6b-4188-9de6-70642d22f6d3 tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Releasing lock "refresh_cache-150f1b69-aa23-4702-a9f4-24025963befe" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:49:14 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Instance destroyed successfully.
Aug 09 02:49:14 user nova-compute[72351]: DEBUG nova.objects.instance [None req-fe901d4d-3d6b-4188-9de6-70642d22f6d3 tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lazy-loading 'resources' on Instance uuid 150f1b69-aa23-4702-a9f4-24025963befe {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:49:14 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-fe901d4d-3d6b-4188-9de6-70642d22f6d3 tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:47:13Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-server-547567417',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumeshelvetestjson-server-547567417',id=49,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBC6Kt1VBrwPlXoj775IMgThdXYe/hd3G1m2PhSYFaCtMLpu/4nO1MCXo8m5UYekLtMKzV6viB1JRAwTbgfITOhrGcZO/1RTBqsi/z0vUwAuglB5045WiXDwqXuvJuthb5A==',key_name='tempest-keypair-647885492',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:47:19Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=<?>,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=4,progress=0,project_id='5ff300d27c9a4dd3b7ecd507b3627d98',ramdisk_id='',reservation_id='r-x20xmo7b',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeShelveTestJSON-1982056968',owner_user_name='tempest-AttachVolumeShelveTestJSON-1982056968-project-member',shelved_at='2023-08-09T02:49:13.369328',shelved_host='user',shelved_image_id='84e39abb-60a1-45ad-b0d9-5bf219191a3f'},tags=<?>,task_state='shelving_offloading',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:49:11Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='ad8fc03436a84bdfb13ba6abecd4f27b',uuid=150f1b69-aa23-4702-a9f4-24025963befe,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='shelved') vif={"id": "0983efc8-e53e-40bd-a721-2924155c20f3", "address": "fa:16:3e:d7:24:ef", "network": {"id": "da9be96c-ce20-4037-999c-230ca799fb06", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-661780305-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.240", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5ff300d27c9a4dd3b7ecd507b3627d98", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0983efc8-e5", "ovs_interfaceid": "0983efc8-e53e-40bd-a721-2924155c20f3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:49:14 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-fe901d4d-3d6b-4188-9de6-70642d22f6d3 tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Converting VIF {"id": "0983efc8-e53e-40bd-a721-2924155c20f3", "address": "fa:16:3e:d7:24:ef", "network": {"id": "da9be96c-ce20-4037-999c-230ca799fb06", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-661780305-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.240", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5ff300d27c9a4dd3b7ecd507b3627d98", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0983efc8-e5", "ovs_interfaceid": "0983efc8-e53e-40bd-a721-2924155c20f3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:49:14 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-fe901d4d-3d6b-4188-9de6-70642d22f6d3 tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:d7:24:ef,bridge_name='br-int',has_traffic_filtering=True,id=0983efc8-e53e-40bd-a721-2924155c20f3,network=Network(da9be96c-ce20-4037-999c-230ca799fb06),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap0983efc8-e5') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:49:14 user nova-compute[72351]: DEBUG os_vif [None req-fe901d4d-3d6b-4188-9de6-70642d22f6d3 tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:d7:24:ef,bridge_name='br-int',has_traffic_filtering=True,id=0983efc8-e53e-40bd-a721-2924155c20f3,network=Network(da9be96c-ce20-4037-999c-230ca799fb06),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap0983efc8-e5') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:49:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:49:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap0983efc8-e5, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:49:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:49:14 user nova-compute[72351]: DEBUG nova.compute.manager [req-df515164-0525-44c8-b1d7-c81be08c486c req-cf4a3187-232a-44f9-8844-7f49a7c7bf3a service nova] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Received event network-changed-0983efc8-e53e-40bd-a721-2924155c20f3 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:49:14 user nova-compute[72351]: DEBUG nova.compute.manager [req-df515164-0525-44c8-b1d7-c81be08c486c req-cf4a3187-232a-44f9-8844-7f49a7c7bf3a service nova] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Refreshing instance network info cache due to event network-changed-0983efc8-e53e-40bd-a721-2924155c20f3. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:49:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-df515164-0525-44c8-b1d7-c81be08c486c req-cf4a3187-232a-44f9-8844-7f49a7c7bf3a service nova] Acquiring lock "refresh_cache-150f1b69-aa23-4702-a9f4-24025963befe" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:49:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-df515164-0525-44c8-b1d7-c81be08c486c req-cf4a3187-232a-44f9-8844-7f49a7c7bf3a service nova] Acquired lock "refresh_cache-150f1b69-aa23-4702-a9f4-24025963befe" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:49:14 user nova-compute[72351]: DEBUG nova.network.neutron [req-df515164-0525-44c8-b1d7-c81be08c486c req-cf4a3187-232a-44f9-8844-7f49a7c7bf3a service nova] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Refreshing network info cache for port 0983efc8-e53e-40bd-a721-2924155c20f3 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:49:14 user nova-compute[72351]: INFO os_vif [None req-fe901d4d-3d6b-4188-9de6-70642d22f6d3 tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:d7:24:ef,bridge_name='br-int',has_traffic_filtering=True,id=0983efc8-e53e-40bd-a721-2924155c20f3,network=Network(da9be96c-ce20-4037-999c-230ca799fb06),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap0983efc8-e5')
Aug 09 02:49:14 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-fe901d4d-3d6b-4188-9de6-70642d22f6d3 tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Deleting instance files /opt/stack/data/nova/instances/150f1b69-aa23-4702-a9f4-24025963befe_del
Aug 09 02:49:14 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-fe901d4d-3d6b-4188-9de6-70642d22f6d3 tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Deletion of /opt/stack/data/nova/instances/150f1b69-aa23-4702-a9f4-24025963befe_del complete
Aug 09 02:49:14 user nova-compute[72351]: INFO nova.scheduler.client.report [None req-fe901d4d-3d6b-4188-9de6-70642d22f6d3 tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Deleted allocations for instance 150f1b69-aa23-4702-a9f4-24025963befe
Aug 09 02:49:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-fe901d4d-3d6b-4188-9de6-70642d22f6d3 tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:49:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-fe901d4d-3d6b-4188-9de6-70642d22f6d3 tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:49:15 user nova-compute[72351]: DEBUG nova.objects.instance [None req-fe901d4d-3d6b-4188-9de6-70642d22f6d3 tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lazy-loading 'numa_topology' on Instance uuid 150f1b69-aa23-4702-a9f4-24025963befe {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:49:15 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-fe901d4d-3d6b-4188-9de6-70642d22f6d3 tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:49:15 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-fe901d4d-3d6b-4188-9de6-70642d22f6d3 tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:49:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-fe901d4d-3d6b-4188-9de6-70642d22f6d3 tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.173s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:49:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-fe901d4d-3d6b-4188-9de6-70642d22f6d3 tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "150f1b69-aa23-4702-a9f4-24025963befe" "released" by "nova.compute.manager.ComputeManager.shelve_instance.<locals>.do_shelve_instance" :: held 5.551s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:49:15 user nova-compute[72351]: DEBUG nova.network.neutron [req-df515164-0525-44c8-b1d7-c81be08c486c req-cf4a3187-232a-44f9-8844-7f49a7c7bf3a service nova] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Updated VIF entry in instance network info cache for port 0983efc8-e53e-40bd-a721-2924155c20f3. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:49:15 user nova-compute[72351]: DEBUG nova.network.neutron [req-df515164-0525-44c8-b1d7-c81be08c486c req-cf4a3187-232a-44f9-8844-7f49a7c7bf3a service nova] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Updating instance_info_cache with network_info: [{"id": "0983efc8-e53e-40bd-a721-2924155c20f3", "address": "fa:16:3e:d7:24:ef", "network": {"id": "da9be96c-ce20-4037-999c-230ca799fb06", "bridge": null, "label": "tempest-AttachVolumeShelveTestJSON-661780305-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.240", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5ff300d27c9a4dd3b7ecd507b3627d98", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "unbound", "details": {}, "devname": "tap0983efc8-e5", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:49:15 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-df515164-0525-44c8-b1d7-c81be08c486c req-cf4a3187-232a-44f9-8844-7f49a7c7bf3a service nova] Releasing lock "refresh_cache-150f1b69-aa23-4702-a9f4-24025963befe" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:49:16 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:49:17 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:49:17 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72351) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10461}}
Aug 09 02:49:18 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:49:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:49:19 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:49:20 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:49:20 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Starting heal instance info cache {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9842}}
Aug 09 02:49:20 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Rebuilding the list of instances to heal {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9846}}
Aug 09 02:49:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "refresh_cache-c6ae885f-dce2-4dc2-98d5-26139b8fb67e" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:49:20 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquired lock "refresh_cache-c6ae885f-dce2-4dc2-98d5-26139b8fb67e" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:49:20 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Forcefully refreshing network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1996}}
Aug 09 02:49:20 user nova-compute[72351]: DEBUG nova.objects.instance [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lazy-loading 'info_cache' on Instance uuid c6ae885f-dce2-4dc2-98d5-26139b8fb67e {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:49:21 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Updating instance_info_cache with network_info: [{"id": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "address": "fa:16:3e:dd:49:4d", "network": {"id": "dba57c3d-6efd-4057-8053-1c6a92bb24f8", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-568585773-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.116", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ec9861e0882e4065ab99ad3f74d6b25f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2771d93d-fa", "ovs_interfaceid": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:49:21 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Releasing lock "refresh_cache-c6ae885f-dce2-4dc2-98d5-26139b8fb67e" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:49:21 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Updated the network info_cache for instance {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9913}}
Aug 09 02:49:22 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:49:23 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:49:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:49:24 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:49:24 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:49:24 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:49:24 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:49:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:49:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:49:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:49:24 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Auditing locally available compute resources for user (node: user) {{(pid=72351) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Aug 09 02:49:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/0d4c65d2-c641-4985-8115-24dcbc15555f/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:49:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/0d4c65d2-c641-4985-8115-24dcbc15555f/disk --force-share --output=json" returned: 0 in 0.142s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:49:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/0d4c65d2-c641-4985-8115-24dcbc15555f/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:49:25 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691549350.2955766, 150f1b69-aa23-4702-a9f4-24025963befe => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:49:25 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] VM Stopped (Lifecycle Event)
Aug 09 02:49:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/0d4c65d2-c641-4985-8115-24dcbc15555f/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:49:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:49:25 user nova-compute[72351]: DEBUG nova.compute.manager [None req-8635a0dc-9b60-42b6-9371-7302416cf327 None None] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:49:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:49:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:49:25 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk --force-share --output=json" returned: 0 in 0.152s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:49:25 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] skipping disk /dev/sdb (vdb) as it is a volume {{(pid=72351) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11216}}
Aug 09 02:49:26 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:49:26 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:49:26 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Hypervisor/Node resource view: name=user free_ram=8573MB free_disk=25.19739532470703GB free_vcpus=10 pci_devices=[{"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72351) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Aug 09 02:49:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:49:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:49:26 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance c6ae885f-dce2-4dc2-98d5-26139b8fb67e actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:49:26 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 0d4c65d2-c641-4985-8115-24dcbc15555f actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:49:26 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Total usable vcpus: 12, total allocated vcpus: 2 {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Aug 09 02:49:26 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Final resource view: name=user phys_ram=16011MB used_ram=1536MB phys_disk=40GB used_disk=2GB total_vcpus=12 used_vcpus=2 pci_stats=[] {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Aug 09 02:49:26 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Refreshing inventories for resource provider 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:804}}
Aug 09 02:49:26 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Updating ProviderTree inventory for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 from _refresh_and_get_inventory using data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) _refresh_and_get_inventory /opt/stack/nova/nova/scheduler/client/report.py:768}}
Aug 09 02:49:26 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Updating inventory in ProviderTree for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 with inventory: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:176}}
Aug 09 02:49:26 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Refreshing aggregate associations for resource provider 8eae9b37-1236-4e6d-91a5-b650de27b009, aggregates: None {{(pid=72351) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:813}}
Aug 09 02:49:26 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Refreshing trait associations for resource provider 8eae9b37-1236-4e6d-91a5-b650de27b009, traits: COMPUTE_VIOMMU_MODEL_INTEL,COMPUTE_STORAGE_BUS_IDE,COMPUTE_GRAPHICS_MODEL_NONE,COMPUTE_NET_VIF_MODEL_E1000E,COMPUTE_IMAGE_TYPE_AKI,COMPUTE_VOLUME_ATTACH_WITH_TAG,COMPUTE_NET_VIF_MODEL_VMXNET3,COMPUTE_SOCKET_PCI_NUMA_AFFINITY,COMPUTE_ACCELERATORS,HW_CPU_X86_SSE41,COMPUTE_NET_ATTACH_INTERFACE_WITH_TAG,COMPUTE_GRAPHICS_MODEL_VGA,COMPUTE_GRAPHICS_MODEL_QXL,COMPUTE_VIOMMU_MODEL_AUTO,COMPUTE_SECURITY_UEFI_SECURE_BOOT,COMPUTE_NET_VIF_MODEL_E1000,COMPUTE_TRUSTED_CERTS,COMPUTE_VOLUME_MULTI_ATTACH,COMPUTE_STORAGE_BUS_FDC,COMPUTE_VOLUME_EXTEND,COMPUTE_GRAPHICS_MODEL_CIRRUS,COMPUTE_GRAPHICS_MODEL_VMVGA,COMPUTE_GRAPHICS_MODEL_BOCHS,COMPUTE_DEVICE_TAGGING,COMPUTE_STORAGE_BUS_SCSI,COMPUTE_STORAGE_BUS_VIRTIO,HW_CPU_X86_SSSE3,HW_CPU_X86_SSE2,COMPUTE_IMAGE_TYPE_AMI,COMPUTE_STORAGE_BUS_USB,COMPUTE_NET_VIF_MODEL_PCNET,COMPUTE_NET_VIF_MODEL_VIRTIO,COMPUTE_IMAGE_TYPE_ISO,HW_CPU_X86_SSE42,HW_CPU_X86_SSE,COMPUTE_IMAGE_TYPE_ARI,COMPUTE_IMAGE_TYPE_RAW,COMPUTE_IMAGE_TYPE_QCOW2,COMPUTE_NET_VIF_MODEL_NE2K_PCI,COMPUTE_NODE,HW_CPU_X86_MMX,COMPUTE_NET_VIF_MODEL_RTL8139,COMPUTE_GRAPHICS_MODEL_VIRTIO,COMPUTE_NET_VIF_MODEL_SPAPR_VLAN,COMPUTE_STORAGE_BUS_SATA,COMPUTE_NET_ATTACH_INTERFACE,COMPUTE_RESCUE_BFV {{(pid=72351) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:825}}
Aug 09 02:49:26 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:49:26 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:49:26 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Compute_service record updated for user:user {{(pid=72351) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Aug 09 02:49:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.415s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:49:29 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:49:29 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:49:34 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:49:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:49:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:49:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:49:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:49:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:49:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-4f4d7a74-b2e8-485f-a716-520b298dcb4b tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Acquiring lock "c6ae885f-dce2-4dc2-98d5-26139b8fb67e" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:49:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-4f4d7a74-b2e8-485f-a716-520b298dcb4b tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "c6ae885f-dce2-4dc2-98d5-26139b8fb67e" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:49:55 user nova-compute[72351]: INFO nova.compute.manager [None req-4f4d7a74-b2e8-485f-a716-520b298dcb4b tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Detaching volume 008f5fa6-2644-4c22-b8cd-28e233cf8857
Aug 09 02:49:55 user nova-compute[72351]: INFO nova.virt.block_device [None req-4f4d7a74-b2e8-485f-a716-520b298dcb4b tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Attempting to driver detach volume 008f5fa6-2644-4c22-b8cd-28e233cf8857 from mountpoint /dev/vdb
Aug 09 02:49:55 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-4f4d7a74-b2e8-485f-a716-520b298dcb4b tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Attempting to detach device vdb from instance c6ae885f-dce2-4dc2-98d5-26139b8fb67e from the persistent domain config. {{(pid=72351) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2477}}
Aug 09 02:49:55 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-4f4d7a74-b2e8-485f-a716-520b298dcb4b tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 02:49:55 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:49:55 user nova-compute[72351]:   <source dev="/dev/sdb"/>
Aug 09 02:49:55 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:49:55 user nova-compute[72351]:   <serial>008f5fa6-2644-4c22-b8cd-28e233cf8857</serial>
Aug 09 02:49:55 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x05" function="0x0"/>
Aug 09 02:49:55 user nova-compute[72351]: </disk>
Aug 09 02:49:55 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 02:49:55 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-4f4d7a74-b2e8-485f-a716-520b298dcb4b tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Successfully detached device vdb from instance c6ae885f-dce2-4dc2-98d5-26139b8fb67e from the persistent domain config.
Aug 09 02:49:55 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-4f4d7a74-b2e8-485f-a716-520b298dcb4b tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance c6ae885f-dce2-4dc2-98d5-26139b8fb67e from the live domain config. {{(pid=72351) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2513}}
Aug 09 02:49:55 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-4f4d7a74-b2e8-485f-a716-520b298dcb4b tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 02:49:55 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:49:55 user nova-compute[72351]:   <source dev="/dev/sdb"/>
Aug 09 02:49:55 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:49:55 user nova-compute[72351]:   <serial>008f5fa6-2644-4c22-b8cd-28e233cf8857</serial>
Aug 09 02:49:55 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x05" function="0x0"/>
Aug 09 02:49:55 user nova-compute[72351]: </disk>
Aug 09 02:49:55 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 02:49:56 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Received event <DeviceRemovedEvent: 1691549396.1251583, c6ae885f-dce2-4dc2-98d5-26139b8fb67e => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=72351) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2360}}
Aug 09 02:49:56 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-4f4d7a74-b2e8-485f-a716-520b298dcb4b tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance c6ae885f-dce2-4dc2-98d5-26139b8fb67e {{(pid=72351) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2589}}
Aug 09 02:49:56 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-4f4d7a74-b2e8-485f-a716-520b298dcb4b tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Successfully detached device vdb from instance c6ae885f-dce2-4dc2-98d5-26139b8fb67e from the live domain config.
Aug 09 02:49:56 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-4f4d7a74-b2e8-485f-a716-520b298dcb4b tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 02:49:56 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f4d7a74-b2e8-485f-a716-520b298dcb4b tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'QgoDJMbr', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb'}, None), 'kwargs': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:49:56 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-4f4d7a74-b2e8-485f-a716-520b298dcb4b tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:49:56 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-4f4d7a74-b2e8-485f-a716-520b298dcb4b tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:49:56 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-4f4d7a74-b2e8-485f-a716-520b298dcb4b tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:49:56 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f4d7a74-b2e8-485f-a716-520b298dcb4b tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 02:49:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:49:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:49:56 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d3e1d8e9-a917-49b2-ad27-1201e36d19d7]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:49:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:49:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:49:56 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[c5a04a2b-2e73-4222-b1ff-ed4803badc3d]: (4, ('tcp: [60] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:49:56 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f4d7a74-b2e8-485f-a716-520b298dcb4b tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [60] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 (non-flash)
Aug 09 02:49:56 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:49:56 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f4d7a74-b2e8-485f-a716-520b298dcb4b tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsi session list stdout=tcp: [60] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 (non-flash)
Aug 09 02:49:56 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:49:56 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f4d7a74-b2e8-485f-a716-520b298dcb4b tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fcfbc707d00>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857'): ({'sdb'}, set())}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 02:49:56 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-4f4d7a74-b2e8-485f-a716-520b298dcb4b tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Removing single pathed devices sdb {{(pid=72351) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Aug 09 02:49:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:49:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:49:56 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[9551bec7-a329-4fd8-92b0-2824457919e1]: (4, ('path checker states:\nup                  1\n\npaths: 0\nbusy: False\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:49:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:49:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:49:56 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a64df98d-707f-48f7-b78e-e1e4b64dfdcd]: (4, ('ok\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:49:56 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-4f4d7a74-b2e8-485f-a716-520b298dcb4b tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Flushing IO for device /dev/sdb {{(pid=72351) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Aug 09 02:49:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:49:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.013s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:49:56 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ac5b4542-153b-4c44-be4c-225ea46e9034]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:49:56 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-4f4d7a74-b2e8-485f-a716-520b298dcb4b tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=72351) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Aug 09 02:49:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:49:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.043s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:49:56 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[f85adbbd-7554-4eb2-98c5-d16ea93a24c0]: (4, ('1', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:49:56 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-4f4d7a74-b2e8-485f-a716-520b298dcb4b tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Aug 09 02:49:56 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-4f4d7a74-b2e8-485f-a716-520b298dcb4b tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] SCSI volumes sdb have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Aug 09 02:49:56 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f4d7a74-b2e8-485f-a716-520b298dcb4b tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857')] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 02:49:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:49:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:49:56 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[feccf78e-1f28-4c55-9e2a-47b12389e1ce]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:49:56 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f4d7a74-b2e8-485f-a716-520b298dcb4b tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:49:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 -p 172.16.0.220:3260 --logout {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:49:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 -p 172.16.0.220:3260 --logout" returned: 0 in 0.042s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:49:56 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[29955175-a3e2-4b32-858b-12837f791372]: (4, ('Logging out of session [sid: 60, target: iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857, portal: 172.16.0.220,3260]\nLogout of [sid: 60, target: iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:49:56 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f4d7a74-b2e8-485f-a716-520b298dcb4b tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 60, target: iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857, portal: 172.16.0.220,3260]
Aug 09 02:49:56 user nova-compute[72351]: Logout of [sid: 60, target: iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857, portal: 172.16.0.220,3260] successful.
Aug 09 02:49:56 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:49:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 -p 172.16.0.220:3260 --op delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:49:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:008f5fa6-2644-4c22-b8cd-28e233cf8857 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:49:56 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[0c001653-9a1d-4630-b93f-18aacf604ad2]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:49:56 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f4d7a74-b2e8-485f-a716-520b298dcb4b tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:49:56 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-4f4d7a74-b2e8-485f-a716-520b298dcb4b tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.244s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:49:56 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f4d7a74-b2e8-485f-a716-520b298dcb4b tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] <== disconnect_volume: return (245ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:49:56 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-4f4d7a74-b2e8-485f-a716-520b298dcb4b tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 02:49:57 user nova-compute[72351]: DEBUG nova.objects.instance [None req-4f4d7a74-b2e8-485f-a716-520b298dcb4b tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lazy-loading 'flavor' on Instance uuid c6ae885f-dce2-4dc2-98d5-26139b8fb67e {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:49:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-4f4d7a74-b2e8-485f-a716-520b298dcb4b tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "c6ae885f-dce2-4dc2-98d5-26139b8fb67e" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.560s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:49:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-9e32cf14-e94b-452e-825a-fb5bf6696c95 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Acquiring lock "c6ae885f-dce2-4dc2-98d5-26139b8fb67e" by "nova.compute.manager.ComputeManager.stop_instance.<locals>.do_stop_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:49:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-9e32cf14-e94b-452e-825a-fb5bf6696c95 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "c6ae885f-dce2-4dc2-98d5-26139b8fb67e" acquired by "nova.compute.manager.ComputeManager.stop_instance.<locals>.do_stop_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:49:57 user nova-compute[72351]: DEBUG nova.compute.manager [None req-9e32cf14-e94b-452e-825a-fb5bf6696c95 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:49:57 user nova-compute[72351]: DEBUG nova.compute.manager [None req-9e32cf14-e94b-452e-825a-fb5bf6696c95 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Stopping instance; current vm_state: active, current task_state: powering-off, current DB power_state: 1, current VM power_state: 1 {{(pid=72351) do_stop_instance /opt/stack/nova/nova/compute/manager.py:3342}}
Aug 09 02:49:57 user nova-compute[72351]: DEBUG nova.objects.instance [None req-9e32cf14-e94b-452e-825a-fb5bf6696c95 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lazy-loading 'flavor' on Instance uuid c6ae885f-dce2-4dc2-98d5-26139b8fb67e {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:49:57 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:49:57 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:49:57 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:49:57 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:49:58 user nova-compute[72351]: DEBUG nova.compute.manager [req-b70a0d15-63f8-41cb-a95b-f5a7073b88cf req-3f985ad0-58ca-46ea-9ca9-92a1b706844d service nova] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Received event network-vif-unplugged-2771d93d-fa0a-4e19-85ac-9d3f5df52782 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:49:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-b70a0d15-63f8-41cb-a95b-f5a7073b88cf req-3f985ad0-58ca-46ea-9ca9-92a1b706844d service nova] Acquiring lock "c6ae885f-dce2-4dc2-98d5-26139b8fb67e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:49:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-b70a0d15-63f8-41cb-a95b-f5a7073b88cf req-3f985ad0-58ca-46ea-9ca9-92a1b706844d service nova] Lock "c6ae885f-dce2-4dc2-98d5-26139b8fb67e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:49:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-b70a0d15-63f8-41cb-a95b-f5a7073b88cf req-3f985ad0-58ca-46ea-9ca9-92a1b706844d service nova] Lock "c6ae885f-dce2-4dc2-98d5-26139b8fb67e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:49:58 user nova-compute[72351]: DEBUG nova.compute.manager [req-b70a0d15-63f8-41cb-a95b-f5a7073b88cf req-3f985ad0-58ca-46ea-9ca9-92a1b706844d service nova] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] No waiting events found dispatching network-vif-unplugged-2771d93d-fa0a-4e19-85ac-9d3f5df52782 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:49:58 user nova-compute[72351]: WARNING nova.compute.manager [req-b70a0d15-63f8-41cb-a95b-f5a7073b88cf req-3f985ad0-58ca-46ea-9ca9-92a1b706844d service nova] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Received unexpected event network-vif-unplugged-2771d93d-fa0a-4e19-85ac-9d3f5df52782 for instance with vm_state active and task_state powering-off.
Aug 09 02:49:58 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Instance destroyed successfully.
Aug 09 02:49:58 user nova-compute[72351]: DEBUG nova.compute.manager [None req-9e32cf14-e94b-452e-825a-fb5bf6696c95 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:49:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-9e32cf14-e94b-452e-825a-fb5bf6696c95 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "c6ae885f-dce2-4dc2-98d5-26139b8fb67e" "released" by "nova.compute.manager.ComputeManager.stop_instance.<locals>.do_stop_instance" :: held 0.831s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:49:59 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:49:59 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:50:00 user nova-compute[72351]: DEBUG nova.compute.manager [req-fd4343cd-686a-4f59-a0f2-e06b7db283b1 req-06244d97-7200-4a75-9837-f29c9625b37d service nova] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Received event network-vif-plugged-2771d93d-fa0a-4e19-85ac-9d3f5df52782 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:50:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-fd4343cd-686a-4f59-a0f2-e06b7db283b1 req-06244d97-7200-4a75-9837-f29c9625b37d service nova] Acquiring lock "c6ae885f-dce2-4dc2-98d5-26139b8fb67e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:50:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-fd4343cd-686a-4f59-a0f2-e06b7db283b1 req-06244d97-7200-4a75-9837-f29c9625b37d service nova] Lock "c6ae885f-dce2-4dc2-98d5-26139b8fb67e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:50:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-fd4343cd-686a-4f59-a0f2-e06b7db283b1 req-06244d97-7200-4a75-9837-f29c9625b37d service nova] Lock "c6ae885f-dce2-4dc2-98d5-26139b8fb67e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:50:00 user nova-compute[72351]: DEBUG nova.compute.manager [req-fd4343cd-686a-4f59-a0f2-e06b7db283b1 req-06244d97-7200-4a75-9837-f29c9625b37d service nova] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] No waiting events found dispatching network-vif-plugged-2771d93d-fa0a-4e19-85ac-9d3f5df52782 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:50:00 user nova-compute[72351]: WARNING nova.compute.manager [req-fd4343cd-686a-4f59-a0f2-e06b7db283b1 req-06244d97-7200-4a75-9837-f29c9625b37d service nova] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Received unexpected event network-vif-plugged-2771d93d-fa0a-4e19-85ac-9d3f5df52782 for instance with vm_state stopped and task_state None.
Aug 09 02:50:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:50:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:50:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:50:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 02:50:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:50:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:50:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:50:13 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691549398.6023252, c6ae885f-dce2-4dc2-98d5-26139b8fb67e => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:50:13 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] VM Stopped (Lifecycle Event)
Aug 09 02:50:13 user nova-compute[72351]: DEBUG nova.compute.manager [None req-e8937d9c-9508-4598-82aa-7e0f2aced3c0 None None] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:50:13 user nova-compute[72351]: DEBUG nova.compute.manager [None req-e8937d9c-9508-4598-82aa-7e0f2aced3c0 None None] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: stopped, current task_state: None, current DB power_state: 4, VM power_state: 4 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:50:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:50:17 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:50:17 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:50:17 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72351) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10461}}
Aug 09 02:50:19 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:50:19 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:50:21 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:50:21 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Starting heal instance info cache {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9842}}
Aug 09 02:50:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "refresh_cache-0d4c65d2-c641-4985-8115-24dcbc15555f" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:50:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquired lock "refresh_cache-0d4c65d2-c641-4985-8115-24dcbc15555f" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:50:22 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Forcefully refreshing network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1996}}
Aug 09 02:50:22 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Updating instance_info_cache with network_info: [{"id": "c0c9db92-f7f2-4a6d-90d0-bfe48762cc6e", "address": "fa:16:3e:93:37:be", "network": {"id": "47c43230-a244-4198-84e4-9ed98c005cd9", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1392403106-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c07a6b254414b41a673468238d3aa62", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc0c9db92-f7", "ovs_interfaceid": "c0c9db92-f7f2-4a6d-90d0-bfe48762cc6e", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:50:22 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Releasing lock "refresh_cache-0d4c65d2-c641-4985-8115-24dcbc15555f" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:50:22 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Updated the network info_cache for instance {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9913}}
Aug 09 02:50:23 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:50:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:50:24 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:50:25 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:50:25 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:50:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:50:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:50:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:50:25 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Auditing locally available compute resources for user (node: user) {{(pid=72351) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Aug 09 02:50:26 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/0d4c65d2-c641-4985-8115-24dcbc15555f/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:50:26 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/0d4c65d2-c641-4985-8115-24dcbc15555f/disk --force-share --output=json" returned: 0 in 0.146s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:50:26 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/0d4c65d2-c641-4985-8115-24dcbc15555f/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:50:26 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/0d4c65d2-c641-4985-8115-24dcbc15555f/disk --force-share --output=json" returned: 0 in 0.132s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:50:26 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:50:26 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk --force-share --output=json" returned: 0 in 0.143s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:50:26 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:50:26 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:50:26 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:50:26 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:50:26 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Hypervisor/Node resource view: name=user free_ram=8678MB free_disk=25.196613311767578GB free_vcpus=11 pci_devices=[{"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72351) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Aug 09 02:50:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:50:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:50:27 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance c6ae885f-dce2-4dc2-98d5-26139b8fb67e actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:50:27 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 0d4c65d2-c641-4985-8115-24dcbc15555f actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:50:27 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Total usable vcpus: 12, total allocated vcpus: 2 {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Aug 09 02:50:27 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Final resource view: name=user phys_ram=16011MB used_ram=1536MB phys_disk=40GB used_disk=2GB total_vcpus=12 used_vcpus=2 pci_stats=[] {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Aug 09 02:50:27 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:50:27 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:50:27 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Compute_service record updated for user:user {{(pid=72351) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Aug 09 02:50:27 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.218s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:50:29 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:50:31 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:50:34 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:50:38 user nova-compute[72351]: DEBUG nova.compute.manager [req-27bd5427-0730-4de6-8ebe-e94df1a2011c req-a7101840-1ad8-4d92-b265-805e34dcd2d4 service nova] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Received event network-changed-c0c9db92-f7f2-4a6d-90d0-bfe48762cc6e {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:50:38 user nova-compute[72351]: DEBUG nova.compute.manager [req-27bd5427-0730-4de6-8ebe-e94df1a2011c req-a7101840-1ad8-4d92-b265-805e34dcd2d4 service nova] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Refreshing instance network info cache due to event network-changed-c0c9db92-f7f2-4a6d-90d0-bfe48762cc6e. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:50:38 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-27bd5427-0730-4de6-8ebe-e94df1a2011c req-a7101840-1ad8-4d92-b265-805e34dcd2d4 service nova] Acquiring lock "refresh_cache-0d4c65d2-c641-4985-8115-24dcbc15555f" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:50:38 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-27bd5427-0730-4de6-8ebe-e94df1a2011c req-a7101840-1ad8-4d92-b265-805e34dcd2d4 service nova] Acquired lock "refresh_cache-0d4c65d2-c641-4985-8115-24dcbc15555f" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:50:38 user nova-compute[72351]: DEBUG nova.network.neutron [req-27bd5427-0730-4de6-8ebe-e94df1a2011c req-a7101840-1ad8-4d92-b265-805e34dcd2d4 service nova] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Refreshing network info cache for port c0c9db92-f7f2-4a6d-90d0-bfe48762cc6e {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:50:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:50:39 user nova-compute[72351]: DEBUG nova.network.neutron [req-27bd5427-0730-4de6-8ebe-e94df1a2011c req-a7101840-1ad8-4d92-b265-805e34dcd2d4 service nova] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Updated VIF entry in instance network info cache for port c0c9db92-f7f2-4a6d-90d0-bfe48762cc6e. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:50:39 user nova-compute[72351]: DEBUG nova.network.neutron [req-27bd5427-0730-4de6-8ebe-e94df1a2011c req-a7101840-1ad8-4d92-b265-805e34dcd2d4 service nova] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Updating instance_info_cache with network_info: [{"id": "c0c9db92-f7f2-4a6d-90d0-bfe48762cc6e", "address": "fa:16:3e:93:37:be", "network": {"id": "47c43230-a244-4198-84e4-9ed98c005cd9", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1392403106-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.134", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c07a6b254414b41a673468238d3aa62", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc0c9db92-f7", "ovs_interfaceid": "c0c9db92-f7f2-4a6d-90d0-bfe48762cc6e", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:50:39 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-27bd5427-0730-4de6-8ebe-e94df1a2011c req-a7101840-1ad8-4d92-b265-805e34dcd2d4 service nova] Releasing lock "refresh_cache-0d4c65d2-c641-4985-8115-24dcbc15555f" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:50:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:50:42 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-281920e8-c7ad-4879-be6d-91d22c2ff5a4 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Acquiring lock "0d4c65d2-c641-4985-8115-24dcbc15555f" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:50:42 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-281920e8-c7ad-4879-be6d-91d22c2ff5a4 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "0d4c65d2-c641-4985-8115-24dcbc15555f" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:50:42 user nova-compute[72351]: DEBUG nova.objects.instance [None req-281920e8-c7ad-4879-be6d-91d22c2ff5a4 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lazy-loading 'flavor' on Instance uuid 0d4c65d2-c641-4985-8115-24dcbc15555f {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:50:42 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-281920e8-c7ad-4879-be6d-91d22c2ff5a4 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "0d4c65d2-c641-4985-8115-24dcbc15555f" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.059s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:50:42 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-281920e8-c7ad-4879-be6d-91d22c2ff5a4 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Acquiring lock "0d4c65d2-c641-4985-8115-24dcbc15555f" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:50:42 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-281920e8-c7ad-4879-be6d-91d22c2ff5a4 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "0d4c65d2-c641-4985-8115-24dcbc15555f" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:50:42 user nova-compute[72351]: INFO nova.compute.manager [None req-281920e8-c7ad-4879-be6d-91d22c2ff5a4 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Attaching volume 3331d510-205f-4cbf-9469-f59034253198 to /dev/vdb
Aug 09 02:50:42 user nova-compute[72351]: DEBUG os_brick.utils [None req-281920e8-c7ad-4879-be6d-91d22c2ff5a4 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:50:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:50:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:50:42 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[b896c31c-cd52-4d6b-a20f-056487aced09]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:a52269602ff4\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:50:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:50:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:50:42 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[959c8fb0-e4eb-438f-a746-1fc8d1d8ef2a]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:50:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:50:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.015s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:50:42 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[4122edee-ee3e-4039-80be-8c228decf692]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:50:42 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[bc16f420-95f3-422f-85c2-bcb16af13cae]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:50:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-281920e8-c7ad-4879-be6d-91d22c2ff5a4 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Running cmd (subprocess): nvme version {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:50:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-281920e8-c7ad-4879-be6d-91d22c2ff5a4 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] 'nvme version' failed. Not Retrying. {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:50:42 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.nvmeof [None req-281920e8-c7ad-4879-be6d-91d22c2ff5a4 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] nvme not present on system {{(pid=72351) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Aug 09 02:50:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:50:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:50:42 user nova-compute[72351]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Aug 09 02:50:42 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[3304415f-95ab-4f43-8078-33f89511fbed]: (4, '') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:50:42 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-281920e8-c7ad-4879-be6d-91d22c2ff5a4 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72351) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Aug 09 02:50:42 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-281920e8-c7ad-4879-be6d-91d22c2ff5a4 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Aug 09 02:50:42 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-281920e8-c7ad-4879-be6d-91d22c2ff5a4 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] LIGHTOS: no hostnqn found. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Aug 09 02:50:42 user nova-compute[72351]: DEBUG os_brick.utils [None req-281920e8-c7ad-4879-be6d-91d22c2ff5a4 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] <== get_connector_properties: return (134ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:a52269602ff4', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:50:42 user nova-compute[72351]: DEBUG nova.virt.block_device [None req-281920e8-c7ad-4879-be6d-91d22c2ff5a4 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Updating existing volume attachment record: 17e42cf3-8117-4475-80f6-798b60dad439 {{(pid=72351) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Aug 09 02:50:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:50:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:50:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:50:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:50:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:50:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:50:56 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-281920e8-c7ad-4879-be6d-91d22c2ff5a4 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 02:50:56 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-281920e8-c7ad-4879-be6d-91d22c2ff5a4 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:3331d510-205f-4cbf-9469-f59034253198'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'FgcSJAKs', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:50:56 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-281920e8-c7ad-4879-be6d-91d22c2ff5a4 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:50:56 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-281920e8-c7ad-4879-be6d-91d22c2ff5a4 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:50:56 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-281920e8-c7ad-4879-be6d-91d22c2ff5a4 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:50:56 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-281920e8-c7ad-4879-be6d-91d22c2ff5a4 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:50:56 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-281920e8-c7ad-4879-be6d-91d22c2ff5a4 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:3331d510-205f-4cbf-9469-f59034253198" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:50:56 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-281920e8-c7ad-4879-be6d-91d22c2ff5a4 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:3331d510-205f-4cbf-9469-f59034253198" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:50:56 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-281920e8-c7ad-4879-be6d-91d22c2ff5a4 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 02:50:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3331d510-205f-4cbf-9469-f59034253198 -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:50:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3331d510-205f-4cbf-9469-f59034253198 -p 172.16.0.220:3260" returned: 21 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:50:56 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[2399a328-bd0c-4538-98d1-757a80f3fc8a]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:50:56 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-281920e8-c7ad-4879-be6d-91d22c2ff5a4 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Aug 09 02:50:56 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:50:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3331d510-205f-4cbf-9469-f59034253198 -p 172.16.0.220:3260 --interface default --op new {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:50:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3331d510-205f-4cbf-9469-f59034253198 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:50:56 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a89ec28b-462a-4ab1-b8ea-6f41edaaa340]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:3331d510-205f-4cbf-9469-f59034253198] added\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:50:56 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-281920e8-c7ad-4879-be6d-91d22c2ff5a4 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:3331d510-205f-4cbf-9469-f59034253198] added
Aug 09 02:50:56 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:50:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3331d510-205f-4cbf-9469-f59034253198 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:50:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3331d510-205f-4cbf-9469-f59034253198 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:50:56 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[497d8682-4916-4437-a28e-d98d05efeb2d]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:50:56 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-281920e8-c7ad-4879-be6d-91d22c2ff5a4 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:50:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3331d510-205f-4cbf-9469-f59034253198 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:50:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3331d510-205f-4cbf-9469-f59034253198 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:50:56 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[48eb5701-b390-40eb-a119-3a42a2dd501d]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:50:56 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-281920e8-c7ad-4879-be6d-91d22c2ff5a4 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:50:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3331d510-205f-4cbf-9469-f59034253198 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v FgcSJAKs {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:50:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3331d510-205f-4cbf-9469-f59034253198 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v FgcSJAKs" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:50:56 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[57f45ce8-35a2-4e14-ab5f-a95e64639b45]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:50:56 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-281920e8-c7ad-4879-be6d-91d22c2ff5a4 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'FgcSJAKs'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:50:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3331d510-205f-4cbf-9469-f59034253198 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:50:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3331d510-205f-4cbf-9469-f59034253198 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:50:56 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[9e983798-0f77-4342-b84b-1882d8b78e8c]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:50:56 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-281920e8-c7ad-4879-be6d-91d22c2ff5a4 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:50:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:50:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 21 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:50:56 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[bc7002ef-10b0-4462-bada-c7038be24a5b]: (4, ('', 'iscsiadm: No active sessions.\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:50:56 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-281920e8-c7ad-4879-be6d-91d22c2ff5a4 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] iscsiadm ('-m', 'session'): stdout= stderr=iscsiadm: No active sessions.
Aug 09 02:50:56 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:50:56 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-281920e8-c7ad-4879-be6d-91d22c2ff5a4 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] iscsi session list stdout= stderr=iscsiadm: No active sessions.
Aug 09 02:50:56 user nova-compute[72351]:  {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:50:56 user nova-compute[72351]: WARNING os_brick.initiator.connectors.iscsi [None req-281920e8-c7ad-4879-be6d-91d22c2ff5a4 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] iscsiadm stderr output when getting sessions: iscsiadm: No active sessions.
Aug 09 02:50:56 user nova-compute[72351]: 
Aug 09 02:50:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3331d510-205f-4cbf-9469-f59034253198 -p 172.16.0.220:3260 --login {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:50:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3331d510-205f-4cbf-9469-f59034253198 -p 172.16.0.220:3260 --login" returned: 0 in 0.044s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:50:56 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ec2c9e6f-29b8-41dd-96b0-6f6a97d3a692]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:3331d510-205f-4cbf-9469-f59034253198, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:3331d510-205f-4cbf-9469-f59034253198, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:50:56 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-281920e8-c7ad-4879-be6d-91d22c2ff5a4 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:3331d510-205f-4cbf-9469-f59034253198, portal: 172.16.0.220,3260]
Aug 09 02:50:56 user nova-compute[72351]: Login to [iface: default, target: iqn.2010-10.org.openstack:3331d510-205f-4cbf-9469-f59034253198, portal: 172.16.0.220,3260] successful.
Aug 09 02:50:56 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:50:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3331d510-205f-4cbf-9469-f59034253198 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:50:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3331d510-205f-4cbf-9469-f59034253198 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.029s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:50:56 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[8f1c39ef-9c7f-446a-a470-f50150d29ae3]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:50:56 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-281920e8-c7ad-4879-be6d-91d22c2ff5a4 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:50:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:50:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.026s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:50:56 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[78c7ad9b-7293-4ba5-be55-95d26ddba6f6]: (4, ('tcp: [61] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3331d510-205f-4cbf-9469-f59034253198 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:50:56 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-281920e8-c7ad-4879-be6d-91d22c2ff5a4 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [61] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3331d510-205f-4cbf-9469-f59034253198 (non-flash)
Aug 09 02:50:56 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:50:56 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-281920e8-c7ad-4879-be6d-91d22c2ff5a4 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] iscsi session list stdout=tcp: [61] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3331d510-205f-4cbf-9469-f59034253198 (non-flash)
Aug 09 02:50:56 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:50:56 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-281920e8-c7ad-4879-be6d-91d22c2ff5a4 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:3331d510-205f-4cbf-9469-f59034253198" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.268s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:50:56 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-281920e8-c7ad-4879-be6d-91d22c2ff5a4 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 02:50:56 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-281920e8-c7ad-4879-be6d-91d22c2ff5a4 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] HCTL ('33', '-', '-', 0) found on session 61 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 02:50:56 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-281920e8-c7ad-4879-be6d-91d22c2ff5a4 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 02:50:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:50:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.016s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:50:56 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[9f24ea41-efee-4c8a-8403-d6d7105af822]: (4, ('- - 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:50:56 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-281920e8-c7ad-4879-be6d-91d22c2ff5a4 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Searching for a device in session 61 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:50:56 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-281920e8-c7ad-4879-be6d-91d22c2ff5a4 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Connected to sdb using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'FgcSJAKs', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 02:50:57 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-281920e8-c7ad-4879-be6d-91d22c2ff5a4 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.296s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:50:57 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-281920e8-c7ad-4879-be6d-91d22c2ff5a4 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] <== connect_volume: return (1298ms) {'type': 'block', 'scsi_wwn': '23538396236646233', 'path': '/dev/sdb'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:50:57 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-281920e8-c7ad-4879-be6d-91d22c2ff5a4 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23538396236646233', 'path': '/dev/sdb'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 02:50:57 user nova-compute[72351]: DEBUG nova.objects.instance [None req-281920e8-c7ad-4879-be6d-91d22c2ff5a4 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lazy-loading 'flavor' on Instance uuid 0d4c65d2-c641-4985-8115-24dcbc15555f {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:50:57 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-281920e8-c7ad-4879-be6d-91d22c2ff5a4 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] attach device xml: <disk type="block" device="disk">
Aug 09 02:50:57 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:50:57 user nova-compute[72351]:   <source dev="/dev/sdb"/>
Aug 09 02:50:57 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:50:57 user nova-compute[72351]:   <serial>3331d510-205f-4cbf-9469-f59034253198</serial>
Aug 09 02:50:57 user nova-compute[72351]: </disk>
Aug 09 02:50:57 user nova-compute[72351]:  {{(pid=72351) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Aug 09 02:50:57 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-281920e8-c7ad-4879-be6d-91d22c2ff5a4 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:50:57 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-281920e8-c7ad-4879-be6d-91d22c2ff5a4 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:50:57 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-281920e8-c7ad-4879-be6d-91d22c2ff5a4 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] No VIF found with MAC fa:16:3e:93:37:be, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:50:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-281920e8-c7ad-4879-be6d-91d22c2ff5a4 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "0d4c65d2-c641-4985-8115-24dcbc15555f" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 15.381s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:50:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-789f739f-6417-43ce-a81f-2ab3a3375ad8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Acquiring lock "0d4c65d2-c641-4985-8115-24dcbc15555f" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:50:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-789f739f-6417-43ce-a81f-2ab3a3375ad8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "0d4c65d2-c641-4985-8115-24dcbc15555f" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:50:58 user nova-compute[72351]: INFO nova.compute.manager [None req-789f739f-6417-43ce-a81f-2ab3a3375ad8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Detaching volume 3331d510-205f-4cbf-9469-f59034253198
Aug 09 02:50:58 user nova-compute[72351]: INFO nova.virt.block_device [None req-789f739f-6417-43ce-a81f-2ab3a3375ad8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Attempting to driver detach volume 3331d510-205f-4cbf-9469-f59034253198 from mountpoint /dev/vdb
Aug 09 02:50:58 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-789f739f-6417-43ce-a81f-2ab3a3375ad8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Attempting to detach device vdb from instance 0d4c65d2-c641-4985-8115-24dcbc15555f from the persistent domain config. {{(pid=72351) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2477}}
Aug 09 02:50:58 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-789f739f-6417-43ce-a81f-2ab3a3375ad8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 02:50:58 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:50:58 user nova-compute[72351]:   <source dev="/dev/sdb"/>
Aug 09 02:50:58 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:50:58 user nova-compute[72351]:   <serial>3331d510-205f-4cbf-9469-f59034253198</serial>
Aug 09 02:50:58 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Aug 09 02:50:58 user nova-compute[72351]: </disk>
Aug 09 02:50:58 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 02:50:58 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-789f739f-6417-43ce-a81f-2ab3a3375ad8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Successfully detached device vdb from instance 0d4c65d2-c641-4985-8115-24dcbc15555f from the persistent domain config.
Aug 09 02:50:58 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-789f739f-6417-43ce-a81f-2ab3a3375ad8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 0d4c65d2-c641-4985-8115-24dcbc15555f from the live domain config. {{(pid=72351) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2513}}
Aug 09 02:50:58 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-789f739f-6417-43ce-a81f-2ab3a3375ad8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 02:50:58 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:50:58 user nova-compute[72351]:   <source dev="/dev/sdb"/>
Aug 09 02:50:58 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:50:58 user nova-compute[72351]:   <serial>3331d510-205f-4cbf-9469-f59034253198</serial>
Aug 09 02:50:58 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Aug 09 02:50:58 user nova-compute[72351]: </disk>
Aug 09 02:50:58 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 02:50:59 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Received event <DeviceRemovedEvent: 1691549459.3225794, 0d4c65d2-c641-4985-8115-24dcbc15555f => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=72351) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2360}}
Aug 09 02:50:59 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-789f739f-6417-43ce-a81f-2ab3a3375ad8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 0d4c65d2-c641-4985-8115-24dcbc15555f {{(pid=72351) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2589}}
Aug 09 02:50:59 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-789f739f-6417-43ce-a81f-2ab3a3375ad8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Successfully detached device vdb from instance 0d4c65d2-c641-4985-8115-24dcbc15555f from the live domain config.
Aug 09 02:50:59 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-789f739f-6417-43ce-a81f-2ab3a3375ad8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 02:50:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-789f739f-6417-43ce-a81f-2ab3a3375ad8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:3331d510-205f-4cbf-9469-f59034253198'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'FgcSJAKs', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb'}, None), 'kwargs': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:50:59 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-789f739f-6417-43ce-a81f-2ab3a3375ad8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:50:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-789f739f-6417-43ce-a81f-2ab3a3375ad8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:50:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-789f739f-6417-43ce-a81f-2ab3a3375ad8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:50:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-789f739f-6417-43ce-a81f-2ab3a3375ad8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:3331d510-205f-4cbf-9469-f59034253198', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 02:50:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:50:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:50:59 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[09bc44ae-d239-4a1e-8b8a-09b8fbb5f6ba]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:3331d510-205f-4cbf-9469-f59034253198\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:50:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:50:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:50:59 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[3d94193c-d76e-44b5-9682-7b59e3f753a3]: (4, ('tcp: [61] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3331d510-205f-4cbf-9469-f59034253198 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:50:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-789f739f-6417-43ce-a81f-2ab3a3375ad8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [61] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3331d510-205f-4cbf-9469-f59034253198 (non-flash)
Aug 09 02:50:59 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:50:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-789f739f-6417-43ce-a81f-2ab3a3375ad8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] iscsi session list stdout=tcp: [61] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3331d510-205f-4cbf-9469-f59034253198 (non-flash)
Aug 09 02:50:59 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:50:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-789f739f-6417-43ce-a81f-2ab3a3375ad8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fcfdc1bfd00>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:3331d510-205f-4cbf-9469-f59034253198'): ({'sdb'}, set())}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 02:50:59 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-789f739f-6417-43ce-a81f-2ab3a3375ad8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Removing single pathed devices sdb {{(pid=72351) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Aug 09 02:50:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:50:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:50:59 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e73e06a3-ccca-49b5-8d64-361b8c395d75]: (4, ('path checker states:\nup                  1\n\npaths: 0\nbusy: False\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:50:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:50:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:50:59 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[4a03db44-2a04-4302-b6ce-791093060d3e]: (4, ('ok\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:50:59 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-789f739f-6417-43ce-a81f-2ab3a3375ad8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Flushing IO for device /dev/sdb {{(pid=72351) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Aug 09 02:50:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:50:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:50:59 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[8a526ef2-1d78-4656-b3cd-7c324658d276]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:50:59 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-789f739f-6417-43ce-a81f-2ab3a3375ad8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=72351) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Aug 09 02:50:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:50:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.043s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:50:59 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[f9562df2-740b-4202-bec2-1bb2ee3bfa9a]: (4, ('1', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:50:59 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-789f739f-6417-43ce-a81f-2ab3a3375ad8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Aug 09 02:50:59 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-789f739f-6417-43ce-a81f-2ab3a3375ad8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] SCSI volumes sdb have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Aug 09 02:50:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-789f739f-6417-43ce-a81f-2ab3a3375ad8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:3331d510-205f-4cbf-9469-f59034253198')] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 02:50:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3331d510-205f-4cbf-9469-f59034253198 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:50:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3331d510-205f-4cbf-9469-f59034253198 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:50:59 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[0f90db07-da4a-447c-aa89-8bfd4cb55b4b]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:50:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-789f739f-6417-43ce-a81f-2ab3a3375ad8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:50:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3331d510-205f-4cbf-9469-f59034253198 -p 172.16.0.220:3260 --logout {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:50:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3331d510-205f-4cbf-9469-f59034253198 -p 172.16.0.220:3260 --logout" returned: 0 in 0.052s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:50:59 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[00a070b2-0b57-46b2-bdb7-0231b7064a06]: (4, ('Logging out of session [sid: 61, target: iqn.2010-10.org.openstack:3331d510-205f-4cbf-9469-f59034253198, portal: 172.16.0.220,3260]\nLogout of [sid: 61, target: iqn.2010-10.org.openstack:3331d510-205f-4cbf-9469-f59034253198, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:50:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-789f739f-6417-43ce-a81f-2ab3a3375ad8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 61, target: iqn.2010-10.org.openstack:3331d510-205f-4cbf-9469-f59034253198, portal: 172.16.0.220,3260]
Aug 09 02:50:59 user nova-compute[72351]: Logout of [sid: 61, target: iqn.2010-10.org.openstack:3331d510-205f-4cbf-9469-f59034253198, portal: 172.16.0.220,3260] successful.
Aug 09 02:50:59 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:50:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3331d510-205f-4cbf-9469-f59034253198 -p 172.16.0.220:3260 --op delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:50:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3331d510-205f-4cbf-9469-f59034253198 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:50:59 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[c618b1f4-ce9d-4154-9bf5-92e7e5770ba7]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:50:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-789f739f-6417-43ce-a81f-2ab3a3375ad8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:50:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-789f739f-6417-43ce-a81f-2ab3a3375ad8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.271s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:50:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-789f739f-6417-43ce-a81f-2ab3a3375ad8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] <== disconnect_volume: return (272ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:50:59 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-789f739f-6417-43ce-a81f-2ab3a3375ad8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 02:50:59 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:51:00 user nova-compute[72351]: DEBUG nova.objects.instance [None req-789f739f-6417-43ce-a81f-2ab3a3375ad8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lazy-loading 'flavor' on Instance uuid 0d4c65d2-c641-4985-8115-24dcbc15555f {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:51:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-789f739f-6417-43ce-a81f-2ab3a3375ad8 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "0d4c65d2-c641-4985-8115-24dcbc15555f" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.554s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:51:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Acquiring lock "150f1b69-aa23-4702-a9f4-24025963befe" by "nova.compute.manager.ComputeManager.unshelve_instance.<locals>.do_unshelve_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:51:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "150f1b69-aa23-4702-a9f4-24025963befe" acquired by "nova.compute.manager.ComputeManager.unshelve_instance.<locals>.do_unshelve_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:51:02 user nova-compute[72351]: INFO nova.compute.manager [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Unshelving
Aug 09 02:51:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:51:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:51:02 user nova-compute[72351]: DEBUG nova.objects.instance [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lazy-loading 'pci_requests' on Instance uuid 150f1b69-aa23-4702-a9f4-24025963befe {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:51:02 user nova-compute[72351]: DEBUG nova.objects.instance [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lazy-loading 'numa_topology' on Instance uuid 150f1b69-aa23-4702-a9f4-24025963befe {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:51:02 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72351) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Aug 09 02:51:02 user nova-compute[72351]: INFO nova.compute.claims [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Claim successful on node user
Aug 09 02:51:03 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:51:03 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:51:03 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.251s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:51:03 user nova-compute[72351]: INFO nova.network.neutron [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Updating port 0983efc8-e53e-40bd-a721-2924155c20f3 with attributes {'binding:host_id': 'user', 'device_owner': 'compute:nova'}
Aug 09 02:51:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:51:03 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Acquiring lock "refresh_cache-150f1b69-aa23-4702-a9f4-24025963befe" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:51:03 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Acquired lock "refresh_cache-150f1b69-aa23-4702-a9f4-24025963befe" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:51:03 user nova-compute[72351]: DEBUG nova.network.neutron [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:51:03 user nova-compute[72351]: DEBUG nova.compute.manager [req-aec06a49-2b0a-4aac-a2dd-1b5dbc14f2da req-1c1e3d5c-730f-4b34-8a6d-d54c10313e44 service nova] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Received event network-changed-0983efc8-e53e-40bd-a721-2924155c20f3 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:51:03 user nova-compute[72351]: DEBUG nova.compute.manager [req-aec06a49-2b0a-4aac-a2dd-1b5dbc14f2da req-1c1e3d5c-730f-4b34-8a6d-d54c10313e44 service nova] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Refreshing instance network info cache due to event network-changed-0983efc8-e53e-40bd-a721-2924155c20f3. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:51:03 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-aec06a49-2b0a-4aac-a2dd-1b5dbc14f2da req-1c1e3d5c-730f-4b34-8a6d-d54c10313e44 service nova] Acquiring lock "refresh_cache-150f1b69-aa23-4702-a9f4-24025963befe" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:51:04 user nova-compute[72351]: DEBUG nova.network.neutron [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Updating instance_info_cache with network_info: [{"id": "0983efc8-e53e-40bd-a721-2924155c20f3", "address": "fa:16:3e:d7:24:ef", "network": {"id": "da9be96c-ce20-4037-999c-230ca799fb06", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-661780305-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.240", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5ff300d27c9a4dd3b7ecd507b3627d98", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0983efc8-e5", "ovs_interfaceid": "0983efc8-e53e-40bd-a721-2924155c20f3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:51:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Releasing lock "refresh_cache-150f1b69-aa23-4702-a9f4-24025963befe" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:51:04 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Creating instance directory {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4717}}
Aug 09 02:51:04 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Creating image(s)
Aug 09 02:51:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Acquiring lock "/opt/stack/data/nova/instances/150f1b69-aa23-4702-a9f4-24025963befe/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:51:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "/opt/stack/data/nova/instances/150f1b69-aa23-4702-a9f4-24025963befe/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:51:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "/opt/stack/data/nova/instances/150f1b69-aa23-4702-a9f4-24025963befe/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:51:04 user nova-compute[72351]: DEBUG nova.objects.instance [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lazy-loading 'trusted_certs' on Instance uuid 150f1b69-aa23-4702-a9f4-24025963befe {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:51:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-aec06a49-2b0a-4aac-a2dd-1b5dbc14f2da req-1c1e3d5c-730f-4b34-8a6d-d54c10313e44 service nova] Acquired lock "refresh_cache-150f1b69-aa23-4702-a9f4-24025963befe" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:51:04 user nova-compute[72351]: DEBUG nova.network.neutron [req-aec06a49-2b0a-4aac-a2dd-1b5dbc14f2da req-1c1e3d5c-730f-4b34-8a6d-d54c10313e44 service nova] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Refreshing network info cache for port 0983efc8-e53e-40bd-a721-2924155c20f3 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:51:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Acquiring lock "aac9bf03ba600a05bb056708c9995070051b324c" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:51:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "aac9bf03ba600a05bb056708c9995070051b324c" acquired by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:51:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/aac9bf03ba600a05bb056708c9995070051b324c.part --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:51:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:51:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/aac9bf03ba600a05bb056708c9995070051b324c.part --force-share --output=json" returned: 0 in 0.132s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:51:05 user nova-compute[72351]: DEBUG nova.virt.images [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] 84e39abb-60a1-45ad-b0d9-5bf219191a3f was qcow2, converting to raw {{(pid=72351) fetch_to_raw /opt/stack/nova/nova/virt/images.py:165}}
Aug 09 02:51:05 user nova-compute[72351]: DEBUG nova.privsep.utils [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=72351) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Aug 09 02:51:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Running cmd (subprocess): qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/aac9bf03ba600a05bb056708c9995070051b324c.part /opt/stack/data/nova/instances/_base/aac9bf03ba600a05bb056708c9995070051b324c.converted {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:51:05 user nova-compute[72351]: DEBUG nova.network.neutron [req-aec06a49-2b0a-4aac-a2dd-1b5dbc14f2da req-1c1e3d5c-730f-4b34-8a6d-d54c10313e44 service nova] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Updated VIF entry in instance network info cache for port 0983efc8-e53e-40bd-a721-2924155c20f3. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:51:05 user nova-compute[72351]: DEBUG nova.network.neutron [req-aec06a49-2b0a-4aac-a2dd-1b5dbc14f2da req-1c1e3d5c-730f-4b34-8a6d-d54c10313e44 service nova] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Updating instance_info_cache with network_info: [{"id": "0983efc8-e53e-40bd-a721-2924155c20f3", "address": "fa:16:3e:d7:24:ef", "network": {"id": "da9be96c-ce20-4037-999c-230ca799fb06", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-661780305-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.240", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5ff300d27c9a4dd3b7ecd507b3627d98", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0983efc8-e5", "ovs_interfaceid": "0983efc8-e53e-40bd-a721-2924155c20f3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:51:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-aec06a49-2b0a-4aac-a2dd-1b5dbc14f2da req-1c1e3d5c-730f-4b34-8a6d-d54c10313e44 service nova] Releasing lock "refresh_cache-150f1b69-aa23-4702-a9f4-24025963befe" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:51:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] CMD "qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/aac9bf03ba600a05bb056708c9995070051b324c.part /opt/stack/data/nova/instances/_base/aac9bf03ba600a05bb056708c9995070051b324c.converted" returned: 0 in 0.306s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:51:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/aac9bf03ba600a05bb056708c9995070051b324c.converted --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:51:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/aac9bf03ba600a05bb056708c9995070051b324c.converted --force-share --output=json" returned: 0 in 0.133s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:51:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "aac9bf03ba600a05bb056708c9995070051b324c" "released" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: held 1.419s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:51:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/aac9bf03ba600a05bb056708c9995070051b324c --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:51:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/aac9bf03ba600a05bb056708c9995070051b324c --force-share --output=json" returned: 0 in 0.134s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:51:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Acquiring lock "aac9bf03ba600a05bb056708c9995070051b324c" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:51:05 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "aac9bf03ba600a05bb056708c9995070051b324c" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:51:05 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/aac9bf03ba600a05bb056708c9995070051b324c --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:51:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/aac9bf03ba600a05bb056708c9995070051b324c --force-share --output=json" returned: 0 in 0.128s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:51:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/aac9bf03ba600a05bb056708c9995070051b324c,backing_fmt=raw /opt/stack/data/nova/instances/150f1b69-aa23-4702-a9f4-24025963befe/disk 1073741824 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:51:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/aac9bf03ba600a05bb056708c9995070051b324c,backing_fmt=raw /opt/stack/data/nova/instances/150f1b69-aa23-4702-a9f4-24025963befe/disk 1073741824" returned: 0 in 0.047s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:51:06 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "aac9bf03ba600a05bb056708c9995070051b324c" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.182s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:51:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/aac9bf03ba600a05bb056708c9995070051b324c --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:51:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/aac9bf03ba600a05bb056708c9995070051b324c --force-share --output=json" returned: 0 in 0.128s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:51:06 user nova-compute[72351]: DEBUG nova.objects.instance [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lazy-loading 'migration_context' on Instance uuid 150f1b69-aa23-4702-a9f4-24025963befe {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:51:06 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Rebasing disk image.
Aug 09 02:51:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:51:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.130s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:51:06 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Running cmd (subprocess): qemu-img rebase -b /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd -F raw /opt/stack/data/nova/instances/150f1b69-aa23-4702-a9f4-24025963befe/disk {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:51:08 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] CMD "qemu-img rebase -b /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd -F raw /opt/stack/data/nova/instances/150f1b69-aa23-4702-a9f4-24025963befe/disk" returned: 0 in 1.835s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:51:08 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Created local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4851}}
Aug 09 02:51:08 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Ensure instance console log exists: /opt/stack/data/nova/instances/150f1b69-aa23-4702-a9f4-24025963befe/console.log {{(pid=72351) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4603}}
Aug 09 02:51:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:51:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:51:08 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:51:08 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Start _get_guest_xml network_info=[{"id": "0983efc8-e53e-40bd-a721-2924155c20f3", "address": "fa:16:3e:d7:24:ef", "network": {"id": "da9be96c-ce20-4037-999c-230ca799fb06", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-661780305-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.240", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5ff300d27c9a4dd3b7ecd507b3627d98", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0983efc8-e5", "ovs_interfaceid": "0983efc8-e53e-40bd-a721-2924155c20f3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b17ce095aa7c47fc4403a80c3009d30e',container_format='bare',created_at=2023-08-09T02:49:10Z,direct_url=<?>,disk_format='qcow2',id=84e39abb-60a1-45ad-b0d9-5bf219191a3f,min_disk=1,min_ram=0,name='tempest-AttachVolumeShelveTestJSON-server-547567417-shelved',owner='5ff300d27c9a4dd3b7ecd507b3627d98',properties=ImageMetaProps,protected=<?>,size=52363264,status='active',tags=<?>,updated_at=2023-08-09T02:49:13Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'size': 0, 'device_type': 'disk', 'encryption_secret_uuid': None, 'guest_format': None, 'encryption_options': None, 'disk_bus': 'virtio', 'encrypted': False, 'encryption_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'image_id': '18ccda7f-31d3-4978-ab9d-4ed2f74eff94'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:51:08 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:51:08 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:51:08 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:51:08 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:51:08 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:51:08 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:51:08 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:51:08 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b17ce095aa7c47fc4403a80c3009d30e',container_format='bare',created_at=2023-08-09T02:49:10Z,direct_url=<?>,disk_format='qcow2',id=84e39abb-60a1-45ad-b0d9-5bf219191a3f,min_disk=1,min_ram=0,name='tempest-AttachVolumeShelveTestJSON-server-547567417-shelved',owner='5ff300d27c9a4dd3b7ecd507b3627d98',properties=ImageMetaProps,protected=<?>,size=52363264,status='active',tags=<?>,updated_at=2023-08-09T02:49:13Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:51:08 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:51:08 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:51:08 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:51:08 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:51:08 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:51:08 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:51:08 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:51:08 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:51:08 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:51:08 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:51:08 user nova-compute[72351]: DEBUG nova.objects.instance [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lazy-loading 'vcpu_model' on Instance uuid 150f1b69-aa23-4702-a9f4-24025963befe {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:51:08 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,compute_id=1,config_drive='',created_at=2023-08-09T02:47:13Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-server-547567417',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumeshelvetestjson-server-547567417',id=49,image_ref='84e39abb-60a1-45ad-b0d9-5bf219191a3f',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name='tempest-keypair-647885492',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:47:19Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=4,progress=0,project_id='5ff300d27c9a4dd3b7ecd507b3627d98',ramdisk_id='',reservation_id='r-x20xmo7b',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeShelveTestJSON-1982056968',owner_user_name='tempest-AttachVolumeShelveTestJSON-1982056968-project-member',shelved_at='2023-08-09T02:49:13.369328',shelved_host='user',shelved_image_id='84e39abb-60a1-45ad-b0d9-5bf219191a3f'},tags=<?>,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:51:03Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='ad8fc03436a84bdfb13ba6abecd4f27b',uuid=150f1b69-aa23-4702-a9f4-24025963befe,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='shelved_offloaded') vif={"id": "0983efc8-e53e-40bd-a721-2924155c20f3", "address": "fa:16:3e:d7:24:ef", "network": {"id": "da9be96c-ce20-4037-999c-230ca799fb06", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-661780305-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.240", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5ff300d27c9a4dd3b7ecd507b3627d98", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0983efc8-e5", "ovs_interfaceid": "0983efc8-e53e-40bd-a721-2924155c20f3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:51:08 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Converting VIF {"id": "0983efc8-e53e-40bd-a721-2924155c20f3", "address": "fa:16:3e:d7:24:ef", "network": {"id": "da9be96c-ce20-4037-999c-230ca799fb06", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-661780305-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.240", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5ff300d27c9a4dd3b7ecd507b3627d98", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0983efc8-e5", "ovs_interfaceid": "0983efc8-e53e-40bd-a721-2924155c20f3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:51:08 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:d7:24:ef,bridge_name='br-int',has_traffic_filtering=True,id=0983efc8-e53e-40bd-a721-2924155c20f3,network=Network(da9be96c-ce20-4037-999c-230ca799fb06),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap0983efc8-e5') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:51:08 user nova-compute[72351]: DEBUG nova.objects.instance [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lazy-loading 'pci_devices' on Instance uuid 150f1b69-aa23-4702-a9f4-24025963befe {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:51:08 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:51:08 user nova-compute[72351]:   <uuid>150f1b69-aa23-4702-a9f4-24025963befe</uuid>
Aug 09 02:51:08 user nova-compute[72351]:   <name>instance-00000031</name>
Aug 09 02:51:08 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:51:08 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:51:08 user nova-compute[72351]:   <metadata>
Aug 09 02:51:08 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:51:08 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:51:08 user nova-compute[72351]:       <nova:name>tempest-AttachVolumeShelveTestJSON-server-547567417</nova:name>
Aug 09 02:51:08 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:51:08</nova:creationTime>
Aug 09 02:51:08 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:51:08 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:51:08 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:51:08 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:51:08 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:51:08 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:51:08 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:51:08 user nova-compute[72351]:       <nova:owner>
Aug 09 02:51:08 user nova-compute[72351]:         <nova:user uuid="ad8fc03436a84bdfb13ba6abecd4f27b">tempest-AttachVolumeShelveTestJSON-1982056968-project-member</nova:user>
Aug 09 02:51:08 user nova-compute[72351]:         <nova:project uuid="5ff300d27c9a4dd3b7ecd507b3627d98">tempest-AttachVolumeShelveTestJSON-1982056968</nova:project>
Aug 09 02:51:08 user nova-compute[72351]:       </nova:owner>
Aug 09 02:51:08 user nova-compute[72351]:       <nova:root type="image" uuid="84e39abb-60a1-45ad-b0d9-5bf219191a3f"/>
Aug 09 02:51:08 user nova-compute[72351]:       <nova:ports>
Aug 09 02:51:08 user nova-compute[72351]:         <nova:port uuid="0983efc8-e53e-40bd-a721-2924155c20f3">
Aug 09 02:51:08 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.7" ipVersion="4"/>
Aug 09 02:51:08 user nova-compute[72351]:         </nova:port>
Aug 09 02:51:08 user nova-compute[72351]:       </nova:ports>
Aug 09 02:51:08 user nova-compute[72351]:     </nova:instance>
Aug 09 02:51:08 user nova-compute[72351]:   </metadata>
Aug 09 02:51:08 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:51:08 user nova-compute[72351]:     <system>
Aug 09 02:51:08 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:51:08 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:51:08 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:51:08 user nova-compute[72351]:       <entry name="serial">150f1b69-aa23-4702-a9f4-24025963befe</entry>
Aug 09 02:51:08 user nova-compute[72351]:       <entry name="uuid">150f1b69-aa23-4702-a9f4-24025963befe</entry>
Aug 09 02:51:08 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:51:08 user nova-compute[72351]:     </system>
Aug 09 02:51:08 user nova-compute[72351]:   </sysinfo>
Aug 09 02:51:08 user nova-compute[72351]:   <os>
Aug 09 02:51:08 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:51:08 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:51:08 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:51:08 user nova-compute[72351]:   </os>
Aug 09 02:51:08 user nova-compute[72351]:   <features>
Aug 09 02:51:08 user nova-compute[72351]:     <acpi/>
Aug 09 02:51:08 user nova-compute[72351]:     <apic/>
Aug 09 02:51:08 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:51:08 user nova-compute[72351]:   </features>
Aug 09 02:51:08 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:51:08 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:51:08 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:51:08 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:51:08 user nova-compute[72351]:   </clock>
Aug 09 02:51:08 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:51:08 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:51:08 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:51:08 user nova-compute[72351]:   </cpu>
Aug 09 02:51:08 user nova-compute[72351]:   <devices>
Aug 09 02:51:08 user nova-compute[72351]:     <disk type="file" device="disk">
Aug 09 02:51:08 user nova-compute[72351]:       <driver name="qemu" type="qcow2" cache="none"/>
Aug 09 02:51:08 user nova-compute[72351]:       <source file="/opt/stack/data/nova/instances/150f1b69-aa23-4702-a9f4-24025963befe/disk"/>
Aug 09 02:51:08 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:51:08 user nova-compute[72351]:     </disk>
Aug 09 02:51:08 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:51:08 user nova-compute[72351]:       <mac address="fa:16:3e:d7:24:ef"/>
Aug 09 02:51:08 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:51:08 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:51:08 user nova-compute[72351]:       <target dev="tap0983efc8-e5"/>
Aug 09 02:51:08 user nova-compute[72351]:     </interface>
Aug 09 02:51:08 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:51:08 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/150f1b69-aa23-4702-a9f4-24025963befe/console.log" append="off"/>
Aug 09 02:51:08 user nova-compute[72351]:     </serial>
Aug 09 02:51:08 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:51:08 user nova-compute[72351]:     <video>
Aug 09 02:51:08 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:51:08 user nova-compute[72351]:     </video>
Aug 09 02:51:08 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:51:08 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:51:08 user nova-compute[72351]:     </rng>
Aug 09 02:51:08 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:51:08 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:51:08 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:51:08 user nova-compute[72351]:     </memballoon>
Aug 09 02:51:08 user nova-compute[72351]:   </devices>
Aug 09 02:51:08 user nova-compute[72351]: </domain>
Aug 09 02:51:08 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:51:08 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,compute_id=1,config_drive='',created_at=2023-08-09T02:47:13Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-server-547567417',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumeshelvetestjson-server-547567417',id=49,image_ref='84e39abb-60a1-45ad-b0d9-5bf219191a3f',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name='tempest-keypair-647885492',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:47:19Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=4,progress=0,project_id='5ff300d27c9a4dd3b7ecd507b3627d98',ramdisk_id='',reservation_id='r-x20xmo7b',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeShelveTestJSON-1982056968',owner_user_name='tempest-AttachVolumeShelveTestJSON-1982056968-project-member',shelved_at='2023-08-09T02:49:13.369328',shelved_host='user',shelved_image_id='84e39abb-60a1-45ad-b0d9-5bf219191a3f'},tags=<?>,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:51:03Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='ad8fc03436a84bdfb13ba6abecd4f27b',uuid=150f1b69-aa23-4702-a9f4-24025963befe,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='shelved_offloaded') vif={"id": "0983efc8-e53e-40bd-a721-2924155c20f3", "address": "fa:16:3e:d7:24:ef", "network": {"id": "da9be96c-ce20-4037-999c-230ca799fb06", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-661780305-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.240", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5ff300d27c9a4dd3b7ecd507b3627d98", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0983efc8-e5", "ovs_interfaceid": "0983efc8-e53e-40bd-a721-2924155c20f3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:51:08 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Converting VIF {"id": "0983efc8-e53e-40bd-a721-2924155c20f3", "address": "fa:16:3e:d7:24:ef", "network": {"id": "da9be96c-ce20-4037-999c-230ca799fb06", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-661780305-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.240", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5ff300d27c9a4dd3b7ecd507b3627d98", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0983efc8-e5", "ovs_interfaceid": "0983efc8-e53e-40bd-a721-2924155c20f3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:51:08 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:d7:24:ef,bridge_name='br-int',has_traffic_filtering=True,id=0983efc8-e53e-40bd-a721-2924155c20f3,network=Network(da9be96c-ce20-4037-999c-230ca799fb06),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap0983efc8-e5') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:51:08 user nova-compute[72351]: DEBUG os_vif [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:d7:24:ef,bridge_name='br-int',has_traffic_filtering=True,id=0983efc8-e53e-40bd-a721-2924155c20f3,network=Network(da9be96c-ce20-4037-999c-230ca799fb06),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap0983efc8-e5') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:51:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:51:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:51:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:51:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:51:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap0983efc8-e5, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:51:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap0983efc8-e5, col_values=(('external_ids', {'iface-id': '0983efc8-e53e-40bd-a721-2924155c20f3', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:d7:24:ef', 'vm-uuid': '150f1b69-aa23-4702-a9f4-24025963befe'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:51:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:51:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:51:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:51:08 user nova-compute[72351]: INFO os_vif [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:d7:24:ef,bridge_name='br-int',has_traffic_filtering=True,id=0983efc8-e53e-40bd-a721-2924155c20f3,network=Network(da9be96c-ce20-4037-999c-230ca799fb06),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap0983efc8-e5')
Aug 09 02:51:08 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:51:08 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] No VIF found with MAC fa:16:3e:d7:24:ef, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:51:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:51:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:51:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:51:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:51:10 user nova-compute[72351]: DEBUG nova.compute.manager [req-2f716c0a-c6ce-42dc-82e7-b20f9dce386e req-82c25818-5b4a-4e9f-ac78-fa951a70821e service nova] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Received event network-vif-plugged-0983efc8-e53e-40bd-a721-2924155c20f3 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:51:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-2f716c0a-c6ce-42dc-82e7-b20f9dce386e req-82c25818-5b4a-4e9f-ac78-fa951a70821e service nova] Acquiring lock "150f1b69-aa23-4702-a9f4-24025963befe-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:51:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-2f716c0a-c6ce-42dc-82e7-b20f9dce386e req-82c25818-5b4a-4e9f-ac78-fa951a70821e service nova] Lock "150f1b69-aa23-4702-a9f4-24025963befe-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:51:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-2f716c0a-c6ce-42dc-82e7-b20f9dce386e req-82c25818-5b4a-4e9f-ac78-fa951a70821e service nova] Lock "150f1b69-aa23-4702-a9f4-24025963befe-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:51:10 user nova-compute[72351]: DEBUG nova.compute.manager [req-2f716c0a-c6ce-42dc-82e7-b20f9dce386e req-82c25818-5b4a-4e9f-ac78-fa951a70821e service nova] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] No waiting events found dispatching network-vif-plugged-0983efc8-e53e-40bd-a721-2924155c20f3 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:51:10 user nova-compute[72351]: WARNING nova.compute.manager [req-2f716c0a-c6ce-42dc-82e7-b20f9dce386e req-82c25818-5b4a-4e9f-ac78-fa951a70821e service nova] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Received unexpected event network-vif-plugged-0983efc8-e53e-40bd-a721-2924155c20f3 for instance with vm_state shelved_offloaded and task_state spawning.
Aug 09 02:51:10 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:51:10 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:51:10 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:51:10 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:51:10 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:51:11 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691549471.8466918, 150f1b69-aa23-4702-a9f4-24025963befe => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:51:11 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] VM Resumed (Lifecycle Event)
Aug 09 02:51:11 user nova-compute[72351]: DEBUG nova.compute.manager [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:51:11 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Guest created on hypervisor {{(pid=72351) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4411}}
Aug 09 02:51:11 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Instance spawned successfully.
Aug 09 02:51:11 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:51:11 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: shelved_offloaded, current task_state: spawning, current DB power_state: 4, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:51:11 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:51:11 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691549471.847399, 150f1b69-aa23-4702-a9f4-24025963befe => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:51:11 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] VM Started (Lifecycle Event)
Aug 09 02:51:11 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:51:11 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Synchronizing instance power state after lifecycle event "Started"; current vm_state: shelved_offloaded, current task_state: spawning, current DB power_state: 4, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:51:11 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:51:12 user nova-compute[72351]: DEBUG nova.compute.manager [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:51:12 user nova-compute[72351]: DEBUG nova.compute.manager [req-4bb168df-3aad-4199-a544-96ff06005140 req-caa463b4-57af-4470-b14c-ddc00c271c31 service nova] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Received event network-vif-plugged-0983efc8-e53e-40bd-a721-2924155c20f3 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:51:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-4bb168df-3aad-4199-a544-96ff06005140 req-caa463b4-57af-4470-b14c-ddc00c271c31 service nova] Acquiring lock "150f1b69-aa23-4702-a9f4-24025963befe-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:51:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-4bb168df-3aad-4199-a544-96ff06005140 req-caa463b4-57af-4470-b14c-ddc00c271c31 service nova] Lock "150f1b69-aa23-4702-a9f4-24025963befe-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:51:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-4bb168df-3aad-4199-a544-96ff06005140 req-caa463b4-57af-4470-b14c-ddc00c271c31 service nova] Lock "150f1b69-aa23-4702-a9f4-24025963befe-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:51:12 user nova-compute[72351]: DEBUG nova.compute.manager [req-4bb168df-3aad-4199-a544-96ff06005140 req-caa463b4-57af-4470-b14c-ddc00c271c31 service nova] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] No waiting events found dispatching network-vif-plugged-0983efc8-e53e-40bd-a721-2924155c20f3 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:51:12 user nova-compute[72351]: WARNING nova.compute.manager [req-4bb168df-3aad-4199-a544-96ff06005140 req-caa463b4-57af-4470-b14c-ddc00c271c31 service nova] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Received unexpected event network-vif-plugged-0983efc8-e53e-40bd-a721-2924155c20f3 for instance with vm_state shelved_offloaded and task_state spawning.
Aug 09 02:51:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1323d045-bf9d-4c65-9f4b-d10b02a6ba2a tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "150f1b69-aa23-4702-a9f4-24025963befe" "released" by "nova.compute.manager.ComputeManager.unshelve_instance.<locals>.do_unshelve_instance" :: held 9.380s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:51:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:51:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:51:17 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:51:17 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72351) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10461}}
Aug 09 02:51:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:51:18 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:51:21 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:51:23 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:51:23 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:51:23 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 02:51:23 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:51:23 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:51:23 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:51:23 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:51:23 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Starting heal instance info cache {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9842}}
Aug 09 02:51:23 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Rebuilding the list of instances to heal {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9846}}
Aug 09 02:51:23 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "refresh_cache-c6ae885f-dce2-4dc2-98d5-26139b8fb67e" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:51:23 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquired lock "refresh_cache-c6ae885f-dce2-4dc2-98d5-26139b8fb67e" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:51:23 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Forcefully refreshing network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1996}}
Aug 09 02:51:23 user nova-compute[72351]: DEBUG nova.objects.instance [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lazy-loading 'info_cache' on Instance uuid c6ae885f-dce2-4dc2-98d5-26139b8fb67e {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:51:24 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Updating instance_info_cache with network_info: [{"id": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "address": "fa:16:3e:dd:49:4d", "network": {"id": "dba57c3d-6efd-4057-8053-1c6a92bb24f8", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-568585773-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.116", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ec9861e0882e4065ab99ad3f74d6b25f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2771d93d-fa", "ovs_interfaceid": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:51:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Releasing lock "refresh_cache-c6ae885f-dce2-4dc2-98d5-26139b8fb67e" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:51:24 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Updated the network info_cache for instance {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9913}}
Aug 09 02:51:24 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:51:25 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:51:26 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:51:26 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:51:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:51:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:51:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:51:26 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Auditing locally available compute resources for user (node: user) {{(pid=72351) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Aug 09 02:51:27 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/0d4c65d2-c641-4985-8115-24dcbc15555f/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:51:27 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/0d4c65d2-c641-4985-8115-24dcbc15555f/disk --force-share --output=json" returned: 0 in 0.140s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:51:27 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/0d4c65d2-c641-4985-8115-24dcbc15555f/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:51:27 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/0d4c65d2-c641-4985-8115-24dcbc15555f/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:51:27 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:51:27 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:51:27 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:51:27 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:51:27 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/150f1b69-aa23-4702-a9f4-24025963befe/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:51:27 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/150f1b69-aa23-4702-a9f4-24025963befe/disk --force-share --output=json" returned: 0 in 0.153s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:51:27 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/150f1b69-aa23-4702-a9f4-24025963befe/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:51:27 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/150f1b69-aa23-4702-a9f4-24025963befe/disk --force-share --output=json" returned: 0 in 0.131s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:51:28 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:51:28 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:51:28 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Hypervisor/Node resource view: name=user free_ram=8548MB free_disk=25.169937133789062GB free_vcpus=10 pci_devices=[{"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72351) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Aug 09 02:51:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:51:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:51:28 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance c6ae885f-dce2-4dc2-98d5-26139b8fb67e actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:51:28 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 0d4c65d2-c641-4985-8115-24dcbc15555f actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:51:28 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 150f1b69-aa23-4702-a9f4-24025963befe actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:51:28 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Total usable vcpus: 12, total allocated vcpus: 3 {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Aug 09 02:51:28 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Final resource view: name=user phys_ram=16011MB used_ram=2048MB phys_disk=40GB used_disk=3GB total_vcpus=12 used_vcpus=3 pci_stats=[] {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Aug 09 02:51:28 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:51:28 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:51:28 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:51:28 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Compute_service record updated for user:user {{(pid=72351) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Aug 09 02:51:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.274s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:51:29 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:51:31 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:51:31 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:51:33 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:51:34 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:51:38 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:51:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:51:43 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:51:48 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:51:48 user nova-compute[72351]: DEBUG nova.objects.instance [None req-d6cf0daf-4c20-4be3-a1fb-4a828c11a40f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lazy-loading 'flavor' on Instance uuid c6ae885f-dce2-4dc2-98d5-26139b8fb67e {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:51:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-d6cf0daf-4c20-4be3-a1fb-4a828c11a40f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Acquiring lock "refresh_cache-c6ae885f-dce2-4dc2-98d5-26139b8fb67e" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:51:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-d6cf0daf-4c20-4be3-a1fb-4a828c11a40f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Acquired lock "refresh_cache-c6ae885f-dce2-4dc2-98d5-26139b8fb67e" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:51:48 user nova-compute[72351]: DEBUG nova.network.neutron [None req-d6cf0daf-4c20-4be3-a1fb-4a828c11a40f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:51:48 user nova-compute[72351]: DEBUG nova.objects.instance [None req-d6cf0daf-4c20-4be3-a1fb-4a828c11a40f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lazy-loading 'info_cache' on Instance uuid c6ae885f-dce2-4dc2-98d5-26139b8fb67e {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG nova.network.neutron [None req-d6cf0daf-4c20-4be3-a1fb-4a828c11a40f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Updating instance_info_cache with network_info: [{"id": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "address": "fa:16:3e:dd:49:4d", "network": {"id": "dba57c3d-6efd-4057-8053-1c6a92bb24f8", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-568585773-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.116", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ec9861e0882e4065ab99ad3f74d6b25f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2771d93d-fa", "ovs_interfaceid": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-d6cf0daf-4c20-4be3-a1fb-4a828c11a40f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Releasing lock "refresh_cache-c6ae885f-dce2-4dc2-98d5-26139b8fb67e" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:51:49 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Instance destroyed successfully.
Aug 09 02:51:49 user nova-compute[72351]: DEBUG nova.objects.instance [None req-d6cf0daf-4c20-4be3-a1fb-4a828c11a40f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lazy-loading 'resources' on Instance uuid c6ae885f-dce2-4dc2-98d5-26139b8fb67e {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-d6cf0daf-4c20-4be3-a1fb-4a828c11a40f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:44:08Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachVolumeTestJSON-server-218910074',display_name='tempest-AttachVolumeTestJSON-server-218910074',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumetestjson-server-218910074',id=47,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBIplDVAhYRmCgaFwiRMKOJ3OlhP3n+pFAzSDpMz7BP4WftIaiuBT1r87OWLI3l8JRBwh8sp/09wt0bUhVeTDk2tWONkdagBfHB77ctUB1RkuePwrxnVW0j07TkHZdA5xrw==',key_name='tempest-keypair-304755592',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:44:14Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=<?>,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=4,progress=0,project_id='ec9861e0882e4065ab99ad3f74d6b25f',ramdisk_id='',reservation_id='r-2vl0h2f4',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=<?>,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeTestJSON-1725368857',owner_user_name='tempest-AttachVolumeTestJSON-1725368857-project-member'},tags=<?>,task_state='powering-on',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:49:59Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='a2bc49dd654f4beca4b8eb5a49cd350d',uuid=c6ae885f-dce2-4dc2-98d5-26139b8fb67e,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='stopped') vif={"id": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "address": "fa:16:3e:dd:49:4d", "network": {"id": "dba57c3d-6efd-4057-8053-1c6a92bb24f8", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-568585773-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.116", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ec9861e0882e4065ab99ad3f74d6b25f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2771d93d-fa", "ovs_interfaceid": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-d6cf0daf-4c20-4be3-a1fb-4a828c11a40f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Converting VIF {"id": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "address": "fa:16:3e:dd:49:4d", "network": {"id": "dba57c3d-6efd-4057-8053-1c6a92bb24f8", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-568585773-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.116", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ec9861e0882e4065ab99ad3f74d6b25f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2771d93d-fa", "ovs_interfaceid": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-d6cf0daf-4c20-4be3-a1fb-4a828c11a40f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:dd:49:4d,bridge_name='br-int',has_traffic_filtering=True,id=2771d93d-fa0a-4e19-85ac-9d3f5df52782,network=Network(dba57c3d-6efd-4057-8053-1c6a92bb24f8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2771d93d-fa') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG os_vif [None req-d6cf0daf-4c20-4be3-a1fb-4a828c11a40f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:dd:49:4d,bridge_name='br-int',has_traffic_filtering=True,id=2771d93d-fa0a-4e19-85ac-9d3f5df52782,network=Network(dba57c3d-6efd-4057-8053-1c6a92bb24f8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2771d93d-fa') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap2771d93d-fa, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-194e37c6-de48-4dab-b559-4027cd7af715 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Acquiring lock "0d4c65d2-c641-4985-8115-24dcbc15555f" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-194e37c6-de48-4dab-b559-4027cd7af715 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "0d4c65d2-c641-4985-8115-24dcbc15555f" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-194e37c6-de48-4dab-b559-4027cd7af715 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Acquiring lock "0d4c65d2-c641-4985-8115-24dcbc15555f-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-194e37c6-de48-4dab-b559-4027cd7af715 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "0d4c65d2-c641-4985-8115-24dcbc15555f-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-194e37c6-de48-4dab-b559-4027cd7af715 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "0d4c65d2-c641-4985-8115-24dcbc15555f-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:51:49 user nova-compute[72351]: INFO nova.compute.manager [None req-194e37c6-de48-4dab-b559-4027cd7af715 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Terminating instance
Aug 09 02:51:49 user nova-compute[72351]: DEBUG nova.compute.manager [None req-194e37c6-de48-4dab-b559-4027cd7af715 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Start destroying the instance on the hypervisor. {{(pid=72351) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3124}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:51:49 user nova-compute[72351]: INFO os_vif [None req-d6cf0daf-4c20-4be3-a1fb-4a828c11a40f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:dd:49:4d,bridge_name='br-int',has_traffic_filtering=True,id=2771d93d-fa0a-4e19-85ac-9d3f5df52782,network=Network(dba57c3d-6efd-4057-8053-1c6a92bb24f8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2771d93d-fa')
Aug 09 02:51:49 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-d6cf0daf-4c20-4be3-a1fb-4a828c11a40f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Start _get_guest_xml network_info=[{"id": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "address": "fa:16:3e:dd:49:4d", "network": {"id": "dba57c3d-6efd-4057-8053-1c6a92bb24f8", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-568585773-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.116", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ec9861e0882e4065ab99ad3f74d6b25f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2771d93d-fa", "ovs_interfaceid": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum=<?>,container_format='bare',created_at=<?>,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=1,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=<?>,status=<?>,tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'size': 0, 'device_type': 'disk', 'encryption_secret_uuid': None, 'guest_format': None, 'encryption_options': None, 'disk_bus': 'virtio', 'encrypted': False, 'encryption_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'image_id': '18ccda7f-31d3-4978-ab9d-4ed2f74eff94'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG nova.objects.instance [None req-d6cf0daf-4c20-4be3-a1fb-4a828c11a40f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lazy-loading 'numa_topology' on Instance uuid c6ae885f-dce2-4dc2-98d5-26139b8fb67e {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:51:49 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-d6cf0daf-4c20-4be3-a1fb-4a828c11a40f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:51:49 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-d6cf0daf-4c20-4be3-a1fb-4a828c11a40f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:51:49 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-d6cf0daf-4c20-4be3-a1fb-4a828c11a40f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-d6cf0daf-4c20-4be3-a1fb-4a828c11a40f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-d6cf0daf-4c20-4be3-a1fb-4a828c11a40f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-d6cf0daf-4c20-4be3-a1fb-4a828c11a40f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-d6cf0daf-4c20-4be3-a1fb-4a828c11a40f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-d6cf0daf-4c20-4be3-a1fb-4a828c11a40f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format='bare',created_at=<?>,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=1,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=<?>,status=<?>,tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-d6cf0daf-4c20-4be3-a1fb-4a828c11a40f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-d6cf0daf-4c20-4be3-a1fb-4a828c11a40f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-d6cf0daf-4c20-4be3-a1fb-4a828c11a40f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-d6cf0daf-4c20-4be3-a1fb-4a828c11a40f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-d6cf0daf-4c20-4be3-a1fb-4a828c11a40f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-d6cf0daf-4c20-4be3-a1fb-4a828c11a40f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-d6cf0daf-4c20-4be3-a1fb-4a828c11a40f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-d6cf0daf-4c20-4be3-a1fb-4a828c11a40f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-d6cf0daf-4c20-4be3-a1fb-4a828c11a40f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-d6cf0daf-4c20-4be3-a1fb-4a828c11a40f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG nova.objects.instance [None req-d6cf0daf-4c20-4be3-a1fb-4a828c11a40f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lazy-loading 'vcpu_model' on Instance uuid c6ae885f-dce2-4dc2-98d5-26139b8fb67e {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-d6cf0daf-4c20-4be3-a1fb-4a828c11a40f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:44:08Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachVolumeTestJSON-server-218910074',display_name='tempest-AttachVolumeTestJSON-server-218910074',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumetestjson-server-218910074',id=47,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBIplDVAhYRmCgaFwiRMKOJ3OlhP3n+pFAzSDpMz7BP4WftIaiuBT1r87OWLI3l8JRBwh8sp/09wt0bUhVeTDk2tWONkdagBfHB77ctUB1RkuePwrxnVW0j07TkHZdA5xrw==',key_name='tempest-keypair-304755592',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:44:14Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=4,progress=0,project_id='ec9861e0882e4065ab99ad3f74d6b25f',ramdisk_id='',reservation_id='r-2vl0h2f4',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=<?>,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeTestJSON-1725368857',owner_user_name='tempest-AttachVolumeTestJSON-1725368857-project-member'},tags=<?>,task_state='powering-on',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:49:59Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='a2bc49dd654f4beca4b8eb5a49cd350d',uuid=c6ae885f-dce2-4dc2-98d5-26139b8fb67e,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='stopped') vif={"id": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "address": "fa:16:3e:dd:49:4d", "network": {"id": "dba57c3d-6efd-4057-8053-1c6a92bb24f8", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-568585773-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.116", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ec9861e0882e4065ab99ad3f74d6b25f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2771d93d-fa", "ovs_interfaceid": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-d6cf0daf-4c20-4be3-a1fb-4a828c11a40f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Converting VIF {"id": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "address": "fa:16:3e:dd:49:4d", "network": {"id": "dba57c3d-6efd-4057-8053-1c6a92bb24f8", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-568585773-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.116", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ec9861e0882e4065ab99ad3f74d6b25f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2771d93d-fa", "ovs_interfaceid": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-d6cf0daf-4c20-4be3-a1fb-4a828c11a40f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:dd:49:4d,bridge_name='br-int',has_traffic_filtering=True,id=2771d93d-fa0a-4e19-85ac-9d3f5df52782,network=Network(dba57c3d-6efd-4057-8053-1c6a92bb24f8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2771d93d-fa') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG nova.objects.instance [None req-d6cf0daf-4c20-4be3-a1fb-4a828c11a40f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lazy-loading 'pci_devices' on Instance uuid c6ae885f-dce2-4dc2-98d5-26139b8fb67e {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-d6cf0daf-4c20-4be3-a1fb-4a828c11a40f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:51:49 user nova-compute[72351]:   <uuid>c6ae885f-dce2-4dc2-98d5-26139b8fb67e</uuid>
Aug 09 02:51:49 user nova-compute[72351]:   <name>instance-0000002f</name>
Aug 09 02:51:49 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:51:49 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:51:49 user nova-compute[72351]:   <metadata>
Aug 09 02:51:49 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:51:49 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:51:49 user nova-compute[72351]:       <nova:name>tempest-AttachVolumeTestJSON-server-218910074</nova:name>
Aug 09 02:51:49 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:51:49</nova:creationTime>
Aug 09 02:51:49 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:51:49 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:51:49 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:51:49 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:51:49 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:51:49 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:51:49 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:51:49 user nova-compute[72351]:       <nova:owner>
Aug 09 02:51:49 user nova-compute[72351]:         <nova:user uuid="a2bc49dd654f4beca4b8eb5a49cd350d">tempest-AttachVolumeTestJSON-1725368857-project-member</nova:user>
Aug 09 02:51:49 user nova-compute[72351]:         <nova:project uuid="ec9861e0882e4065ab99ad3f74d6b25f">tempest-AttachVolumeTestJSON-1725368857</nova:project>
Aug 09 02:51:49 user nova-compute[72351]:       </nova:owner>
Aug 09 02:51:49 user nova-compute[72351]:       <nova:root type="image" uuid="18ccda7f-31d3-4978-ab9d-4ed2f74eff94"/>
Aug 09 02:51:49 user nova-compute[72351]:       <nova:ports>
Aug 09 02:51:49 user nova-compute[72351]:         <nova:port uuid="2771d93d-fa0a-4e19-85ac-9d3f5df52782">
Aug 09 02:51:49 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.12" ipVersion="4"/>
Aug 09 02:51:49 user nova-compute[72351]:         </nova:port>
Aug 09 02:51:49 user nova-compute[72351]:       </nova:ports>
Aug 09 02:51:49 user nova-compute[72351]:     </nova:instance>
Aug 09 02:51:49 user nova-compute[72351]:   </metadata>
Aug 09 02:51:49 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:51:49 user nova-compute[72351]:     <system>
Aug 09 02:51:49 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:51:49 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:51:49 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:51:49 user nova-compute[72351]:       <entry name="serial">c6ae885f-dce2-4dc2-98d5-26139b8fb67e</entry>
Aug 09 02:51:49 user nova-compute[72351]:       <entry name="uuid">c6ae885f-dce2-4dc2-98d5-26139b8fb67e</entry>
Aug 09 02:51:49 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:51:49 user nova-compute[72351]:     </system>
Aug 09 02:51:49 user nova-compute[72351]:   </sysinfo>
Aug 09 02:51:49 user nova-compute[72351]:   <os>
Aug 09 02:51:49 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:51:49 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:51:49 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:51:49 user nova-compute[72351]:   </os>
Aug 09 02:51:49 user nova-compute[72351]:   <features>
Aug 09 02:51:49 user nova-compute[72351]:     <acpi/>
Aug 09 02:51:49 user nova-compute[72351]:     <apic/>
Aug 09 02:51:49 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:51:49 user nova-compute[72351]:   </features>
Aug 09 02:51:49 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:51:49 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:51:49 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:51:49 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:51:49 user nova-compute[72351]:   </clock>
Aug 09 02:51:49 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:51:49 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:51:49 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:51:49 user nova-compute[72351]:   </cpu>
Aug 09 02:51:49 user nova-compute[72351]:   <devices>
Aug 09 02:51:49 user nova-compute[72351]:     <disk type="file" device="disk">
Aug 09 02:51:49 user nova-compute[72351]:       <driver name="qemu" type="qcow2" cache="none"/>
Aug 09 02:51:49 user nova-compute[72351]:       <source file="/opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk"/>
Aug 09 02:51:49 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:51:49 user nova-compute[72351]:     </disk>
Aug 09 02:51:49 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:51:49 user nova-compute[72351]:       <mac address="fa:16:3e:dd:49:4d"/>
Aug 09 02:51:49 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:51:49 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:51:49 user nova-compute[72351]:       <target dev="tap2771d93d-fa"/>
Aug 09 02:51:49 user nova-compute[72351]:     </interface>
Aug 09 02:51:49 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:51:49 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/console.log" append="off"/>
Aug 09 02:51:49 user nova-compute[72351]:     </serial>
Aug 09 02:51:49 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:51:49 user nova-compute[72351]:     <video>
Aug 09 02:51:49 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:51:49 user nova-compute[72351]:     </video>
Aug 09 02:51:49 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:51:49 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:51:49 user nova-compute[72351]:     </rng>
Aug 09 02:51:49 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:51:49 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:51:49 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:51:49 user nova-compute[72351]:     </memballoon>
Aug 09 02:51:49 user nova-compute[72351]:   </devices>
Aug 09 02:51:49 user nova-compute[72351]: </domain>
Aug 09 02:51:49 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-d6cf0daf-4c20-4be3-a1fb-4a828c11a40f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG nova.compute.manager [req-ef93e71e-c3f8-4309-9dbf-305d425efc83 req-657927a1-a5db-4d3d-a49a-225cf58d70c1 service nova] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Received event network-vif-unplugged-c0c9db92-f7f2-4a6d-90d0-bfe48762cc6e {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ef93e71e-c3f8-4309-9dbf-305d425efc83 req-657927a1-a5db-4d3d-a49a-225cf58d70c1 service nova] Acquiring lock "0d4c65d2-c641-4985-8115-24dcbc15555f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ef93e71e-c3f8-4309-9dbf-305d425efc83 req-657927a1-a5db-4d3d-a49a-225cf58d70c1 service nova] Lock "0d4c65d2-c641-4985-8115-24dcbc15555f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-ef93e71e-c3f8-4309-9dbf-305d425efc83 req-657927a1-a5db-4d3d-a49a-225cf58d70c1 service nova] Lock "0d4c65d2-c641-4985-8115-24dcbc15555f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG nova.compute.manager [req-ef93e71e-c3f8-4309-9dbf-305d425efc83 req-657927a1-a5db-4d3d-a49a-225cf58d70c1 service nova] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] No waiting events found dispatching network-vif-unplugged-c0c9db92-f7f2-4a6d-90d0-bfe48762cc6e {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG nova.compute.manager [req-ef93e71e-c3f8-4309-9dbf-305d425efc83 req-657927a1-a5db-4d3d-a49a-225cf58d70c1 service nova] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Received event network-vif-unplugged-c0c9db92-f7f2-4a6d-90d0-bfe48762cc6e for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-d6cf0daf-4c20-4be3-a1fb-4a828c11a40f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk --force-share --output=json" returned: 0 in 0.159s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-d6cf0daf-4c20-4be3-a1fb-4a828c11a40f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-d6cf0daf-4c20-4be3-a1fb-4a828c11a40f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG nova.objects.instance [None req-d6cf0daf-4c20-4be3-a1fb-4a828c11a40f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lazy-loading 'trusted_certs' on Instance uuid c6ae885f-dce2-4dc2-98d5-26139b8fb67e {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-d6cf0daf-4c20-4be3-a1fb-4a828c11a40f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-d6cf0daf-4c20-4be3-a1fb-4a828c11a40f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.137s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-d6cf0daf-4c20-4be3-a1fb-4a828c11a40f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Checking if we can resize image /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk. size=1073741824 {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-d6cf0daf-4c20-4be3-a1fb-4a828c11a40f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-d6cf0daf-4c20-4be3-a1fb-4a828c11a40f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk --force-share --output=json" returned: 0 in 0.153s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-d6cf0daf-4c20-4be3-a1fb-4a828c11a40f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Cannot resize image /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk to a smaller size. {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG nova.objects.instance [None req-d6cf0daf-4c20-4be3-a1fb-4a828c11a40f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lazy-loading 'migration_context' on Instance uuid c6ae885f-dce2-4dc2-98d5-26139b8fb67e {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-d6cf0daf-4c20-4be3-a1fb-4a828c11a40f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:44:08Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachVolumeTestJSON-server-218910074',display_name='tempest-AttachVolumeTestJSON-server-218910074',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumetestjson-server-218910074',id=47,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBIplDVAhYRmCgaFwiRMKOJ3OlhP3n+pFAzSDpMz7BP4WftIaiuBT1r87OWLI3l8JRBwh8sp/09wt0bUhVeTDk2tWONkdagBfHB77ctUB1RkuePwrxnVW0j07TkHZdA5xrw==',key_name='tempest-keypair-304755592',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:44:14Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=<?>,power_state=4,progress=0,project_id='ec9861e0882e4065ab99ad3f74d6b25f',ramdisk_id='',reservation_id='r-2vl0h2f4',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=<?>,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeTestJSON-1725368857',owner_user_name='tempest-AttachVolumeTestJSON-1725368857-project-member'},tags=<?>,task_state='powering-on',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:49:59Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='a2bc49dd654f4beca4b8eb5a49cd350d',uuid=c6ae885f-dce2-4dc2-98d5-26139b8fb67e,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='stopped') vif={"id": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "address": "fa:16:3e:dd:49:4d", "network": {"id": "dba57c3d-6efd-4057-8053-1c6a92bb24f8", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-568585773-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.116", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ec9861e0882e4065ab99ad3f74d6b25f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2771d93d-fa", "ovs_interfaceid": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-d6cf0daf-4c20-4be3-a1fb-4a828c11a40f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Converting VIF {"id": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "address": "fa:16:3e:dd:49:4d", "network": {"id": "dba57c3d-6efd-4057-8053-1c6a92bb24f8", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-568585773-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.116", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ec9861e0882e4065ab99ad3f74d6b25f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2771d93d-fa", "ovs_interfaceid": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-d6cf0daf-4c20-4be3-a1fb-4a828c11a40f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:dd:49:4d,bridge_name='br-int',has_traffic_filtering=True,id=2771d93d-fa0a-4e19-85ac-9d3f5df52782,network=Network(dba57c3d-6efd-4057-8053-1c6a92bb24f8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2771d93d-fa') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG os_vif [None req-d6cf0daf-4c20-4be3-a1fb-4a828c11a40f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:dd:49:4d,bridge_name='br-int',has_traffic_filtering=True,id=2771d93d-fa0a-4e19-85ac-9d3f5df52782,network=Network(dba57c3d-6efd-4057-8053-1c6a92bb24f8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2771d93d-fa') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap2771d93d-fa, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap2771d93d-fa, col_values=(('external_ids', {'iface-id': '2771d93d-fa0a-4e19-85ac-9d3f5df52782', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:dd:49:4d', 'vm-uuid': 'c6ae885f-dce2-4dc2-98d5-26139b8fb67e'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:51:49 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Instance destroyed successfully.
Aug 09 02:51:49 user nova-compute[72351]: DEBUG nova.objects.instance [None req-194e37c6-de48-4dab-b559-4027cd7af715 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lazy-loading 'resources' on Instance uuid 0d4c65d2-c641-4985-8115-24dcbc15555f {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:51:49 user nova-compute[72351]: INFO os_vif [None req-d6cf0daf-4c20-4be3-a1fb-4a828c11a40f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:dd:49:4d,bridge_name='br-int',has_traffic_filtering=True,id=2771d93d-fa0a-4e19-85ac-9d3f5df52782,network=Network(dba57c3d-6efd-4057-8053-1c6a92bb24f8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2771d93d-fa')
Aug 09 02:51:49 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-194e37c6-de48-4dab-b559-4027cd7af715 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:48:47Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachVolumeNegativeTest-server-469412905',display_name='tempest-AttachVolumeNegativeTest-server-469412905',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumenegativetest-server-469412905',id=50,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHgU3jYUGV9FAA8guL1Tj+BZlg1d7wi47Hb565DUMcyiQvfNEhdjrF/FS2zNWPjiwHt3nj4XcSDefgraL9MvGTKIwkeBYQhe+INpMsQvhbKox5N3Z6a2GDe0PbyTrwSXjw==',key_name='tempest-keypair-27411291',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:48:53Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='8c07a6b254414b41a673468238d3aa62',ramdisk_id='',reservation_id='r-7d8pn0x5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeNegativeTest-758771161',owner_user_name='tempest-AttachVolumeNegativeTest-758771161-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:48:54Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='17f7007adf02474e9b019cd8af099ff8',uuid=0d4c65d2-c641-4985-8115-24dcbc15555f,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "c0c9db92-f7f2-4a6d-90d0-bfe48762cc6e", "address": "fa:16:3e:93:37:be", "network": {"id": "47c43230-a244-4198-84e4-9ed98c005cd9", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1392403106-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.134", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c07a6b254414b41a673468238d3aa62", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc0c9db92-f7", "ovs_interfaceid": "c0c9db92-f7f2-4a6d-90d0-bfe48762cc6e", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-194e37c6-de48-4dab-b559-4027cd7af715 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Converting VIF {"id": "c0c9db92-f7f2-4a6d-90d0-bfe48762cc6e", "address": "fa:16:3e:93:37:be", "network": {"id": "47c43230-a244-4198-84e4-9ed98c005cd9", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1392403106-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.134", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "8c07a6b254414b41a673468238d3aa62", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc0c9db92-f7", "ovs_interfaceid": "c0c9db92-f7f2-4a6d-90d0-bfe48762cc6e", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-194e37c6-de48-4dab-b559-4027cd7af715 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:93:37:be,bridge_name='br-int',has_traffic_filtering=True,id=c0c9db92-f7f2-4a6d-90d0-bfe48762cc6e,network=Network(47c43230-a244-4198-84e4-9ed98c005cd9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc0c9db92-f7') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG os_vif [None req-194e37c6-de48-4dab-b559-4027cd7af715 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:93:37:be,bridge_name='br-int',has_traffic_filtering=True,id=c0c9db92-f7f2-4a6d-90d0-bfe48762cc6e,network=Network(47c43230-a244-4198-84e4-9ed98c005cd9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc0c9db92-f7') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapc0c9db92-f7, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:51:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:51:49 user nova-compute[72351]: INFO os_vif [None req-194e37c6-de48-4dab-b559-4027cd7af715 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:93:37:be,bridge_name='br-int',has_traffic_filtering=True,id=c0c9db92-f7f2-4a6d-90d0-bfe48762cc6e,network=Network(47c43230-a244-4198-84e4-9ed98c005cd9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc0c9db92-f7')
Aug 09 02:51:49 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-194e37c6-de48-4dab-b559-4027cd7af715 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Deleting instance files /opt/stack/data/nova/instances/0d4c65d2-c641-4985-8115-24dcbc15555f_del
Aug 09 02:51:49 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-194e37c6-de48-4dab-b559-4027cd7af715 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Deletion of /opt/stack/data/nova/instances/0d4c65d2-c641-4985-8115-24dcbc15555f_del complete
Aug 09 02:51:50 user nova-compute[72351]: INFO nova.compute.manager [None req-194e37c6-de48-4dab-b559-4027cd7af715 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Took 0.90 seconds to destroy the instance on the hypervisor.
Aug 09 02:51:50 user nova-compute[72351]: DEBUG oslo.service.loopingcall [None req-194e37c6-de48-4dab-b559-4027cd7af715 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72351) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Aug 09 02:51:50 user nova-compute[72351]: DEBUG nova.compute.manager [-] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Deallocating network for instance {{(pid=72351) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2263}}
Aug 09 02:51:50 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] deallocate_for_instance() {{(pid=72351) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1795}}
Aug 09 02:51:50 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:51:51 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Took 0.96 seconds to deallocate network for instance.
Aug 09 02:51:51 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-194e37c6-de48-4dab-b559-4027cd7af715 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:51:51 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-194e37c6-de48-4dab-b559-4027cd7af715 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:51:51 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-194e37c6-de48-4dab-b559-4027cd7af715 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:51:51 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-194e37c6-de48-4dab-b559-4027cd7af715 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:51:51 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:51:51 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:51:51 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-194e37c6-de48-4dab-b559-4027cd7af715 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.182s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:51:51 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:51:51 user nova-compute[72351]: INFO nova.scheduler.client.report [None req-194e37c6-de48-4dab-b559-4027cd7af715 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Deleted allocations for instance 0d4c65d2-c641-4985-8115-24dcbc15555f
Aug 09 02:51:51 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-194e37c6-de48-4dab-b559-4027cd7af715 tempest-AttachVolumeNegativeTest-758771161 tempest-AttachVolumeNegativeTest-758771161-project-member] Lock "0d4c65d2-c641-4985-8115-24dcbc15555f" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.246s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:51:51 user nova-compute[72351]: DEBUG nova.compute.manager [req-b6fb51f9-1fe2-4835-8a74-88ea94005a9e req-f3ca5219-b0d4-4822-b01c-974f924b0b11 service nova] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Received event network-vif-plugged-2771d93d-fa0a-4e19-85ac-9d3f5df52782 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:51:51 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-b6fb51f9-1fe2-4835-8a74-88ea94005a9e req-f3ca5219-b0d4-4822-b01c-974f924b0b11 service nova] Acquiring lock "c6ae885f-dce2-4dc2-98d5-26139b8fb67e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:51:51 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-b6fb51f9-1fe2-4835-8a74-88ea94005a9e req-f3ca5219-b0d4-4822-b01c-974f924b0b11 service nova] Lock "c6ae885f-dce2-4dc2-98d5-26139b8fb67e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:51:51 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-b6fb51f9-1fe2-4835-8a74-88ea94005a9e req-f3ca5219-b0d4-4822-b01c-974f924b0b11 service nova] Lock "c6ae885f-dce2-4dc2-98d5-26139b8fb67e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:51:51 user nova-compute[72351]: DEBUG nova.compute.manager [req-b6fb51f9-1fe2-4835-8a74-88ea94005a9e req-f3ca5219-b0d4-4822-b01c-974f924b0b11 service nova] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] No waiting events found dispatching network-vif-plugged-2771d93d-fa0a-4e19-85ac-9d3f5df52782 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:51:51 user nova-compute[72351]: WARNING nova.compute.manager [req-b6fb51f9-1fe2-4835-8a74-88ea94005a9e req-f3ca5219-b0d4-4822-b01c-974f924b0b11 service nova] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Received unexpected event network-vif-plugged-2771d93d-fa0a-4e19-85ac-9d3f5df52782 for instance with vm_state stopped and task_state powering-on.
Aug 09 02:51:51 user nova-compute[72351]: DEBUG nova.compute.manager [req-efcb71d0-b50d-4a05-93d7-f9b8e7867b9e req-7d0b13f4-1115-4709-8b91-95adcbcf138e service nova] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Received event network-vif-plugged-c0c9db92-f7f2-4a6d-90d0-bfe48762cc6e {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:51:51 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-efcb71d0-b50d-4a05-93d7-f9b8e7867b9e req-7d0b13f4-1115-4709-8b91-95adcbcf138e service nova] Acquiring lock "0d4c65d2-c641-4985-8115-24dcbc15555f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:51:51 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-efcb71d0-b50d-4a05-93d7-f9b8e7867b9e req-7d0b13f4-1115-4709-8b91-95adcbcf138e service nova] Lock "0d4c65d2-c641-4985-8115-24dcbc15555f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:51:51 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-efcb71d0-b50d-4a05-93d7-f9b8e7867b9e req-7d0b13f4-1115-4709-8b91-95adcbcf138e service nova] Lock "0d4c65d2-c641-4985-8115-24dcbc15555f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:51:51 user nova-compute[72351]: DEBUG nova.compute.manager [req-efcb71d0-b50d-4a05-93d7-f9b8e7867b9e req-7d0b13f4-1115-4709-8b91-95adcbcf138e service nova] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] No waiting events found dispatching network-vif-plugged-c0c9db92-f7f2-4a6d-90d0-bfe48762cc6e {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:51:51 user nova-compute[72351]: WARNING nova.compute.manager [req-efcb71d0-b50d-4a05-93d7-f9b8e7867b9e req-7d0b13f4-1115-4709-8b91-95adcbcf138e service nova] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Received unexpected event network-vif-plugged-c0c9db92-f7f2-4a6d-90d0-bfe48762cc6e for instance with vm_state deleted and task_state None.
Aug 09 02:51:51 user nova-compute[72351]: DEBUG nova.compute.manager [req-efcb71d0-b50d-4a05-93d7-f9b8e7867b9e req-7d0b13f4-1115-4709-8b91-95adcbcf138e service nova] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Received event network-vif-deleted-c0c9db92-f7f2-4a6d-90d0-bfe48762cc6e {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:51:51 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:51:51 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:51:51 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:51:53 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691549513.3891172, c6ae885f-dce2-4dc2-98d5-26139b8fb67e => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:51:53 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] VM Resumed (Lifecycle Event)
Aug 09 02:51:53 user nova-compute[72351]: DEBUG nova.compute.manager [None req-d6cf0daf-4c20-4be3-a1fb-4a828c11a40f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:51:53 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Instance rebooted successfully.
Aug 09 02:51:53 user nova-compute[72351]: DEBUG nova.compute.manager [None req-d6cf0daf-4c20-4be3-a1fb-4a828c11a40f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:51:53 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:51:53 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: stopped, current task_state: powering-on, current DB power_state: 4, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:51:53 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] During sync_power_state the instance has a pending task (powering-on). Skip.
Aug 09 02:51:53 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691549513.3929443, c6ae885f-dce2-4dc2-98d5-26139b8fb67e => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:51:53 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] VM Started (Lifecycle Event)
Aug 09 02:51:53 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:51:53 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Synchronizing instance power state after lifecycle event "Started"; current vm_state: stopped, current task_state: powering-on, current DB power_state: 4, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:51:53 user nova-compute[72351]: DEBUG nova.compute.manager [req-a25582ab-8030-42bf-a52a-1dbc86ace70f req-2e7112a9-8613-4552-83df-cd47e6d914b0 service nova] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Received event network-vif-plugged-2771d93d-fa0a-4e19-85ac-9d3f5df52782 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:51:53 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a25582ab-8030-42bf-a52a-1dbc86ace70f req-2e7112a9-8613-4552-83df-cd47e6d914b0 service nova] Acquiring lock "c6ae885f-dce2-4dc2-98d5-26139b8fb67e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:51:53 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a25582ab-8030-42bf-a52a-1dbc86ace70f req-2e7112a9-8613-4552-83df-cd47e6d914b0 service nova] Lock "c6ae885f-dce2-4dc2-98d5-26139b8fb67e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:51:53 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a25582ab-8030-42bf-a52a-1dbc86ace70f req-2e7112a9-8613-4552-83df-cd47e6d914b0 service nova] Lock "c6ae885f-dce2-4dc2-98d5-26139b8fb67e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:51:53 user nova-compute[72351]: DEBUG nova.compute.manager [req-a25582ab-8030-42bf-a52a-1dbc86ace70f req-2e7112a9-8613-4552-83df-cd47e6d914b0 service nova] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] No waiting events found dispatching network-vif-plugged-2771d93d-fa0a-4e19-85ac-9d3f5df52782 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:51:53 user nova-compute[72351]: WARNING nova.compute.manager [req-a25582ab-8030-42bf-a52a-1dbc86ace70f req-2e7112a9-8613-4552-83df-cd47e6d914b0 service nova] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Received unexpected event network-vif-plugged-2771d93d-fa0a-4e19-85ac-9d3f5df52782 for instance with vm_state active and task_state None.
Aug 09 02:51:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:51:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:51:59 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:51:59 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:51:59 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 02:51:59 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:51:59 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:51:59 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:52:04 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691549509.9387724, 0d4c65d2-c641-4985-8115-24dcbc15555f => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:52:04 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] VM Stopped (Lifecycle Event)
Aug 09 02:52:04 user nova-compute[72351]: DEBUG nova.compute.manager [None req-0b369772-bc17-4ffa-a4b0-2d8d7231f60d None None] [instance: 0d4c65d2-c641-4985-8115-24dcbc15555f] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:52:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:52:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:52:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:52:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:52:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:52:17 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:52:17 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72351) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10461}}
Aug 09 02:52:19 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:52:19 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:52:20 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:52:21 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:52:23 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:52:23 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Starting heal instance info cache {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9842}}
Aug 09 02:52:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:52:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "refresh_cache-150f1b69-aa23-4702-a9f4-24025963befe" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:52:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquired lock "refresh_cache-150f1b69-aa23-4702-a9f4-24025963befe" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:52:24 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Forcefully refreshing network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1996}}
Aug 09 02:52:24 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Updating instance_info_cache with network_info: [{"id": "0983efc8-e53e-40bd-a721-2924155c20f3", "address": "fa:16:3e:d7:24:ef", "network": {"id": "da9be96c-ce20-4037-999c-230ca799fb06", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-661780305-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.240", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5ff300d27c9a4dd3b7ecd507b3627d98", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0983efc8-e5", "ovs_interfaceid": "0983efc8-e53e-40bd-a721-2924155c20f3", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:52:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Releasing lock "refresh_cache-150f1b69-aa23-4702-a9f4-24025963befe" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:52:24 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Updated the network info_cache for instance {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9913}}
Aug 09 02:52:24 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:52:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:52:26 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:52:27 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:52:27 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:52:27 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:52:27 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:52:27 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:52:27 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Auditing locally available compute resources for user (node: user) {{(pid=72351) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Aug 09 02:52:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:52:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk --force-share --output=json" returned: 0 in 0.149s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:52:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:52:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk --force-share --output=json" returned: 0 in 0.147s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:52:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/150f1b69-aa23-4702-a9f4-24025963befe/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:52:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/150f1b69-aa23-4702-a9f4-24025963befe/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:52:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/150f1b69-aa23-4702-a9f4-24025963befe/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:52:28 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/150f1b69-aa23-4702-a9f4-24025963befe/disk --force-share --output=json" returned: 0 in 0.147s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:52:29 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:52:29 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:52:29 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Hypervisor/Node resource view: name=user free_ram=8634MB free_disk=25.19614028930664GB free_vcpus=10 pci_devices=[{"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72351) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Aug 09 02:52:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:52:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:52:29 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance c6ae885f-dce2-4dc2-98d5-26139b8fb67e actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:52:29 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 150f1b69-aa23-4702-a9f4-24025963befe actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:52:29 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Total usable vcpus: 12, total allocated vcpus: 2 {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Aug 09 02:52:29 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Final resource view: name=user phys_ram=16011MB used_ram=1536MB phys_disk=40GB used_disk=2GB total_vcpus=12 used_vcpus=2 pci_stats=[] {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Aug 09 02:52:29 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:52:29 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:52:29 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Compute_service record updated for user:user {{(pid=72351) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Aug 09 02:52:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.248s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:52:29 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:52:33 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:52:34 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:52:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:52:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:52:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:52:46 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._run_image_cache_manager_pass {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:52:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "storage-registry-lock" by "nova.virt.storage_users.register_storage_use.<locals>.do_register_storage_use" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:52:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "storage-registry-lock" acquired by "nova.virt.storage_users.register_storage_use.<locals>.do_register_storage_use" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:52:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "storage-registry-lock" "released" by "nova.virt.storage_users.register_storage_use.<locals>.do_register_storage_use" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:52:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "storage-registry-lock" by "nova.virt.storage_users.get_storage_users.<locals>.do_get_storage_users" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:52:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "storage-registry-lock" acquired by "nova.virt.storage_users.get_storage_users.<locals>.do_get_storage_users" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:52:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "storage-registry-lock" "released" by "nova.virt.storage_users.get_storage_users.<locals>.do_get_storage_users" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:52:46 user nova-compute[72351]: DEBUG nova.virt.libvirt.imagecache [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Verify base images {{(pid=72351) _age_and_verify_cached_images /opt/stack/nova/nova/virt/libvirt/imagecache.py:314}}
Aug 09 02:52:46 user nova-compute[72351]: DEBUG nova.virt.libvirt.imagecache [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Image id 18ccda7f-31d3-4978-ab9d-4ed2f74eff94 yields fingerprint f54924f2669dce52d08d283dcdc864ffcb0b07cd {{(pid=72351) _age_and_verify_cached_images /opt/stack/nova/nova/virt/libvirt/imagecache.py:319}}
Aug 09 02:52:46 user nova-compute[72351]: INFO nova.virt.libvirt.imagecache [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] image 18ccda7f-31d3-4978-ab9d-4ed2f74eff94 at (/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd): checking
Aug 09 02:52:46 user nova-compute[72351]: DEBUG nova.virt.libvirt.imagecache [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] image 18ccda7f-31d3-4978-ab9d-4ed2f74eff94 at (/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd): image is in use {{(pid=72351) _mark_in_use /opt/stack/nova/nova/virt/libvirt/imagecache.py:279}}
Aug 09 02:52:46 user nova-compute[72351]: DEBUG nova.virt.libvirt.imagecache [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Image id  yields fingerprint da39a3ee5e6b4b0d3255bfef95601890afd80709 {{(pid=72351) _age_and_verify_cached_images /opt/stack/nova/nova/virt/libvirt/imagecache.py:319}}
Aug 09 02:52:46 user nova-compute[72351]: DEBUG nova.virt.libvirt.imagecache [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] c6ae885f-dce2-4dc2-98d5-26139b8fb67e is a valid instance name {{(pid=72351) _list_backing_images /opt/stack/nova/nova/virt/libvirt/imagecache.py:126}}
Aug 09 02:52:46 user nova-compute[72351]: DEBUG nova.virt.libvirt.imagecache [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] c6ae885f-dce2-4dc2-98d5-26139b8fb67e has a disk file {{(pid=72351) _list_backing_images /opt/stack/nova/nova/virt/libvirt/imagecache.py:129}}
Aug 09 02:52:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:52:47 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk --force-share --output=json" returned: 0 in 0.162s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:52:47 user nova-compute[72351]: DEBUG nova.virt.libvirt.imagecache [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance c6ae885f-dce2-4dc2-98d5-26139b8fb67e is backed by f54924f2669dce52d08d283dcdc864ffcb0b07cd {{(pid=72351) _list_backing_images /opt/stack/nova/nova/virt/libvirt/imagecache.py:141}}
Aug 09 02:52:47 user nova-compute[72351]: DEBUG nova.virt.libvirt.imagecache [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] 150f1b69-aa23-4702-a9f4-24025963befe is a valid instance name {{(pid=72351) _list_backing_images /opt/stack/nova/nova/virt/libvirt/imagecache.py:126}}
Aug 09 02:52:47 user nova-compute[72351]: DEBUG nova.virt.libvirt.imagecache [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] 150f1b69-aa23-4702-a9f4-24025963befe has a disk file {{(pid=72351) _list_backing_images /opt/stack/nova/nova/virt/libvirt/imagecache.py:129}}
Aug 09 02:52:47 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/150f1b69-aa23-4702-a9f4-24025963befe/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:52:47 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/150f1b69-aa23-4702-a9f4-24025963befe/disk --force-share --output=json" returned: 0 in 0.153s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:52:47 user nova-compute[72351]: DEBUG nova.virt.libvirt.imagecache [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 150f1b69-aa23-4702-a9f4-24025963befe is backed by f54924f2669dce52d08d283dcdc864ffcb0b07cd {{(pid=72351) _list_backing_images /opt/stack/nova/nova/virt/libvirt/imagecache.py:141}}
Aug 09 02:52:47 user nova-compute[72351]: WARNING nova.virt.libvirt.imagecache [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Unknown base file: /opt/stack/data/nova/instances/_base/ccad13889b377d9d232f2cde88dc197c0daa4f29
Aug 09 02:52:47 user nova-compute[72351]: WARNING nova.virt.libvirt.imagecache [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Unknown base file: /opt/stack/data/nova/instances/_base/d13d16d0ab57e27681822ca11b323960a692a169
Aug 09 02:52:47 user nova-compute[72351]: WARNING nova.virt.libvirt.imagecache [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Unknown base file: /opt/stack/data/nova/instances/_base/6f51e218d942a068c64b64903ff80f9c3e6dee79
Aug 09 02:52:47 user nova-compute[72351]: WARNING nova.virt.libvirt.imagecache [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Unknown base file: /opt/stack/data/nova/instances/_base/78680203fb06db2bb9e4fc0abad2df7c1c424054
Aug 09 02:52:47 user nova-compute[72351]: WARNING nova.virt.libvirt.imagecache [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Unknown base file: /opt/stack/data/nova/instances/_base/ad7b4169e2e08fd64509e88d2ad0f00130ea9294
Aug 09 02:52:47 user nova-compute[72351]: WARNING nova.virt.libvirt.imagecache [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Unknown base file: /opt/stack/data/nova/instances/_base/2d18b85f2b82a300c273485804088c44d5324c8d
Aug 09 02:52:47 user nova-compute[72351]: WARNING nova.virt.libvirt.imagecache [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Unknown base file: /opt/stack/data/nova/instances/_base/aac9bf03ba600a05bb056708c9995070051b324c
Aug 09 02:52:47 user nova-compute[72351]: WARNING nova.virt.libvirt.imagecache [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Unknown base file: /opt/stack/data/nova/instances/_base/90e420c550f5519f016964725db870eb284e3d12
Aug 09 02:52:47 user nova-compute[72351]: WARNING nova.virt.libvirt.imagecache [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Unknown base file: /opt/stack/data/nova/instances/_base/85c4e52d495f393d9b5d4e4fdc36691ebb7ca1a2
Aug 09 02:52:47 user nova-compute[72351]: INFO nova.virt.libvirt.imagecache [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Active base files: /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd
Aug 09 02:52:47 user nova-compute[72351]: INFO nova.virt.libvirt.imagecache [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Removable base files: /opt/stack/data/nova/instances/_base/ccad13889b377d9d232f2cde88dc197c0daa4f29 /opt/stack/data/nova/instances/_base/d13d16d0ab57e27681822ca11b323960a692a169 /opt/stack/data/nova/instances/_base/6f51e218d942a068c64b64903ff80f9c3e6dee79 /opt/stack/data/nova/instances/_base/78680203fb06db2bb9e4fc0abad2df7c1c424054 /opt/stack/data/nova/instances/_base/ad7b4169e2e08fd64509e88d2ad0f00130ea9294 /opt/stack/data/nova/instances/_base/2d18b85f2b82a300c273485804088c44d5324c8d /opt/stack/data/nova/instances/_base/aac9bf03ba600a05bb056708c9995070051b324c /opt/stack/data/nova/instances/_base/90e420c550f5519f016964725db870eb284e3d12 /opt/stack/data/nova/instances/_base/85c4e52d495f393d9b5d4e4fdc36691ebb7ca1a2
Aug 09 02:52:47 user nova-compute[72351]: INFO nova.virt.libvirt.imagecache [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Base, swap or ephemeral file too young to remove: /opt/stack/data/nova/instances/_base/ccad13889b377d9d232f2cde88dc197c0daa4f29
Aug 09 02:52:47 user nova-compute[72351]: INFO nova.virt.libvirt.imagecache [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Base, swap or ephemeral file too young to remove: /opt/stack/data/nova/instances/_base/d13d16d0ab57e27681822ca11b323960a692a169
Aug 09 02:52:47 user nova-compute[72351]: INFO nova.virt.libvirt.imagecache [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Base, swap or ephemeral file too young to remove: /opt/stack/data/nova/instances/_base/6f51e218d942a068c64b64903ff80f9c3e6dee79
Aug 09 02:52:47 user nova-compute[72351]: INFO nova.virt.libvirt.imagecache [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Base, swap or ephemeral file too young to remove: /opt/stack/data/nova/instances/_base/78680203fb06db2bb9e4fc0abad2df7c1c424054
Aug 09 02:52:47 user nova-compute[72351]: INFO nova.virt.libvirt.imagecache [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Base, swap or ephemeral file too young to remove: /opt/stack/data/nova/instances/_base/ad7b4169e2e08fd64509e88d2ad0f00130ea9294
Aug 09 02:52:47 user nova-compute[72351]: INFO nova.virt.libvirt.imagecache [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Base, swap or ephemeral file too young to remove: /opt/stack/data/nova/instances/_base/2d18b85f2b82a300c273485804088c44d5324c8d
Aug 09 02:52:47 user nova-compute[72351]: INFO nova.virt.libvirt.imagecache [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Base, swap or ephemeral file too young to remove: /opt/stack/data/nova/instances/_base/aac9bf03ba600a05bb056708c9995070051b324c
Aug 09 02:52:47 user nova-compute[72351]: INFO nova.virt.libvirt.imagecache [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Base, swap or ephemeral file too young to remove: /opt/stack/data/nova/instances/_base/90e420c550f5519f016964725db870eb284e3d12
Aug 09 02:52:47 user nova-compute[72351]: INFO nova.virt.libvirt.imagecache [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Base, swap or ephemeral file too young to remove: /opt/stack/data/nova/instances/_base/85c4e52d495f393d9b5d4e4fdc36691ebb7ca1a2
Aug 09 02:52:47 user nova-compute[72351]: DEBUG nova.virt.libvirt.imagecache [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Verification complete {{(pid=72351) _age_and_verify_cached_images /opt/stack/nova/nova/virt/libvirt/imagecache.py:350}}
Aug 09 02:52:47 user nova-compute[72351]: DEBUG nova.virt.libvirt.imagecache [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Verify swap images {{(pid=72351) _age_and_verify_swap_images /opt/stack/nova/nova/virt/libvirt/imagecache.py:299}}
Aug 09 02:52:47 user nova-compute[72351]: DEBUG nova.virt.libvirt.imagecache [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Verify ephemeral images {{(pid=72351) _age_and_verify_ephemeral_images /opt/stack/nova/nova/virt/libvirt/imagecache.py:284}}
Aug 09 02:52:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:52:53 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-8b4f083a-648b-4238-9542-bf279ad01617 tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Acquiring lock "150f1b69-aa23-4702-a9f4-24025963befe" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:52:53 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-8b4f083a-648b-4238-9542-bf279ad01617 tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "150f1b69-aa23-4702-a9f4-24025963befe" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:52:53 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-8b4f083a-648b-4238-9542-bf279ad01617 tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Acquiring lock "150f1b69-aa23-4702-a9f4-24025963befe-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:52:53 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-8b4f083a-648b-4238-9542-bf279ad01617 tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "150f1b69-aa23-4702-a9f4-24025963befe-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:52:53 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-8b4f083a-648b-4238-9542-bf279ad01617 tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "150f1b69-aa23-4702-a9f4-24025963befe-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:52:53 user nova-compute[72351]: INFO nova.compute.manager [None req-8b4f083a-648b-4238-9542-bf279ad01617 tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Terminating instance
Aug 09 02:52:53 user nova-compute[72351]: DEBUG nova.compute.manager [None req-8b4f083a-648b-4238-9542-bf279ad01617 tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Start destroying the instance on the hypervisor. {{(pid=72351) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3124}}
Aug 09 02:52:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:52:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:52:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:52:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:52:53 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Instance destroyed successfully.
Aug 09 02:52:53 user nova-compute[72351]: DEBUG nova.objects.instance [None req-8b4f083a-648b-4238-9542-bf279ad01617 tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lazy-loading 'resources' on Instance uuid 150f1b69-aa23-4702-a9f4-24025963befe {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:52:53 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-8b4f083a-648b-4238-9542-bf279ad01617 tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,compute_id=1,config_drive='',created_at=2023-08-09T02:47:13Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-server-547567417',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumeshelvetestjson-server-547567417',id=49,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBC6Kt1VBrwPlXoj775IMgThdXYe/hd3G1m2PhSYFaCtMLpu/4nO1MCXo8m5UYekLtMKzV6viB1JRAwTbgfITOhrGcZO/1RTBqsi/z0vUwAuglB5045WiXDwqXuvJuthb5A==',key_name='tempest-keypair-647885492',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:51:12Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='5ff300d27c9a4dd3b7ecd507b3627d98',ramdisk_id='',reservation_id='r-x20xmo7b',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeShelveTestJSON-1982056968',owner_user_name='tempest-AttachVolumeShelveTestJSON-1982056968-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:51:12Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='ad8fc03436a84bdfb13ba6abecd4f27b',uuid=150f1b69-aa23-4702-a9f4-24025963befe,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "0983efc8-e53e-40bd-a721-2924155c20f3", "address": "fa:16:3e:d7:24:ef", "network": {"id": "da9be96c-ce20-4037-999c-230ca799fb06", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-661780305-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.240", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5ff300d27c9a4dd3b7ecd507b3627d98", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0983efc8-e5", "ovs_interfaceid": "0983efc8-e53e-40bd-a721-2924155c20f3", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:52:53 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-8b4f083a-648b-4238-9542-bf279ad01617 tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Converting VIF {"id": "0983efc8-e53e-40bd-a721-2924155c20f3", "address": "fa:16:3e:d7:24:ef", "network": {"id": "da9be96c-ce20-4037-999c-230ca799fb06", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-661780305-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.240", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5ff300d27c9a4dd3b7ecd507b3627d98", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0983efc8-e5", "ovs_interfaceid": "0983efc8-e53e-40bd-a721-2924155c20f3", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:52:53 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-8b4f083a-648b-4238-9542-bf279ad01617 tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:d7:24:ef,bridge_name='br-int',has_traffic_filtering=True,id=0983efc8-e53e-40bd-a721-2924155c20f3,network=Network(da9be96c-ce20-4037-999c-230ca799fb06),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap0983efc8-e5') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:52:53 user nova-compute[72351]: DEBUG os_vif [None req-8b4f083a-648b-4238-9542-bf279ad01617 tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:d7:24:ef,bridge_name='br-int',has_traffic_filtering=True,id=0983efc8-e53e-40bd-a721-2924155c20f3,network=Network(da9be96c-ce20-4037-999c-230ca799fb06),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap0983efc8-e5') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:52:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:52:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap0983efc8-e5, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:52:53 user nova-compute[72351]: DEBUG nova.compute.manager [req-0df8e473-01ba-41fb-a91f-80a0cde6c1c4 req-bc018902-fb4b-4325-8cc6-52c3cb452d85 service nova] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Received event network-vif-unplugged-0983efc8-e53e-40bd-a721-2924155c20f3 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:52:53 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-0df8e473-01ba-41fb-a91f-80a0cde6c1c4 req-bc018902-fb4b-4325-8cc6-52c3cb452d85 service nova] Acquiring lock "150f1b69-aa23-4702-a9f4-24025963befe-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:52:53 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-0df8e473-01ba-41fb-a91f-80a0cde6c1c4 req-bc018902-fb4b-4325-8cc6-52c3cb452d85 service nova] Lock "150f1b69-aa23-4702-a9f4-24025963befe-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:52:53 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-0df8e473-01ba-41fb-a91f-80a0cde6c1c4 req-bc018902-fb4b-4325-8cc6-52c3cb452d85 service nova] Lock "150f1b69-aa23-4702-a9f4-24025963befe-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:52:53 user nova-compute[72351]: DEBUG nova.compute.manager [req-0df8e473-01ba-41fb-a91f-80a0cde6c1c4 req-bc018902-fb4b-4325-8cc6-52c3cb452d85 service nova] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] No waiting events found dispatching network-vif-unplugged-0983efc8-e53e-40bd-a721-2924155c20f3 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:52:53 user nova-compute[72351]: DEBUG nova.compute.manager [req-0df8e473-01ba-41fb-a91f-80a0cde6c1c4 req-bc018902-fb4b-4325-8cc6-52c3cb452d85 service nova] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Received event network-vif-unplugged-0983efc8-e53e-40bd-a721-2924155c20f3 for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:52:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:52:53 user nova-compute[72351]: INFO os_vif [None req-8b4f083a-648b-4238-9542-bf279ad01617 tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:d7:24:ef,bridge_name='br-int',has_traffic_filtering=True,id=0983efc8-e53e-40bd-a721-2924155c20f3,network=Network(da9be96c-ce20-4037-999c-230ca799fb06),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap0983efc8-e5')
Aug 09 02:52:53 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-8b4f083a-648b-4238-9542-bf279ad01617 tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Deleting instance files /opt/stack/data/nova/instances/150f1b69-aa23-4702-a9f4-24025963befe_del
Aug 09 02:52:53 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-8b4f083a-648b-4238-9542-bf279ad01617 tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Deletion of /opt/stack/data/nova/instances/150f1b69-aa23-4702-a9f4-24025963befe_del complete
Aug 09 02:52:53 user nova-compute[72351]: INFO nova.compute.manager [None req-8b4f083a-648b-4238-9542-bf279ad01617 tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Took 0.80 seconds to destroy the instance on the hypervisor.
Aug 09 02:52:53 user nova-compute[72351]: DEBUG oslo.service.loopingcall [None req-8b4f083a-648b-4238-9542-bf279ad01617 tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72351) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Aug 09 02:52:53 user nova-compute[72351]: DEBUG nova.compute.manager [-] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Deallocating network for instance {{(pid=72351) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2263}}
Aug 09 02:52:53 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] deallocate_for_instance() {{(pid=72351) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1795}}
Aug 09 02:52:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:52:54 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:52:54 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Took 0.87 seconds to deallocate network for instance.
Aug 09 02:52:54 user nova-compute[72351]: DEBUG nova.compute.manager [req-8b0f4fa6-4710-470e-819c-a60c916457f1 req-609273ad-e6cc-4649-8c41-fc13c5192931 service nova] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Received event network-vif-deleted-0983efc8-e53e-40bd-a721-2924155c20f3 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:52:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-8b4f083a-648b-4238-9542-bf279ad01617 tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:52:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-8b4f083a-648b-4238-9542-bf279ad01617 tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:52:54 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-8b4f083a-648b-4238-9542-bf279ad01617 tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:52:54 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-8b4f083a-648b-4238-9542-bf279ad01617 tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:52:54 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-8b4f083a-648b-4238-9542-bf279ad01617 tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.134s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:52:54 user nova-compute[72351]: INFO nova.scheduler.client.report [None req-8b4f083a-648b-4238-9542-bf279ad01617 tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Deleted allocations for instance 150f1b69-aa23-4702-a9f4-24025963befe
Aug 09 02:52:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-8b4f083a-648b-4238-9542-bf279ad01617 tempest-AttachVolumeShelveTestJSON-1982056968 tempest-AttachVolumeShelveTestJSON-1982056968-project-member] Lock "150f1b69-aa23-4702-a9f4-24025963befe" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 1.991s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:52:55 user nova-compute[72351]: DEBUG nova.compute.manager [req-bc49f417-c3e8-401c-9ce7-06bcd96c0061 req-f24176b4-3688-4bcc-9981-abbd9173321f service nova] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Received event network-vif-plugged-0983efc8-e53e-40bd-a721-2924155c20f3 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:52:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-bc49f417-c3e8-401c-9ce7-06bcd96c0061 req-f24176b4-3688-4bcc-9981-abbd9173321f service nova] Acquiring lock "150f1b69-aa23-4702-a9f4-24025963befe-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:52:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-bc49f417-c3e8-401c-9ce7-06bcd96c0061 req-f24176b4-3688-4bcc-9981-abbd9173321f service nova] Lock "150f1b69-aa23-4702-a9f4-24025963befe-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:52:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-bc49f417-c3e8-401c-9ce7-06bcd96c0061 req-f24176b4-3688-4bcc-9981-abbd9173321f service nova] Lock "150f1b69-aa23-4702-a9f4-24025963befe-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:52:55 user nova-compute[72351]: DEBUG nova.compute.manager [req-bc49f417-c3e8-401c-9ce7-06bcd96c0061 req-f24176b4-3688-4bcc-9981-abbd9173321f service nova] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] No waiting events found dispatching network-vif-plugged-0983efc8-e53e-40bd-a721-2924155c20f3 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:52:55 user nova-compute[72351]: WARNING nova.compute.manager [req-bc49f417-c3e8-401c-9ce7-06bcd96c0061 req-f24176b4-3688-4bcc-9981-abbd9173321f service nova] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Received unexpected event network-vif-plugged-0983efc8-e53e-40bd-a721-2924155c20f3 for instance with vm_state deleted and task_state None.
Aug 09 02:52:58 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:52:59 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:53:03 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:53:08 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691549573.686501, 150f1b69-aa23-4702-a9f4-24025963befe => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:53:08 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] VM Stopped (Lifecycle Event)
Aug 09 02:53:08 user nova-compute[72351]: DEBUG nova.compute.manager [None req-3ae6390d-8bf8-45ca-b8b0-eb8fd40a03e1 None None] [instance: 150f1b69-aa23-4702-a9f4-24025963befe] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:53:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:53:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:53:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 02:53:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:53:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:53:08 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:53:13 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:53:18 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:53:19 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:53:19 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72351) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10461}}
Aug 09 02:53:20 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:53:23 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:53:23 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:53:23 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Starting heal instance info cache {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9842}}
Aug 09 02:53:23 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Didn't find any instances for network info cache update. {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9928}}
Aug 09 02:53:23 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:53:26 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:53:27 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:53:27 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:53:28 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:53:28 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:53:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:53:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:53:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:53:28 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Auditing locally available compute resources for user (node: user) {{(pid=72351) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Aug 09 02:53:29 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:53:29 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:53:29 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:53:29 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e/disk --force-share --output=json" returned: 0 in 0.132s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:53:29 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:53:29 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:53:29 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Hypervisor/Node resource view: name=user free_ram=8704MB free_disk=25.21459197998047GB free_vcpus=11 pci_devices=[{"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72351) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Aug 09 02:53:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:53:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:53:29 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance c6ae885f-dce2-4dc2-98d5-26139b8fb67e actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:53:29 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Total usable vcpus: 12, total allocated vcpus: 1 {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Aug 09 02:53:29 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Final resource view: name=user phys_ram=16011MB used_ram=1024MB phys_disk=40GB used_disk=1GB total_vcpus=12 used_vcpus=1 pci_stats=[] {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Aug 09 02:53:29 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:53:29 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:53:29 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Compute_service record updated for user:user {{(pid=72351) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Aug 09 02:53:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.185s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:53:32 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:53:33 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:53:33 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:53:35 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._cleanup_expired_console_auth_tokens {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:53:36 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._run_pending_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:53:36 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Cleaning up deleted instances {{(pid=72351) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11129}}
Aug 09 02:53:36 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] There are 0 instances to clean {{(pid=72351) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11138}}
Aug 09 02:53:38 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:53:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:53:39 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-9c891da4-3649-4129-a88b-2a6d65f11a5c tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Acquiring lock "c6ae885f-dce2-4dc2-98d5-26139b8fb67e" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:53:39 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-9c891da4-3649-4129-a88b-2a6d65f11a5c tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "c6ae885f-dce2-4dc2-98d5-26139b8fb67e" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:53:39 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-9c891da4-3649-4129-a88b-2a6d65f11a5c tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Acquiring lock "c6ae885f-dce2-4dc2-98d5-26139b8fb67e-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:53:39 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-9c891da4-3649-4129-a88b-2a6d65f11a5c tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "c6ae885f-dce2-4dc2-98d5-26139b8fb67e-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:53:39 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-9c891da4-3649-4129-a88b-2a6d65f11a5c tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "c6ae885f-dce2-4dc2-98d5-26139b8fb67e-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:53:39 user nova-compute[72351]: INFO nova.compute.manager [None req-9c891da4-3649-4129-a88b-2a6d65f11a5c tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Terminating instance
Aug 09 02:53:39 user nova-compute[72351]: DEBUG nova.compute.manager [None req-9c891da4-3649-4129-a88b-2a6d65f11a5c tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Start destroying the instance on the hypervisor. {{(pid=72351) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3124}}
Aug 09 02:53:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:53:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:53:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:53:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:53:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:53:39 user nova-compute[72351]: DEBUG nova.compute.manager [req-5536d3b9-299a-421f-8ee6-e791a75195e1 req-a1075b86-02f9-49fd-baa6-e1153e385893 service nova] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Received event network-vif-unplugged-2771d93d-fa0a-4e19-85ac-9d3f5df52782 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:53:39 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-5536d3b9-299a-421f-8ee6-e791a75195e1 req-a1075b86-02f9-49fd-baa6-e1153e385893 service nova] Acquiring lock "c6ae885f-dce2-4dc2-98d5-26139b8fb67e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:53:39 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-5536d3b9-299a-421f-8ee6-e791a75195e1 req-a1075b86-02f9-49fd-baa6-e1153e385893 service nova] Lock "c6ae885f-dce2-4dc2-98d5-26139b8fb67e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:53:39 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-5536d3b9-299a-421f-8ee6-e791a75195e1 req-a1075b86-02f9-49fd-baa6-e1153e385893 service nova] Lock "c6ae885f-dce2-4dc2-98d5-26139b8fb67e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:53:39 user nova-compute[72351]: DEBUG nova.compute.manager [req-5536d3b9-299a-421f-8ee6-e791a75195e1 req-a1075b86-02f9-49fd-baa6-e1153e385893 service nova] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] No waiting events found dispatching network-vif-unplugged-2771d93d-fa0a-4e19-85ac-9d3f5df52782 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:53:39 user nova-compute[72351]: DEBUG nova.compute.manager [req-5536d3b9-299a-421f-8ee6-e791a75195e1 req-a1075b86-02f9-49fd-baa6-e1153e385893 service nova] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Received event network-vif-unplugged-2771d93d-fa0a-4e19-85ac-9d3f5df52782 for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:53:40 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Instance destroyed successfully.
Aug 09 02:53:40 user nova-compute[72351]: DEBUG nova.objects.instance [None req-9c891da4-3649-4129-a88b-2a6d65f11a5c tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lazy-loading 'resources' on Instance uuid c6ae885f-dce2-4dc2-98d5-26139b8fb67e {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:53:40 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-9c891da4-3649-4129-a88b-2a6d65f11a5c tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:44:08Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachVolumeTestJSON-server-218910074',display_name='tempest-AttachVolumeTestJSON-server-218910074',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumetestjson-server-218910074',id=47,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBIplDVAhYRmCgaFwiRMKOJ3OlhP3n+pFAzSDpMz7BP4WftIaiuBT1r87OWLI3l8JRBwh8sp/09wt0bUhVeTDk2tWONkdagBfHB77ctUB1RkuePwrxnVW0j07TkHZdA5xrw==',key_name='tempest-keypair-304755592',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:44:14Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='ec9861e0882e4065ab99ad3f74d6b25f',ramdisk_id='',reservation_id='r-2vl0h2f4',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeTestJSON-1725368857',owner_user_name='tempest-AttachVolumeTestJSON-1725368857-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:51:53Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='a2bc49dd654f4beca4b8eb5a49cd350d',uuid=c6ae885f-dce2-4dc2-98d5-26139b8fb67e,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "address": "fa:16:3e:dd:49:4d", "network": {"id": "dba57c3d-6efd-4057-8053-1c6a92bb24f8", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-568585773-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.116", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ec9861e0882e4065ab99ad3f74d6b25f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2771d93d-fa", "ovs_interfaceid": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:53:40 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-9c891da4-3649-4129-a88b-2a6d65f11a5c tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Converting VIF {"id": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "address": "fa:16:3e:dd:49:4d", "network": {"id": "dba57c3d-6efd-4057-8053-1c6a92bb24f8", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-568585773-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.116", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ec9861e0882e4065ab99ad3f74d6b25f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2771d93d-fa", "ovs_interfaceid": "2771d93d-fa0a-4e19-85ac-9d3f5df52782", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:53:40 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-9c891da4-3649-4129-a88b-2a6d65f11a5c tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:dd:49:4d,bridge_name='br-int',has_traffic_filtering=True,id=2771d93d-fa0a-4e19-85ac-9d3f5df52782,network=Network(dba57c3d-6efd-4057-8053-1c6a92bb24f8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2771d93d-fa') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:53:40 user nova-compute[72351]: DEBUG os_vif [None req-9c891da4-3649-4129-a88b-2a6d65f11a5c tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:dd:49:4d,bridge_name='br-int',has_traffic_filtering=True,id=2771d93d-fa0a-4e19-85ac-9d3f5df52782,network=Network(dba57c3d-6efd-4057-8053-1c6a92bb24f8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2771d93d-fa') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:53:40 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:53:40 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap2771d93d-fa, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:53:40 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:53:40 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:53:40 user nova-compute[72351]: INFO os_vif [None req-9c891da4-3649-4129-a88b-2a6d65f11a5c tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:dd:49:4d,bridge_name='br-int',has_traffic_filtering=True,id=2771d93d-fa0a-4e19-85ac-9d3f5df52782,network=Network(dba57c3d-6efd-4057-8053-1c6a92bb24f8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2771d93d-fa')
Aug 09 02:53:40 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-9c891da4-3649-4129-a88b-2a6d65f11a5c tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Deleting instance files /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e_del
Aug 09 02:53:40 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-9c891da4-3649-4129-a88b-2a6d65f11a5c tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Deletion of /opt/stack/data/nova/instances/c6ae885f-dce2-4dc2-98d5-26139b8fb67e_del complete
Aug 09 02:53:40 user nova-compute[72351]: INFO nova.compute.manager [None req-9c891da4-3649-4129-a88b-2a6d65f11a5c tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Took 0.85 seconds to destroy the instance on the hypervisor.
Aug 09 02:53:40 user nova-compute[72351]: DEBUG oslo.service.loopingcall [None req-9c891da4-3649-4129-a88b-2a6d65f11a5c tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72351) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Aug 09 02:53:40 user nova-compute[72351]: DEBUG nova.compute.manager [-] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Deallocating network for instance {{(pid=72351) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2263}}
Aug 09 02:53:40 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] deallocate_for_instance() {{(pid=72351) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1795}}
Aug 09 02:53:41 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:53:41 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:53:41 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Took 0.93 seconds to deallocate network for instance.
Aug 09 02:53:41 user nova-compute[72351]: DEBUG nova.compute.manager [req-8654383f-1f95-4f9e-945d-de3fa9a40a66 req-6e380596-e146-43d7-9f2b-12bb3da92929 service nova] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Received event network-vif-deleted-2771d93d-fa0a-4e19-85ac-9d3f5df52782 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:53:41 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-9c891da4-3649-4129-a88b-2a6d65f11a5c tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:53:41 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-9c891da4-3649-4129-a88b-2a6d65f11a5c tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:53:41 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-9c891da4-3649-4129-a88b-2a6d65f11a5c tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:53:41 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-9c891da4-3649-4129-a88b-2a6d65f11a5c tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:53:41 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-9c891da4-3649-4129-a88b-2a6d65f11a5c tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.114s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:53:41 user nova-compute[72351]: INFO nova.scheduler.client.report [None req-9c891da4-3649-4129-a88b-2a6d65f11a5c tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Deleted allocations for instance c6ae885f-dce2-4dc2-98d5-26139b8fb67e
Aug 09 02:53:41 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-9c891da4-3649-4129-a88b-2a6d65f11a5c tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "c6ae885f-dce2-4dc2-98d5-26139b8fb67e" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.069s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:53:41 user nova-compute[72351]: DEBUG nova.compute.manager [req-cf05d097-048a-4389-aba1-9b2043c1fab0 req-5cdb065c-d4b9-4b41-a074-4b12455dd65e service nova] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Received event network-vif-plugged-2771d93d-fa0a-4e19-85ac-9d3f5df52782 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:53:41 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-cf05d097-048a-4389-aba1-9b2043c1fab0 req-5cdb065c-d4b9-4b41-a074-4b12455dd65e service nova] Acquiring lock "c6ae885f-dce2-4dc2-98d5-26139b8fb67e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:53:41 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-cf05d097-048a-4389-aba1-9b2043c1fab0 req-5cdb065c-d4b9-4b41-a074-4b12455dd65e service nova] Lock "c6ae885f-dce2-4dc2-98d5-26139b8fb67e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:53:41 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-cf05d097-048a-4389-aba1-9b2043c1fab0 req-5cdb065c-d4b9-4b41-a074-4b12455dd65e service nova] Lock "c6ae885f-dce2-4dc2-98d5-26139b8fb67e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:53:41 user nova-compute[72351]: DEBUG nova.compute.manager [req-cf05d097-048a-4389-aba1-9b2043c1fab0 req-5cdb065c-d4b9-4b41-a074-4b12455dd65e service nova] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] No waiting events found dispatching network-vif-plugged-2771d93d-fa0a-4e19-85ac-9d3f5df52782 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:53:41 user nova-compute[72351]: WARNING nova.compute.manager [req-cf05d097-048a-4389-aba1-9b2043c1fab0 req-5cdb065c-d4b9-4b41-a074-4b12455dd65e service nova] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Received unexpected event network-vif-plugged-2771d93d-fa0a-4e19-85ac-9d3f5df52782 for instance with vm_state deleted and task_state None.
Aug 09 02:53:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:53:45 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:53:50 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:53:51 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._cleanup_incomplete_migrations {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:53:51 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Cleaning up deleted instances with incomplete migration  {{(pid=72351) _cleanup_incomplete_migrations /opt/stack/nova/nova/compute/manager.py:11167}}
Aug 09 02:53:55 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691549620.2523396, c6ae885f-dce2-4dc2-98d5-26139b8fb67e => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:53:55 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] VM Stopped (Lifecycle Event)
Aug 09 02:53:55 user nova-compute[72351]: DEBUG nova.compute.manager [None req-19842e9e-bc0d-4b2f-85b5-8e498163b6cd None None] [instance: c6ae885f-dce2-4dc2-98d5-26139b8fb67e] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:53:55 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:53:57 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:54:00 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:54:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:54:10 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:54:10 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._sync_power_states {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:54:15 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:54:19 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:54:19 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72351) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10461}}
Aug 09 02:54:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:54:20 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:54:23 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:54:23 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Starting heal instance info cache {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9842}}
Aug 09 02:54:23 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Rebuilding the list of instances to heal {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9846}}
Aug 09 02:54:23 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Didn't find any instances for network info cache update. {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9928}}
Aug 09 02:54:24 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:54:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:54:26 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:54:28 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:54:29 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:54:29 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:54:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:54:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:54:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:54:29 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Auditing locally available compute resources for user (node: user) {{(pid=72351) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Aug 09 02:54:30 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:54:30 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:54:30 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:54:30 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Hypervisor/Node resource view: name=user free_ram=8899MB free_disk=25.238941192626953GB free_vcpus=12 pci_devices=[{"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72351) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Aug 09 02:54:30 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:54:30 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:54:30 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Total usable vcpus: 12, total allocated vcpus: 0 {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Aug 09 02:54:30 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Final resource view: name=user phys_ram=16011MB used_ram=512MB phys_disk=40GB used_disk=0GB total_vcpus=12 used_vcpus=0 pci_stats=[] {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Aug 09 02:54:30 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Refreshing inventories for resource provider 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:804}}
Aug 09 02:54:30 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Updating ProviderTree inventory for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 from _refresh_and_get_inventory using data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) _refresh_and_get_inventory /opt/stack/nova/nova/scheduler/client/report.py:768}}
Aug 09 02:54:30 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Updating inventory in ProviderTree for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 with inventory: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:176}}
Aug 09 02:54:30 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Refreshing aggregate associations for resource provider 8eae9b37-1236-4e6d-91a5-b650de27b009, aggregates: None {{(pid=72351) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:813}}
Aug 09 02:54:30 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Refreshing trait associations for resource provider 8eae9b37-1236-4e6d-91a5-b650de27b009, traits: COMPUTE_VIOMMU_MODEL_INTEL,COMPUTE_STORAGE_BUS_IDE,COMPUTE_GRAPHICS_MODEL_NONE,COMPUTE_NET_VIF_MODEL_E1000E,COMPUTE_IMAGE_TYPE_AKI,COMPUTE_VOLUME_ATTACH_WITH_TAG,COMPUTE_NET_VIF_MODEL_VMXNET3,COMPUTE_SOCKET_PCI_NUMA_AFFINITY,COMPUTE_ACCELERATORS,HW_CPU_X86_SSE41,COMPUTE_NET_ATTACH_INTERFACE_WITH_TAG,COMPUTE_GRAPHICS_MODEL_VGA,COMPUTE_GRAPHICS_MODEL_QXL,COMPUTE_VIOMMU_MODEL_AUTO,COMPUTE_SECURITY_UEFI_SECURE_BOOT,COMPUTE_NET_VIF_MODEL_E1000,COMPUTE_TRUSTED_CERTS,COMPUTE_VOLUME_MULTI_ATTACH,COMPUTE_STORAGE_BUS_FDC,COMPUTE_VOLUME_EXTEND,COMPUTE_GRAPHICS_MODEL_CIRRUS,COMPUTE_GRAPHICS_MODEL_VMVGA,COMPUTE_GRAPHICS_MODEL_BOCHS,COMPUTE_DEVICE_TAGGING,COMPUTE_STORAGE_BUS_SCSI,COMPUTE_STORAGE_BUS_VIRTIO,HW_CPU_X86_SSSE3,HW_CPU_X86_SSE2,COMPUTE_IMAGE_TYPE_AMI,COMPUTE_STORAGE_BUS_USB,COMPUTE_NET_VIF_MODEL_PCNET,COMPUTE_NET_VIF_MODEL_VIRTIO,COMPUTE_IMAGE_TYPE_ISO,HW_CPU_X86_SSE42,HW_CPU_X86_SSE,COMPUTE_IMAGE_TYPE_ARI,COMPUTE_IMAGE_TYPE_RAW,COMPUTE_IMAGE_TYPE_QCOW2,COMPUTE_NET_VIF_MODEL_NE2K_PCI,COMPUTE_NODE,HW_CPU_X86_MMX,COMPUTE_NET_VIF_MODEL_RTL8139,COMPUTE_GRAPHICS_MODEL_VIRTIO,COMPUTE_NET_VIF_MODEL_SPAPR_VLAN,COMPUTE_STORAGE_BUS_SATA,COMPUTE_NET_ATTACH_INTERFACE,COMPUTE_RESCUE_BFV {{(pid=72351) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:825}}
Aug 09 02:54:30 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:54:30 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:54:30 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Compute_service record updated for user:user {{(pid=72351) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Aug 09 02:54:30 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.484s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:54:32 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:54:34 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Acquiring lock "ccc96aeb-4de7-4511-a668-e3dee04e22f5" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:54:34 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "ccc96aeb-4de7-4511-a668-e3dee04e22f5" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:54:34 user nova-compute[72351]: DEBUG nova.compute.manager [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Starting instance... {{(pid=72351) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2406}}
Aug 09 02:54:34 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:54:34 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:54:34 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72351) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Aug 09 02:54:34 user nova-compute[72351]: INFO nova.compute.claims [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Claim successful on node user
Aug 09 02:54:34 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:54:34 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:54:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.183s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:54:35 user nova-compute[72351]: DEBUG nova.compute.manager [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Start building networks asynchronously for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2803}}
Aug 09 02:54:35 user nova-compute[72351]: DEBUG nova.compute.manager [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Allocating IP information in the background. {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1956}}
Aug 09 02:54:35 user nova-compute[72351]: DEBUG nova.network.neutron [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] allocate_for_instance() {{(pid=72351) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Aug 09 02:54:35 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Aug 09 02:54:35 user nova-compute[72351]: DEBUG nova.compute.manager [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Start building block device mappings for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2838}}
Aug 09 02:54:35 user nova-compute[72351]: DEBUG nova.policy [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'a2bc49dd654f4beca4b8eb5a49cd350d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec9861e0882e4065ab99ad3f74d6b25f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72351) authorize /opt/stack/nova/nova/policy.py:203}}
Aug 09 02:54:35 user nova-compute[72351]: DEBUG nova.compute.manager [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Start spawning the instance on the hypervisor. {{(pid=72351) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2612}}
Aug 09 02:54:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Creating instance directory {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4717}}
Aug 09 02:54:35 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Creating image(s)
Aug 09 02:54:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Acquiring lock "/opt/stack/data/nova/instances/ccc96aeb-4de7-4511-a668-e3dee04e22f5/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:54:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "/opt/stack/data/nova/instances/ccc96aeb-4de7-4511-a668-e3dee04e22f5/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:54:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "/opt/stack/data/nova/instances/ccc96aeb-4de7-4511-a668-e3dee04e22f5/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.008s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:54:35 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:54:35 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:54:35 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.161s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:54:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Acquiring lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:54:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:54:35 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:54:35 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.134s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:54:35 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/ccc96aeb-4de7-4511-a668-e3dee04e22f5/disk 1073741824 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:54:35 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/ccc96aeb-4de7-4511-a668-e3dee04e22f5/disk 1073741824" returned: 0 in 0.054s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:54:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.196s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:54:35 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:54:35 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.159s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:54:35 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Checking if we can resize image /opt/stack/data/nova/instances/ccc96aeb-4de7-4511-a668-e3dee04e22f5/disk. size=1073741824 {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Aug 09 02:54:35 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ccc96aeb-4de7-4511-a668-e3dee04e22f5/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:54:35 user nova-compute[72351]: DEBUG nova.network.neutron [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Successfully created port: 07ee9224-5c7b-4362-82a4-54731362c757 {{(pid=72351) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Aug 09 02:54:35 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ccc96aeb-4de7-4511-a668-e3dee04e22f5/disk --force-share --output=json" returned: 0 in 0.167s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:54:35 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Cannot resize image /opt/stack/data/nova/instances/ccc96aeb-4de7-4511-a668-e3dee04e22f5/disk to a smaller size. {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Aug 09 02:54:35 user nova-compute[72351]: DEBUG nova.objects.instance [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lazy-loading 'migration_context' on Instance uuid ccc96aeb-4de7-4511-a668-e3dee04e22f5 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:54:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Created local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4851}}
Aug 09 02:54:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Ensure instance console log exists: /opt/stack/data/nova/instances/ccc96aeb-4de7-4511-a668-e3dee04e22f5/console.log {{(pid=72351) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4603}}
Aug 09 02:54:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:54:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:54:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:54:36 user nova-compute[72351]: DEBUG nova.network.neutron [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Successfully updated port: 07ee9224-5c7b-4362-82a4-54731362c757 {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 02:54:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Acquiring lock "refresh_cache-ccc96aeb-4de7-4511-a668-e3dee04e22f5" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:54:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Acquired lock "refresh_cache-ccc96aeb-4de7-4511-a668-e3dee04e22f5" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:54:36 user nova-compute[72351]: DEBUG nova.network.neutron [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 02:54:36 user nova-compute[72351]: DEBUG nova.compute.manager [req-07b21a21-f95e-4956-ac80-6678754cf171 req-2bf8eb73-3b55-4391-b6c5-896c7ea4776d service nova] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Received event network-changed-07ee9224-5c7b-4362-82a4-54731362c757 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:54:36 user nova-compute[72351]: DEBUG nova.compute.manager [req-07b21a21-f95e-4956-ac80-6678754cf171 req-2bf8eb73-3b55-4391-b6c5-896c7ea4776d service nova] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Refreshing instance network info cache due to event network-changed-07ee9224-5c7b-4362-82a4-54731362c757. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:54:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-07b21a21-f95e-4956-ac80-6678754cf171 req-2bf8eb73-3b55-4391-b6c5-896c7ea4776d service nova] Acquiring lock "refresh_cache-ccc96aeb-4de7-4511-a668-e3dee04e22f5" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:54:36 user nova-compute[72351]: DEBUG nova.network.neutron [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 02:54:37 user nova-compute[72351]: DEBUG nova.network.neutron [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Updating instance_info_cache with network_info: [{"id": "07ee9224-5c7b-4362-82a4-54731362c757", "address": "fa:16:3e:a4:6f:41", "network": {"id": "dba57c3d-6efd-4057-8053-1c6a92bb24f8", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-568585773-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ec9861e0882e4065ab99ad3f74d6b25f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap07ee9224-5c", "ovs_interfaceid": "07ee9224-5c7b-4362-82a4-54731362c757", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:54:37 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Releasing lock "refresh_cache-ccc96aeb-4de7-4511-a668-e3dee04e22f5" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:54:37 user nova-compute[72351]: DEBUG nova.compute.manager [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Instance network_info: |[{"id": "07ee9224-5c7b-4362-82a4-54731362c757", "address": "fa:16:3e:a4:6f:41", "network": {"id": "dba57c3d-6efd-4057-8053-1c6a92bb24f8", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-568585773-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ec9861e0882e4065ab99ad3f74d6b25f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap07ee9224-5c", "ovs_interfaceid": "07ee9224-5c7b-4362-82a4-54731362c757", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1971}}
Aug 09 02:54:37 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-07b21a21-f95e-4956-ac80-6678754cf171 req-2bf8eb73-3b55-4391-b6c5-896c7ea4776d service nova] Acquired lock "refresh_cache-ccc96aeb-4de7-4511-a668-e3dee04e22f5" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:54:37 user nova-compute[72351]: DEBUG nova.network.neutron [req-07b21a21-f95e-4956-ac80-6678754cf171 req-2bf8eb73-3b55-4391-b6c5-896c7ea4776d service nova] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Refreshing network info cache for port 07ee9224-5c7b-4362-82a4-54731362c757 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:54:37 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Start _get_guest_xml network_info=[{"id": "07ee9224-5c7b-4362-82a4-54731362c757", "address": "fa:16:3e:a4:6f:41", "network": {"id": "dba57c3d-6efd-4057-8053-1c6a92bb24f8", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-568585773-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ec9861e0882e4065ab99ad3f74d6b25f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap07ee9224-5c", "ovs_interfaceid": "07ee9224-5c7b-4362-82a4-54731362c757", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'size': 0, 'device_type': 'disk', 'encryption_secret_uuid': None, 'guest_format': None, 'encryption_options': None, 'disk_bus': 'virtio', 'encrypted': False, 'encryption_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'image_id': '18ccda7f-31d3-4978-ab9d-4ed2f74eff94'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 02:54:37 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:54:37 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:54:37 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 02:54:37 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 02:54:37 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 02:54:37 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 02:54:37 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 02:54:37 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 02:54:37 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 02:54:37 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 02:54:37 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 02:54:37 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 02:54:37 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 02:54:37 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 02:54:37 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 02:54:37 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 02:54:37 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 02:54:37 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 02:54:37 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:54:34Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachVolumeTestJSON-server-552342611',display_name='tempest-AttachVolumeTestJSON-server-552342611',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumetestjson-server-552342611',id=51,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBIUwHXpH2fJ3g3wNlmi7NgH+LwCmDsQaQHWhrNQiMNmpopZzyLyOCqErm6qlmfocX44iiHaCxZQak7p6ZXLZG4tz3AK9MrpUOgwFC/D5lYRrszrhgIM72DjhVn9gCRfaoQ==',key_name='tempest-keypair-1729558399',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='ec9861e0882e4065ab99ad3f74d6b25f',ramdisk_id='',reservation_id='r-i5wzpbxv',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeTestJSON-1725368857',owner_user_name='tempest-AttachVolumeTestJSON-1725368857-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:54:35Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='a2bc49dd654f4beca4b8eb5a49cd350d',uuid=ccc96aeb-4de7-4511-a668-e3dee04e22f5,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "07ee9224-5c7b-4362-82a4-54731362c757", "address": "fa:16:3e:a4:6f:41", "network": {"id": "dba57c3d-6efd-4057-8053-1c6a92bb24f8", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-568585773-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ec9861e0882e4065ab99ad3f74d6b25f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap07ee9224-5c", "ovs_interfaceid": "07ee9224-5c7b-4362-82a4-54731362c757", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 02:54:37 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Converting VIF {"id": "07ee9224-5c7b-4362-82a4-54731362c757", "address": "fa:16:3e:a4:6f:41", "network": {"id": "dba57c3d-6efd-4057-8053-1c6a92bb24f8", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-568585773-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ec9861e0882e4065ab99ad3f74d6b25f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap07ee9224-5c", "ovs_interfaceid": "07ee9224-5c7b-4362-82a4-54731362c757", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:54:37 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:a4:6f:41,bridge_name='br-int',has_traffic_filtering=True,id=07ee9224-5c7b-4362-82a4-54731362c757,network=Network(dba57c3d-6efd-4057-8053-1c6a92bb24f8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap07ee9224-5c') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:54:37 user nova-compute[72351]: DEBUG nova.objects.instance [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lazy-loading 'pci_devices' on Instance uuid ccc96aeb-4de7-4511-a668-e3dee04e22f5 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:54:37 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] End _get_guest_xml xml=<domain type="kvm">
Aug 09 02:54:37 user nova-compute[72351]:   <uuid>ccc96aeb-4de7-4511-a668-e3dee04e22f5</uuid>
Aug 09 02:54:37 user nova-compute[72351]:   <name>instance-00000033</name>
Aug 09 02:54:37 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 02:54:37 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 02:54:37 user nova-compute[72351]:   <metadata>
Aug 09 02:54:37 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 02:54:37 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 02:54:37 user nova-compute[72351]:       <nova:name>tempest-AttachVolumeTestJSON-server-552342611</nova:name>
Aug 09 02:54:37 user nova-compute[72351]:       <nova:creationTime>2023-08-09 02:54:37</nova:creationTime>
Aug 09 02:54:37 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 02:54:37 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 02:54:37 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 02:54:37 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 02:54:37 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 02:54:37 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 02:54:37 user nova-compute[72351]:       </nova:flavor>
Aug 09 02:54:37 user nova-compute[72351]:       <nova:owner>
Aug 09 02:54:37 user nova-compute[72351]:         <nova:user uuid="a2bc49dd654f4beca4b8eb5a49cd350d">tempest-AttachVolumeTestJSON-1725368857-project-member</nova:user>
Aug 09 02:54:37 user nova-compute[72351]:         <nova:project uuid="ec9861e0882e4065ab99ad3f74d6b25f">tempest-AttachVolumeTestJSON-1725368857</nova:project>
Aug 09 02:54:37 user nova-compute[72351]:       </nova:owner>
Aug 09 02:54:37 user nova-compute[72351]:       <nova:root type="image" uuid="18ccda7f-31d3-4978-ab9d-4ed2f74eff94"/>
Aug 09 02:54:37 user nova-compute[72351]:       <nova:ports>
Aug 09 02:54:37 user nova-compute[72351]:         <nova:port uuid="07ee9224-5c7b-4362-82a4-54731362c757">
Aug 09 02:54:37 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.10" ipVersion="4"/>
Aug 09 02:54:37 user nova-compute[72351]:         </nova:port>
Aug 09 02:54:37 user nova-compute[72351]:       </nova:ports>
Aug 09 02:54:37 user nova-compute[72351]:     </nova:instance>
Aug 09 02:54:37 user nova-compute[72351]:   </metadata>
Aug 09 02:54:37 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 02:54:37 user nova-compute[72351]:     <system>
Aug 09 02:54:37 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 02:54:37 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 02:54:37 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 02:54:37 user nova-compute[72351]:       <entry name="serial">ccc96aeb-4de7-4511-a668-e3dee04e22f5</entry>
Aug 09 02:54:37 user nova-compute[72351]:       <entry name="uuid">ccc96aeb-4de7-4511-a668-e3dee04e22f5</entry>
Aug 09 02:54:37 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 02:54:37 user nova-compute[72351]:     </system>
Aug 09 02:54:37 user nova-compute[72351]:   </sysinfo>
Aug 09 02:54:37 user nova-compute[72351]:   <os>
Aug 09 02:54:37 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 02:54:37 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 02:54:37 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 02:54:37 user nova-compute[72351]:   </os>
Aug 09 02:54:37 user nova-compute[72351]:   <features>
Aug 09 02:54:37 user nova-compute[72351]:     <acpi/>
Aug 09 02:54:37 user nova-compute[72351]:     <apic/>
Aug 09 02:54:37 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 02:54:37 user nova-compute[72351]:   </features>
Aug 09 02:54:37 user nova-compute[72351]:   <clock offset="utc">
Aug 09 02:54:37 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 02:54:37 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 02:54:37 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 02:54:37 user nova-compute[72351]:   </clock>
Aug 09 02:54:37 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 02:54:37 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 02:54:37 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 02:54:37 user nova-compute[72351]:   </cpu>
Aug 09 02:54:37 user nova-compute[72351]:   <devices>
Aug 09 02:54:37 user nova-compute[72351]:     <disk type="file" device="disk">
Aug 09 02:54:37 user nova-compute[72351]:       <driver name="qemu" type="qcow2" cache="none"/>
Aug 09 02:54:37 user nova-compute[72351]:       <source file="/opt/stack/data/nova/instances/ccc96aeb-4de7-4511-a668-e3dee04e22f5/disk"/>
Aug 09 02:54:37 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 02:54:37 user nova-compute[72351]:     </disk>
Aug 09 02:54:37 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 02:54:37 user nova-compute[72351]:       <mac address="fa:16:3e:a4:6f:41"/>
Aug 09 02:54:37 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:54:37 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 02:54:37 user nova-compute[72351]:       <target dev="tap07ee9224-5c"/>
Aug 09 02:54:37 user nova-compute[72351]:     </interface>
Aug 09 02:54:37 user nova-compute[72351]:     <serial type="pty">
Aug 09 02:54:37 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/ccc96aeb-4de7-4511-a668-e3dee04e22f5/console.log" append="off"/>
Aug 09 02:54:37 user nova-compute[72351]:     </serial>
Aug 09 02:54:37 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 02:54:37 user nova-compute[72351]:     <video>
Aug 09 02:54:37 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 02:54:37 user nova-compute[72351]:     </video>
Aug 09 02:54:37 user nova-compute[72351]:     <rng model="virtio">
Aug 09 02:54:37 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 02:54:37 user nova-compute[72351]:     </rng>
Aug 09 02:54:37 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 02:54:37 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 02:54:37 user nova-compute[72351]:       <stats period="10"/>
Aug 09 02:54:37 user nova-compute[72351]:     </memballoon>
Aug 09 02:54:37 user nova-compute[72351]:   </devices>
Aug 09 02:54:37 user nova-compute[72351]: </domain>
Aug 09 02:54:37 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 02:54:37 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:54:34Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachVolumeTestJSON-server-552342611',display_name='tempest-AttachVolumeTestJSON-server-552342611',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumetestjson-server-552342611',id=51,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBIUwHXpH2fJ3g3wNlmi7NgH+LwCmDsQaQHWhrNQiMNmpopZzyLyOCqErm6qlmfocX44iiHaCxZQak7p6ZXLZG4tz3AK9MrpUOgwFC/D5lYRrszrhgIM72DjhVn9gCRfaoQ==',key_name='tempest-keypair-1729558399',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='ec9861e0882e4065ab99ad3f74d6b25f',ramdisk_id='',reservation_id='r-i5wzpbxv',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeTestJSON-1725368857',owner_user_name='tempest-AttachVolumeTestJSON-1725368857-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T02:54:35Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='a2bc49dd654f4beca4b8eb5a49cd350d',uuid=ccc96aeb-4de7-4511-a668-e3dee04e22f5,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "07ee9224-5c7b-4362-82a4-54731362c757", "address": "fa:16:3e:a4:6f:41", "network": {"id": "dba57c3d-6efd-4057-8053-1c6a92bb24f8", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-568585773-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ec9861e0882e4065ab99ad3f74d6b25f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap07ee9224-5c", "ovs_interfaceid": "07ee9224-5c7b-4362-82a4-54731362c757", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 02:54:37 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Converting VIF {"id": "07ee9224-5c7b-4362-82a4-54731362c757", "address": "fa:16:3e:a4:6f:41", "network": {"id": "dba57c3d-6efd-4057-8053-1c6a92bb24f8", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-568585773-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ec9861e0882e4065ab99ad3f74d6b25f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap07ee9224-5c", "ovs_interfaceid": "07ee9224-5c7b-4362-82a4-54731362c757", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:54:37 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:a4:6f:41,bridge_name='br-int',has_traffic_filtering=True,id=07ee9224-5c7b-4362-82a4-54731362c757,network=Network(dba57c3d-6efd-4057-8053-1c6a92bb24f8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap07ee9224-5c') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:54:37 user nova-compute[72351]: DEBUG os_vif [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:a4:6f:41,bridge_name='br-int',has_traffic_filtering=True,id=07ee9224-5c7b-4362-82a4-54731362c757,network=Network(dba57c3d-6efd-4057-8053-1c6a92bb24f8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap07ee9224-5c') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 02:54:37 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:54:37 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:54:37 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 02:54:37 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:54:37 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap07ee9224-5c, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:54:37 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap07ee9224-5c, col_values=(('external_ids', {'iface-id': '07ee9224-5c7b-4362-82a4-54731362c757', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:a4:6f:41', 'vm-uuid': 'ccc96aeb-4de7-4511-a668-e3dee04e22f5'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:54:37 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:54:37 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:54:37 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:54:37 user nova-compute[72351]: INFO os_vif [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:a4:6f:41,bridge_name='br-int',has_traffic_filtering=True,id=07ee9224-5c7b-4362-82a4-54731362c757,network=Network(dba57c3d-6efd-4057-8053-1c6a92bb24f8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap07ee9224-5c')
Aug 09 02:54:37 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:54:37 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] No VIF found with MAC fa:16:3e:a4:6f:41, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:54:37 user nova-compute[72351]: DEBUG nova.network.neutron [req-07b21a21-f95e-4956-ac80-6678754cf171 req-2bf8eb73-3b55-4391-b6c5-896c7ea4776d service nova] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Updated VIF entry in instance network info cache for port 07ee9224-5c7b-4362-82a4-54731362c757. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:54:37 user nova-compute[72351]: DEBUG nova.network.neutron [req-07b21a21-f95e-4956-ac80-6678754cf171 req-2bf8eb73-3b55-4391-b6c5-896c7ea4776d service nova] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Updating instance_info_cache with network_info: [{"id": "07ee9224-5c7b-4362-82a4-54731362c757", "address": "fa:16:3e:a4:6f:41", "network": {"id": "dba57c3d-6efd-4057-8053-1c6a92bb24f8", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-568585773-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ec9861e0882e4065ab99ad3f74d6b25f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap07ee9224-5c", "ovs_interfaceid": "07ee9224-5c7b-4362-82a4-54731362c757", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:54:37 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-07b21a21-f95e-4956-ac80-6678754cf171 req-2bf8eb73-3b55-4391-b6c5-896c7ea4776d service nova] Releasing lock "refresh_cache-ccc96aeb-4de7-4511-a668-e3dee04e22f5" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:54:38 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:54:38 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:54:38 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:54:38 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:54:39 user nova-compute[72351]: DEBUG nova.compute.manager [req-a7e84923-ad3f-45df-81d9-2d3799217bf9 req-b5e73e7d-b709-4129-aad6-26411b4e0ec6 service nova] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Received event network-vif-plugged-07ee9224-5c7b-4362-82a4-54731362c757 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:54:39 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a7e84923-ad3f-45df-81d9-2d3799217bf9 req-b5e73e7d-b709-4129-aad6-26411b4e0ec6 service nova] Acquiring lock "ccc96aeb-4de7-4511-a668-e3dee04e22f5-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:54:39 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a7e84923-ad3f-45df-81d9-2d3799217bf9 req-b5e73e7d-b709-4129-aad6-26411b4e0ec6 service nova] Lock "ccc96aeb-4de7-4511-a668-e3dee04e22f5-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:54:39 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-a7e84923-ad3f-45df-81d9-2d3799217bf9 req-b5e73e7d-b709-4129-aad6-26411b4e0ec6 service nova] Lock "ccc96aeb-4de7-4511-a668-e3dee04e22f5-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:54:39 user nova-compute[72351]: DEBUG nova.compute.manager [req-a7e84923-ad3f-45df-81d9-2d3799217bf9 req-b5e73e7d-b709-4129-aad6-26411b4e0ec6 service nova] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] No waiting events found dispatching network-vif-plugged-07ee9224-5c7b-4362-82a4-54731362c757 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:54:39 user nova-compute[72351]: WARNING nova.compute.manager [req-a7e84923-ad3f-45df-81d9-2d3799217bf9 req-b5e73e7d-b709-4129-aad6-26411b4e0ec6 service nova] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Received unexpected event network-vif-plugged-07ee9224-5c7b-4362-82a4-54731362c757 for instance with vm_state building and task_state spawning.
Aug 09 02:54:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:54:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:54:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:54:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:54:41 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691549681.0495434, ccc96aeb-4de7-4511-a668-e3dee04e22f5 => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:54:41 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] VM Resumed (Lifecycle Event)
Aug 09 02:54:41 user nova-compute[72351]: DEBUG nova.compute.manager [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 02:54:41 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Guest created on hypervisor {{(pid=72351) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4411}}
Aug 09 02:54:41 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Instance spawned successfully.
Aug 09 02:54:41 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:894}}
Aug 09 02:54:41 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:54:41 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Found default for hw_cdrom_bus of ide {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:54:41 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Found default for hw_disk_bus of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:54:41 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Found default for hw_input_bus of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:54:41 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Found default for hw_pointer_model of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:54:41 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Found default for hw_video_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:54:41 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Found default for hw_vif_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 02:54:41 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:54:41 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:54:41 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691549681.051064, ccc96aeb-4de7-4511-a668-e3dee04e22f5 => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:54:41 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] VM Started (Lifecycle Event)
Aug 09 02:54:41 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:54:41 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 02:54:41 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 02:54:41 user nova-compute[72351]: INFO nova.compute.manager [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Took 5.95 seconds to spawn the instance on the hypervisor.
Aug 09 02:54:41 user nova-compute[72351]: DEBUG nova.compute.manager [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:54:41 user nova-compute[72351]: DEBUG nova.compute.manager [req-8e1c8eab-5141-4a38-b11b-690e9845b4cc req-98bc1738-b40b-4628-83d4-f4164c53ce91 service nova] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Received event network-vif-plugged-07ee9224-5c7b-4362-82a4-54731362c757 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:54:41 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-8e1c8eab-5141-4a38-b11b-690e9845b4cc req-98bc1738-b40b-4628-83d4-f4164c53ce91 service nova] Acquiring lock "ccc96aeb-4de7-4511-a668-e3dee04e22f5-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:54:41 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-8e1c8eab-5141-4a38-b11b-690e9845b4cc req-98bc1738-b40b-4628-83d4-f4164c53ce91 service nova] Lock "ccc96aeb-4de7-4511-a668-e3dee04e22f5-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:54:41 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-8e1c8eab-5141-4a38-b11b-690e9845b4cc req-98bc1738-b40b-4628-83d4-f4164c53ce91 service nova] Lock "ccc96aeb-4de7-4511-a668-e3dee04e22f5-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:54:41 user nova-compute[72351]: DEBUG nova.compute.manager [req-8e1c8eab-5141-4a38-b11b-690e9845b4cc req-98bc1738-b40b-4628-83d4-f4164c53ce91 service nova] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] No waiting events found dispatching network-vif-plugged-07ee9224-5c7b-4362-82a4-54731362c757 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:54:41 user nova-compute[72351]: WARNING nova.compute.manager [req-8e1c8eab-5141-4a38-b11b-690e9845b4cc req-98bc1738-b40b-4628-83d4-f4164c53ce91 service nova] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Received unexpected event network-vif-plugged-07ee9224-5c7b-4362-82a4-54731362c757 for instance with vm_state building and task_state spawning.
Aug 09 02:54:41 user nova-compute[72351]: INFO nova.compute.manager [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Took 6.46 seconds to build instance.
Aug 09 02:54:41 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-6a32e696-c388-4947-ac91-df35d710104f tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "ccc96aeb-4de7-4511-a668-e3dee04e22f5" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 6.549s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:54:42 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:54:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:54:47 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:54:52 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:54:52 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:54:52 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5003 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 02:54:52 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:54:52 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:54:52 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:54:57 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:54:59 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:55:02 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:55:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:55:07 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:55:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:55:12 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:55:17 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:55:20 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:55:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:55:21 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:55:21 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72351) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10461}}
Aug 09 02:55:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:55:24 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:55:25 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:55:25 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Starting heal instance info cache {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9842}}
Aug 09 02:55:25 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Rebuilding the list of instances to heal {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9846}}
Aug 09 02:55:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "refresh_cache-ccc96aeb-4de7-4511-a668-e3dee04e22f5" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:55:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquired lock "refresh_cache-ccc96aeb-4de7-4511-a668-e3dee04e22f5" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:55:25 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Forcefully refreshing network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1996}}
Aug 09 02:55:25 user nova-compute[72351]: DEBUG nova.objects.instance [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lazy-loading 'info_cache' on Instance uuid ccc96aeb-4de7-4511-a668-e3dee04e22f5 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:55:26 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Updating instance_info_cache with network_info: [{"id": "07ee9224-5c7b-4362-82a4-54731362c757", "address": "fa:16:3e:a4:6f:41", "network": {"id": "dba57c3d-6efd-4057-8053-1c6a92bb24f8", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-568585773-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ec9861e0882e4065ab99ad3f74d6b25f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap07ee9224-5c", "ovs_interfaceid": "07ee9224-5c7b-4362-82a4-54731362c757", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:55:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Releasing lock "refresh_cache-ccc96aeb-4de7-4511-a668-e3dee04e22f5" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:55:26 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Updated the network info_cache for instance {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9913}}
Aug 09 02:55:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:55:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:55:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5003 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 02:55:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:55:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:55:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:55:27 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:55:29 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:55:29 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:55:31 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:55:31 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:55:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:55:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:55:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:55:31 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Auditing locally available compute resources for user (node: user) {{(pid=72351) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Aug 09 02:55:32 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ccc96aeb-4de7-4511-a668-e3dee04e22f5/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:55:32 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ccc96aeb-4de7-4511-a668-e3dee04e22f5/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:55:32 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ccc96aeb-4de7-4511-a668-e3dee04e22f5/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:55:32 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ccc96aeb-4de7-4511-a668-e3dee04e22f5/disk --force-share --output=json" returned: 0 in 0.130s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:55:32 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:55:32 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:55:32 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:55:32 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Hypervisor/Node resource view: name=user free_ram=8751MB free_disk=25.208087921142578GB free_vcpus=11 pci_devices=[{"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72351) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Aug 09 02:55:32 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:55:32 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:55:32 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance ccc96aeb-4de7-4511-a668-e3dee04e22f5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:55:32 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Total usable vcpus: 12, total allocated vcpus: 1 {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Aug 09 02:55:32 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Final resource view: name=user phys_ram=16011MB used_ram=1024MB phys_disk=40GB used_disk=1GB total_vcpus=12 used_vcpus=1 pci_stats=[] {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Aug 09 02:55:32 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:55:32 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:55:32 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Compute_service record updated for user:user {{(pid=72351) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Aug 09 02:55:32 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.202s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:55:37 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:55:37 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:55:37 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 02:55:37 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:55:37 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:55:37 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:55:38 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:55:42 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:55:47 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:55:52 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:55:57 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:56:02 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:56:07 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:56:12 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:56:17 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:56:17 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:56:17 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5001 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 02:56:17 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:56:17 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:56:17 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:56:20 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:56:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:56:23 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:56:23 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72351) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10461}}
Aug 09 02:56:25 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:56:25 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Starting heal instance info cache {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9842}}
Aug 09 02:56:25 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Rebuilding the list of instances to heal {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9846}}
Aug 09 02:56:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "refresh_cache-ccc96aeb-4de7-4511-a668-e3dee04e22f5" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:56:25 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquired lock "refresh_cache-ccc96aeb-4de7-4511-a668-e3dee04e22f5" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:56:25 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Forcefully refreshing network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1996}}
Aug 09 02:56:25 user nova-compute[72351]: DEBUG nova.objects.instance [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lazy-loading 'info_cache' on Instance uuid ccc96aeb-4de7-4511-a668-e3dee04e22f5 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:56:26 user nova-compute[72351]: DEBUG nova.compute.manager [req-57126890-4d26-4d6e-b948-824a11c7492e req-610bf770-49c5-4ad8-88bc-69a8bb33d5fe service nova] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Received event network-changed-07ee9224-5c7b-4362-82a4-54731362c757 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:56:26 user nova-compute[72351]: DEBUG nova.compute.manager [req-57126890-4d26-4d6e-b948-824a11c7492e req-610bf770-49c5-4ad8-88bc-69a8bb33d5fe service nova] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Refreshing instance network info cache due to event network-changed-07ee9224-5c7b-4362-82a4-54731362c757. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 02:56:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-57126890-4d26-4d6e-b948-824a11c7492e req-610bf770-49c5-4ad8-88bc-69a8bb33d5fe service nova] Acquiring lock "refresh_cache-ccc96aeb-4de7-4511-a668-e3dee04e22f5" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 02:56:26 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Updating instance_info_cache with network_info: [{"id": "07ee9224-5c7b-4362-82a4-54731362c757", "address": "fa:16:3e:a4:6f:41", "network": {"id": "dba57c3d-6efd-4057-8053-1c6a92bb24f8", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-568585773-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.182", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ec9861e0882e4065ab99ad3f74d6b25f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap07ee9224-5c", "ovs_interfaceid": "07ee9224-5c7b-4362-82a4-54731362c757", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:56:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Releasing lock "refresh_cache-ccc96aeb-4de7-4511-a668-e3dee04e22f5" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:56:26 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Updated the network info_cache for instance {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9913}}
Aug 09 02:56:26 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-57126890-4d26-4d6e-b948-824a11c7492e req-610bf770-49c5-4ad8-88bc-69a8bb33d5fe service nova] Acquired lock "refresh_cache-ccc96aeb-4de7-4511-a668-e3dee04e22f5" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 02:56:26 user nova-compute[72351]: DEBUG nova.network.neutron [req-57126890-4d26-4d6e-b948-824a11c7492e req-610bf770-49c5-4ad8-88bc-69a8bb33d5fe service nova] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Refreshing network info cache for port 07ee9224-5c7b-4362-82a4-54731362c757 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 02:56:26 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:56:27 user nova-compute[72351]: DEBUG nova.network.neutron [req-57126890-4d26-4d6e-b948-824a11c7492e req-610bf770-49c5-4ad8-88bc-69a8bb33d5fe service nova] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Updated VIF entry in instance network info cache for port 07ee9224-5c7b-4362-82a4-54731362c757. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 02:56:27 user nova-compute[72351]: DEBUG nova.network.neutron [req-57126890-4d26-4d6e-b948-824a11c7492e req-610bf770-49c5-4ad8-88bc-69a8bb33d5fe service nova] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Updating instance_info_cache with network_info: [{"id": "07ee9224-5c7b-4362-82a4-54731362c757", "address": "fa:16:3e:a4:6f:41", "network": {"id": "dba57c3d-6efd-4057-8053-1c6a92bb24f8", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-568585773-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.182", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ec9861e0882e4065ab99ad3f74d6b25f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap07ee9224-5c", "ovs_interfaceid": "07ee9224-5c7b-4362-82a4-54731362c757", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:56:27 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-57126890-4d26-4d6e-b948-824a11c7492e req-610bf770-49c5-4ad8-88bc-69a8bb33d5fe service nova] Releasing lock "refresh_cache-ccc96aeb-4de7-4511-a668-e3dee04e22f5" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 02:56:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:56:29 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:56:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-df33f55e-8108-4993-936e-fc46d3909473 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Acquiring lock "ccc96aeb-4de7-4511-a668-e3dee04e22f5" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:56:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-df33f55e-8108-4993-936e-fc46d3909473 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "ccc96aeb-4de7-4511-a668-e3dee04e22f5" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:56:29 user nova-compute[72351]: DEBUG nova.objects.instance [None req-df33f55e-8108-4993-936e-fc46d3909473 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lazy-loading 'flavor' on Instance uuid ccc96aeb-4de7-4511-a668-e3dee04e22f5 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:56:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-df33f55e-8108-4993-936e-fc46d3909473 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "ccc96aeb-4de7-4511-a668-e3dee04e22f5" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.051s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:56:29 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:56:30 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-df33f55e-8108-4993-936e-fc46d3909473 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Acquiring lock "ccc96aeb-4de7-4511-a668-e3dee04e22f5" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:56:30 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-df33f55e-8108-4993-936e-fc46d3909473 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "ccc96aeb-4de7-4511-a668-e3dee04e22f5" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:56:30 user nova-compute[72351]: INFO nova.compute.manager [None req-df33f55e-8108-4993-936e-fc46d3909473 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Attaching volume 7d4f3c7d-7d2c-480f-8bc1-fe2ed2e58ecf to /dev/vdb
Aug 09 02:56:30 user nova-compute[72351]: DEBUG os_brick.utils [None req-df33f55e-8108-4993-936e-fc46d3909473 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:56:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:56:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.015s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:56:30 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[4b667f8b-5467-41bb-8d28-83ac690e99f7]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:a52269602ff4\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:56:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:56:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.015s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:56:30 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[42f156e2-98ee-46fe-9bac-8dfe8b5d383e]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:56:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:56:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.016s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:56:30 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[faf7506e-3415-41fd-be48-d1c7db13b617]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:56:30 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[467ec404-068a-497e-9eba-44e2b70d5d3c]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:56:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-df33f55e-8108-4993-936e-fc46d3909473 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Running cmd (subprocess): nvme version {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:56:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-df33f55e-8108-4993-936e-fc46d3909473 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] 'nvme version' failed. Not Retrying. {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:56:30 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.nvmeof [None req-df33f55e-8108-4993-936e-fc46d3909473 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] nvme not present on system {{(pid=72351) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Aug 09 02:56:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:56:30 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:56:30 user nova-compute[72351]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Aug 09 02:56:30 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[1192a24d-3d8e-4942-8fe7-4d368cd74ab4]: (4, '') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:56:30 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-df33f55e-8108-4993-936e-fc46d3909473 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72351) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Aug 09 02:56:30 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-df33f55e-8108-4993-936e-fc46d3909473 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Aug 09 02:56:30 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-df33f55e-8108-4993-936e-fc46d3909473 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] LIGHTOS: no hostnqn found. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Aug 09 02:56:30 user nova-compute[72351]: DEBUG os_brick.utils [None req-df33f55e-8108-4993-936e-fc46d3909473 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] <== get_connector_properties: return (122ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:a52269602ff4', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:56:30 user nova-compute[72351]: DEBUG nova.virt.block_device [None req-df33f55e-8108-4993-936e-fc46d3909473 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Updating existing volume attachment record: 5ae3fb0b-1aaf-49bd-864d-5141ce358313 {{(pid=72351) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Aug 09 02:56:30 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:56:30 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:56:31 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:56:31 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:56:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:56:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:56:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:56:31 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Auditing locally available compute resources for user (node: user) {{(pid=72351) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Aug 09 02:56:32 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ccc96aeb-4de7-4511-a668-e3dee04e22f5/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:56:32 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ccc96aeb-4de7-4511-a668-e3dee04e22f5/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:56:32 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ccc96aeb-4de7-4511-a668-e3dee04e22f5/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:56:32 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ccc96aeb-4de7-4511-a668-e3dee04e22f5/disk --force-share --output=json" returned: 0 in 0.149s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:56:32 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:56:32 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:56:32 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:56:32 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Hypervisor/Node resource view: name=user free_ram=8757MB free_disk=25.198509216308594GB free_vcpus=11 pci_devices=[{"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72351) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Aug 09 02:56:32 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:56:32 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:56:32 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance ccc96aeb-4de7-4511-a668-e3dee04e22f5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 02:56:32 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Total usable vcpus: 12, total allocated vcpus: 1 {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Aug 09 02:56:32 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Final resource view: name=user phys_ram=16011MB used_ram=1024MB phys_disk=40GB used_disk=1GB total_vcpus=12 used_vcpus=1 pci_stats=[] {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Aug 09 02:56:32 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:56:32 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:56:32 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Compute_service record updated for user:user {{(pid=72351) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Aug 09 02:56:32 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.169s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:56:34 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:56:37 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:56:37 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-df33f55e-8108-4993-936e-fc46d3909473 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 02:56:37 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-df33f55e-8108-4993-936e-fc46d3909473 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:7d4f3c7d-7d2c-480f-8bc1-fe2ed2e58ecf'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'qankzYOy', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:56:37 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-df33f55e-8108-4993-936e-fc46d3909473 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:56:37 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-df33f55e-8108-4993-936e-fc46d3909473 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:56:37 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-df33f55e-8108-4993-936e-fc46d3909473 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:56:37 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-df33f55e-8108-4993-936e-fc46d3909473 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:56:37 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-df33f55e-8108-4993-936e-fc46d3909473 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:7d4f3c7d-7d2c-480f-8bc1-fe2ed2e58ecf" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:56:37 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-df33f55e-8108-4993-936e-fc46d3909473 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:7d4f3c7d-7d2c-480f-8bc1-fe2ed2e58ecf" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:56:37 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-df33f55e-8108-4993-936e-fc46d3909473 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 02:56:37 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7d4f3c7d-7d2c-480f-8bc1-fe2ed2e58ecf -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:56:38 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7d4f3c7d-7d2c-480f-8bc1-fe2ed2e58ecf -p 172.16.0.220:3260" returned: 21 in 0.056s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:56:38 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[7b275780-acac-44d7-be1e-497bb67e55d6]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:56:38 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-df33f55e-8108-4993-936e-fc46d3909473 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Aug 09 02:56:38 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:56:38 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7d4f3c7d-7d2c-480f-8bc1-fe2ed2e58ecf -p 172.16.0.220:3260 --interface default --op new {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:56:38 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7d4f3c7d-7d2c-480f-8bc1-fe2ed2e58ecf -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.031s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:56:38 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e53dd9d5-6db8-41d8-a9c1-dcb3936f76c9]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:7d4f3c7d-7d2c-480f-8bc1-fe2ed2e58ecf] added\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:56:38 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-df33f55e-8108-4993-936e-fc46d3909473 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:7d4f3c7d-7d2c-480f-8bc1-fe2ed2e58ecf] added
Aug 09 02:56:38 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:56:38 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7d4f3c7d-7d2c-480f-8bc1-fe2ed2e58ecf -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:56:38 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7d4f3c7d-7d2c-480f-8bc1-fe2ed2e58ecf -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.045s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:56:38 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[9847dda8-5ec8-41bd-b7cd-85067e341167]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:56:38 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-df33f55e-8108-4993-936e-fc46d3909473 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:56:38 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7d4f3c7d-7d2c-480f-8bc1-fe2ed2e58ecf -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:56:38 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7d4f3c7d-7d2c-480f-8bc1-fe2ed2e58ecf -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.034s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:56:38 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[35443546-c6bc-4ef3-8ff3-85c7d83fc709]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:56:38 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-df33f55e-8108-4993-936e-fc46d3909473 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:56:38 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7d4f3c7d-7d2c-480f-8bc1-fe2ed2e58ecf -p 172.16.0.220:3260 --op update -n node.session.auth.username -v qankzYOy {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:56:38 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7d4f3c7d-7d2c-480f-8bc1-fe2ed2e58ecf -p 172.16.0.220:3260 --op update -n node.session.auth.username -v qankzYOy" returned: 0 in 0.024s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:56:38 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d5c66948-68fa-4f14-af17-6c02f54c0199]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:56:38 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-df33f55e-8108-4993-936e-fc46d3909473 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'qankzYOy'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:56:38 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7d4f3c7d-7d2c-480f-8bc1-fe2ed2e58ecf -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:56:38 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7d4f3c7d-7d2c-480f-8bc1-fe2ed2e58ecf -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:56:38 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[f6987f80-9ae7-4ec4-9461-7436c72549a1]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:56:38 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-df33f55e-8108-4993-936e-fc46d3909473 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:56:38 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:56:38 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 21 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:56:38 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[f489606f-4004-46db-ad71-1353e082c704]: (4, ('', 'iscsiadm: No active sessions.\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:56:38 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-df33f55e-8108-4993-936e-fc46d3909473 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsiadm ('-m', 'session'): stdout= stderr=iscsiadm: No active sessions.
Aug 09 02:56:38 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:56:38 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-df33f55e-8108-4993-936e-fc46d3909473 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsi session list stdout= stderr=iscsiadm: No active sessions.
Aug 09 02:56:38 user nova-compute[72351]:  {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:56:38 user nova-compute[72351]: WARNING os_brick.initiator.connectors.iscsi [None req-df33f55e-8108-4993-936e-fc46d3909473 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsiadm stderr output when getting sessions: iscsiadm: No active sessions.
Aug 09 02:56:38 user nova-compute[72351]: 
Aug 09 02:56:38 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7d4f3c7d-7d2c-480f-8bc1-fe2ed2e58ecf -p 172.16.0.220:3260 --login {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:56:38 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7d4f3c7d-7d2c-480f-8bc1-fe2ed2e58ecf -p 172.16.0.220:3260 --login" returned: 0 in 0.048s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:56:38 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ef5f4aa0-c8fa-4588-9163-ea784176ecd0]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:7d4f3c7d-7d2c-480f-8bc1-fe2ed2e58ecf, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:7d4f3c7d-7d2c-480f-8bc1-fe2ed2e58ecf, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:56:38 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-df33f55e-8108-4993-936e-fc46d3909473 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:7d4f3c7d-7d2c-480f-8bc1-fe2ed2e58ecf, portal: 172.16.0.220,3260]
Aug 09 02:56:38 user nova-compute[72351]: Login to [iface: default, target: iqn.2010-10.org.openstack:7d4f3c7d-7d2c-480f-8bc1-fe2ed2e58ecf, portal: 172.16.0.220,3260] successful.
Aug 09 02:56:38 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:56:38 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7d4f3c7d-7d2c-480f-8bc1-fe2ed2e58ecf -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:56:38 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7d4f3c7d-7d2c-480f-8bc1-fe2ed2e58ecf -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:56:38 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a3b14720-d399-4bf6-a699-72d4ee2ce1c2]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:56:38 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-df33f55e-8108-4993-936e-fc46d3909473 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:56:38 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:56:38 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.026s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:56:38 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[8c9bf14a-55dd-47e5-9a67-7af49b2898ba]: (4, ('tcp: [62] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7d4f3c7d-7d2c-480f-8bc1-fe2ed2e58ecf (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:56:38 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-df33f55e-8108-4993-936e-fc46d3909473 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [62] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7d4f3c7d-7d2c-480f-8bc1-fe2ed2e58ecf (non-flash)
Aug 09 02:56:38 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:56:38 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-df33f55e-8108-4993-936e-fc46d3909473 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsi session list stdout=tcp: [62] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7d4f3c7d-7d2c-480f-8bc1-fe2ed2e58ecf (non-flash)
Aug 09 02:56:38 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:56:38 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-df33f55e-8108-4993-936e-fc46d3909473 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:7d4f3c7d-7d2c-480f-8bc1-fe2ed2e58ecf" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.371s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:56:38 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-df33f55e-8108-4993-936e-fc46d3909473 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 02:56:38 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-df33f55e-8108-4993-936e-fc46d3909473 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] HCTL ('33', '-', '-', 0) found on session 62 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 02:56:38 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-df33f55e-8108-4993-936e-fc46d3909473 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 02:56:38 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:56:38 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:56:38 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[9922b1f3-1c2e-4d19-9f98-0ff7cd8afca8]: (4, ('- - 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:56:38 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-df33f55e-8108-4993-936e-fc46d3909473 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Searching for a device in session 62 and hctl ['33', '*', '*', 0] yield: None {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:56:39 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-df33f55e-8108-4993-936e-fc46d3909473 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Searching for a device in session 62 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:56:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-df33f55e-8108-4993-936e-fc46d3909473 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Connected to sdb using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'qankzYOy', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 02:56:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-df33f55e-8108-4993-936e-fc46d3909473 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.406s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:56:39 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-df33f55e-8108-4993-936e-fc46d3909473 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] <== connect_volume: return (1408ms) {'type': 'block', 'scsi_wwn': '26264333832616563', 'path': '/dev/sdb'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:56:39 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-df33f55e-8108-4993-936e-fc46d3909473 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26264333832616563', 'path': '/dev/sdb'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 02:56:39 user nova-compute[72351]: DEBUG nova.objects.instance [None req-df33f55e-8108-4993-936e-fc46d3909473 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lazy-loading 'flavor' on Instance uuid ccc96aeb-4de7-4511-a668-e3dee04e22f5 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:56:39 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-df33f55e-8108-4993-936e-fc46d3909473 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] attach device xml: <disk type="block" device="disk">
Aug 09 02:56:39 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:56:39 user nova-compute[72351]:   <source dev="/dev/sdb"/>
Aug 09 02:56:39 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:56:39 user nova-compute[72351]:   <serial>7d4f3c7d-7d2c-480f-8bc1-fe2ed2e58ecf</serial>
Aug 09 02:56:39 user nova-compute[72351]: </disk>
Aug 09 02:56:39 user nova-compute[72351]:  {{(pid=72351) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Aug 09 02:56:39 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-df33f55e-8108-4993-936e-fc46d3909473 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:56:39 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-df33f55e-8108-4993-936e-fc46d3909473 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:56:39 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-df33f55e-8108-4993-936e-fc46d3909473 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] No VIF found with MAC fa:16:3e:a4:6f:41, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:56:39 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-df33f55e-8108-4993-936e-fc46d3909473 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "ccc96aeb-4de7-4511-a668-e3dee04e22f5" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 9.920s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:56:42 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:56:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:56:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-8426d1be-dcbd-49d4-90e5-67b5df35d7d7 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Acquiring lock "ccc96aeb-4de7-4511-a668-e3dee04e22f5" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:56:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-8426d1be-dcbd-49d4-90e5-67b5df35d7d7 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "ccc96aeb-4de7-4511-a668-e3dee04e22f5" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:56:44 user nova-compute[72351]: DEBUG nova.objects.instance [None req-8426d1be-dcbd-49d4-90e5-67b5df35d7d7 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lazy-loading 'flavor' on Instance uuid ccc96aeb-4de7-4511-a668-e3dee04e22f5 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:56:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-8426d1be-dcbd-49d4-90e5-67b5df35d7d7 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "ccc96aeb-4de7-4511-a668-e3dee04e22f5" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.065s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:56:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-8426d1be-dcbd-49d4-90e5-67b5df35d7d7 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Acquiring lock "ccc96aeb-4de7-4511-a668-e3dee04e22f5" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:56:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-8426d1be-dcbd-49d4-90e5-67b5df35d7d7 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "ccc96aeb-4de7-4511-a668-e3dee04e22f5" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:56:44 user nova-compute[72351]: INFO nova.compute.manager [None req-8426d1be-dcbd-49d4-90e5-67b5df35d7d7 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Attaching volume b0933701-a6a9-44db-8e5e-be76f4c96de6 to /dev/vdc
Aug 09 02:56:44 user nova-compute[72351]: DEBUG os_brick.utils [None req-8426d1be-dcbd-49d4-90e5-67b5df35d7d7 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:56:44 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:56:44 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:56:44 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[2d75bd0f-68ef-4e9b-a8af-627fabc51dd6]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:a52269602ff4\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:56:44 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:56:44 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:56:44 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[bf2c8dc1-9c8c-4b71-bec2-079bcd3db69d]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:56:44 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:56:44 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:56:44 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[f665cc8e-0574-4e8e-b564-b3bfe53edc1e]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:56:44 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[247d779a-7af7-4668-997a-840fb03ed843]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:56:44 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-8426d1be-dcbd-49d4-90e5-67b5df35d7d7 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Running cmd (subprocess): nvme version {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:56:44 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-8426d1be-dcbd-49d4-90e5-67b5df35d7d7 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] 'nvme version' failed. Not Retrying. {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:56:44 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.nvmeof [None req-8426d1be-dcbd-49d4-90e5-67b5df35d7d7 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] nvme not present on system {{(pid=72351) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Aug 09 02:56:44 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:56:44 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 02:56:44 user nova-compute[72351]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Aug 09 02:56:44 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[81197296-f4fa-47d0-9abf-0c35ea36c545]: (4, '') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:56:44 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-8426d1be-dcbd-49d4-90e5-67b5df35d7d7 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72351) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Aug 09 02:56:44 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-8426d1be-dcbd-49d4-90e5-67b5df35d7d7 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Aug 09 02:56:44 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-8426d1be-dcbd-49d4-90e5-67b5df35d7d7 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] LIGHTOS: no hostnqn found. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Aug 09 02:56:44 user nova-compute[72351]: DEBUG os_brick.utils [None req-8426d1be-dcbd-49d4-90e5-67b5df35d7d7 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] <== get_connector_properties: return (139ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:a52269602ff4', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:56:44 user nova-compute[72351]: DEBUG nova.virt.block_device [None req-8426d1be-dcbd-49d4-90e5-67b5df35d7d7 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Updating existing volume attachment record: 0b221721-a1b0-4417-8041-8bf671918f72 {{(pid=72351) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Aug 09 02:56:47 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:56:47 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:56:52 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:56:53 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-8426d1be-dcbd-49d4-90e5-67b5df35d7d7 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 02:56:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-8426d1be-dcbd-49d4-90e5-67b5df35d7d7 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:b0933701-a6a9-44db-8e5e-be76f4c96de6'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'ReiFGxhn', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:56:53 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-8426d1be-dcbd-49d4-90e5-67b5df35d7d7 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:56:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-8426d1be-dcbd-49d4-90e5-67b5df35d7d7 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:56:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-8426d1be-dcbd-49d4-90e5-67b5df35d7d7 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:56:53 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-8426d1be-dcbd-49d4-90e5-67b5df35d7d7 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:56:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-8426d1be-dcbd-49d4-90e5-67b5df35d7d7 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:b0933701-a6a9-44db-8e5e-be76f4c96de6" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:56:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-8426d1be-dcbd-49d4-90e5-67b5df35d7d7 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:b0933701-a6a9-44db-8e5e-be76f4c96de6" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:56:53 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-8426d1be-dcbd-49d4-90e5-67b5df35d7d7 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 02:56:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b0933701-a6a9-44db-8e5e-be76f4c96de6 -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:56:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b0933701-a6a9-44db-8e5e-be76f4c96de6 -p 172.16.0.220:3260" returned: 21 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:56:53 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e45db3f6-8869-41ca-bc2c-9e3a8b0943f0]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:56:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-8426d1be-dcbd-49d4-90e5-67b5df35d7d7 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Aug 09 02:56:53 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:56:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b0933701-a6a9-44db-8e5e-be76f4c96de6 -p 172.16.0.220:3260 --interface default --op new {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:56:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b0933701-a6a9-44db-8e5e-be76f4c96de6 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:56:53 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ecef7e01-9413-41ef-a5dc-8b15068aba1c]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:b0933701-a6a9-44db-8e5e-be76f4c96de6] added\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:56:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-8426d1be-dcbd-49d4-90e5-67b5df35d7d7 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:b0933701-a6a9-44db-8e5e-be76f4c96de6] added
Aug 09 02:56:53 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:56:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b0933701-a6a9-44db-8e5e-be76f4c96de6 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:56:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b0933701-a6a9-44db-8e5e-be76f4c96de6 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:56:53 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[6ec0f305-59ef-4fe3-a427-899023874e7b]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:56:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-8426d1be-dcbd-49d4-90e5-67b5df35d7d7 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:56:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b0933701-a6a9-44db-8e5e-be76f4c96de6 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:56:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b0933701-a6a9-44db-8e5e-be76f4c96de6 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:56:53 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[57bb4657-5f5d-43cd-bf6b-47aec75efede]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:56:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-8426d1be-dcbd-49d4-90e5-67b5df35d7d7 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:56:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b0933701-a6a9-44db-8e5e-be76f4c96de6 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v ReiFGxhn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:56:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b0933701-a6a9-44db-8e5e-be76f4c96de6 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v ReiFGxhn" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:56:53 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[13d53e0a-6da3-4b13-b251-b0c89b0acbbe]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:56:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-8426d1be-dcbd-49d4-90e5-67b5df35d7d7 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'ReiFGxhn'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:56:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b0933701-a6a9-44db-8e5e-be76f4c96de6 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:56:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b0933701-a6a9-44db-8e5e-be76f4c96de6 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:56:53 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[06fbd5f2-3e2c-4e3b-8284-da034eba8c20]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:56:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-8426d1be-dcbd-49d4-90e5-67b5df35d7d7 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:56:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:56:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:56:53 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[2ee88bc4-b510-4034-a94e-77e44f03cf2b]: (4, ('tcp: [62] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7d4f3c7d-7d2c-480f-8bc1-fe2ed2e58ecf (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:56:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-8426d1be-dcbd-49d4-90e5-67b5df35d7d7 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [62] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7d4f3c7d-7d2c-480f-8bc1-fe2ed2e58ecf (non-flash)
Aug 09 02:56:53 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:56:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-8426d1be-dcbd-49d4-90e5-67b5df35d7d7 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsi session list stdout=tcp: [62] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7d4f3c7d-7d2c-480f-8bc1-fe2ed2e58ecf (non-flash)
Aug 09 02:56:53 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:56:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b0933701-a6a9-44db-8e5e-be76f4c96de6 -p 172.16.0.220:3260 --login {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:56:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b0933701-a6a9-44db-8e5e-be76f4c96de6 -p 172.16.0.220:3260 --login" returned: 0 in 0.048s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:56:53 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[018bc5e1-ebe2-4eb3-8a25-a483ccdf418c]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:b0933701-a6a9-44db-8e5e-be76f4c96de6, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:b0933701-a6a9-44db-8e5e-be76f4c96de6, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:56:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-8426d1be-dcbd-49d4-90e5-67b5df35d7d7 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:b0933701-a6a9-44db-8e5e-be76f4c96de6, portal: 172.16.0.220,3260]
Aug 09 02:56:53 user nova-compute[72351]: Login to [iface: default, target: iqn.2010-10.org.openstack:b0933701-a6a9-44db-8e5e-be76f4c96de6, portal: 172.16.0.220,3260] successful.
Aug 09 02:56:53 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:56:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b0933701-a6a9-44db-8e5e-be76f4c96de6 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:56:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b0933701-a6a9-44db-8e5e-be76f4c96de6 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:56:53 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e4a1c59b-1448-4b1f-a821-a030427e80e9]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:56:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-8426d1be-dcbd-49d4-90e5-67b5df35d7d7 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:56:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:56:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:56:53 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d7112bbc-6be5-47a4-87a8-36d0ffa8b117]: (4, ('tcp: [62] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7d4f3c7d-7d2c-480f-8bc1-fe2ed2e58ecf (non-flash)\ntcp: [63] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b0933701-a6a9-44db-8e5e-be76f4c96de6 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:56:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-8426d1be-dcbd-49d4-90e5-67b5df35d7d7 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [62] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7d4f3c7d-7d2c-480f-8bc1-fe2ed2e58ecf (non-flash)
Aug 09 02:56:53 user nova-compute[72351]: tcp: [63] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b0933701-a6a9-44db-8e5e-be76f4c96de6 (non-flash)
Aug 09 02:56:53 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:56:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-8426d1be-dcbd-49d4-90e5-67b5df35d7d7 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsi session list stdout=tcp: [62] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7d4f3c7d-7d2c-480f-8bc1-fe2ed2e58ecf (non-flash)
Aug 09 02:56:53 user nova-compute[72351]: tcp: [63] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b0933701-a6a9-44db-8e5e-be76f4c96de6 (non-flash)
Aug 09 02:56:53 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:56:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-8426d1be-dcbd-49d4-90e5-67b5df35d7d7 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:b0933701-a6a9-44db-8e5e-be76f4c96de6" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.250s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:56:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-8426d1be-dcbd-49d4-90e5-67b5df35d7d7 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 02:56:53 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-8426d1be-dcbd-49d4-90e5-67b5df35d7d7 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] HCTL ('34', '-', '-', 0) found on session 63 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 02:56:53 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-8426d1be-dcbd-49d4-90e5-67b5df35d7d7 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Scanning host 34 c: -, t: -, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 02:56:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:56:53 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:56:53 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[5676c2da-ac30-417f-8f24-a76761a54b33]: (4, ('- - 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:56:53 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-8426d1be-dcbd-49d4-90e5-67b5df35d7d7 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Searching for a device in session 63 and hctl ['34', '*', '*', 0] yield: sdc {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 02:56:53 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-8426d1be-dcbd-49d4-90e5-67b5df35d7d7 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Connected to sdc using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'ReiFGxhn', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 02:56:54 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-8426d1be-dcbd-49d4-90e5-67b5df35d7d7 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.281s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:56:54 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-8426d1be-dcbd-49d4-90e5-67b5df35d7d7 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] <== connect_volume: return (1283ms) {'type': 'block', 'scsi_wwn': '23431663438323561', 'path': '/dev/sdc'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:56:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-8426d1be-dcbd-49d4-90e5-67b5df35d7d7 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23431663438323561', 'path': '/dev/sdc'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 02:56:54 user nova-compute[72351]: DEBUG nova.objects.instance [None req-8426d1be-dcbd-49d4-90e5-67b5df35d7d7 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lazy-loading 'flavor' on Instance uuid ccc96aeb-4de7-4511-a668-e3dee04e22f5 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:56:54 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-8426d1be-dcbd-49d4-90e5-67b5df35d7d7 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] attach device xml: <disk type="block" device="disk">
Aug 09 02:56:54 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:56:54 user nova-compute[72351]:   <source dev="/dev/sdc"/>
Aug 09 02:56:54 user nova-compute[72351]:   <target dev="vdc" bus="virtio"/>
Aug 09 02:56:54 user nova-compute[72351]:   <serial>b0933701-a6a9-44db-8e5e-be76f4c96de6</serial>
Aug 09 02:56:54 user nova-compute[72351]: </disk>
Aug 09 02:56:54 user nova-compute[72351]:  {{(pid=72351) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Aug 09 02:56:55 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-8426d1be-dcbd-49d4-90e5-67b5df35d7d7 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:56:55 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-8426d1be-dcbd-49d4-90e5-67b5df35d7d7 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:56:55 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-8426d1be-dcbd-49d4-90e5-67b5df35d7d7 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] No BDM found with device name vdc, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 02:56:55 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-8426d1be-dcbd-49d4-90e5-67b5df35d7d7 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] No VIF found with MAC fa:16:3e:a4:6f:41, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 02:56:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-8426d1be-dcbd-49d4-90e5-67b5df35d7d7 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "ccc96aeb-4de7-4511-a668-e3dee04e22f5" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 10.519s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:56:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-213f0c84-2aac-471b-a84c-76a6aebbd53e tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Acquiring lock "ccc96aeb-4de7-4511-a668-e3dee04e22f5" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:56:55 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-213f0c84-2aac-471b-a84c-76a6aebbd53e tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "ccc96aeb-4de7-4511-a668-e3dee04e22f5" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:56:55 user nova-compute[72351]: INFO nova.compute.manager [None req-213f0c84-2aac-471b-a84c-76a6aebbd53e tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Detaching volume 7d4f3c7d-7d2c-480f-8bc1-fe2ed2e58ecf
Aug 09 02:56:55 user nova-compute[72351]: INFO nova.virt.block_device [None req-213f0c84-2aac-471b-a84c-76a6aebbd53e tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Attempting to driver detach volume 7d4f3c7d-7d2c-480f-8bc1-fe2ed2e58ecf from mountpoint /dev/vdb
Aug 09 02:56:55 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-213f0c84-2aac-471b-a84c-76a6aebbd53e tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Attempting to detach device vdb from instance ccc96aeb-4de7-4511-a668-e3dee04e22f5 from the persistent domain config. {{(pid=72351) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2477}}
Aug 09 02:56:55 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-213f0c84-2aac-471b-a84c-76a6aebbd53e tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 02:56:55 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:56:55 user nova-compute[72351]:   <source dev="/dev/sdb"/>
Aug 09 02:56:55 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:56:55 user nova-compute[72351]:   <serial>7d4f3c7d-7d2c-480f-8bc1-fe2ed2e58ecf</serial>
Aug 09 02:56:55 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Aug 09 02:56:55 user nova-compute[72351]: </disk>
Aug 09 02:56:55 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 02:56:55 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-213f0c84-2aac-471b-a84c-76a6aebbd53e tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Successfully detached device vdb from instance ccc96aeb-4de7-4511-a668-e3dee04e22f5 from the persistent domain config.
Aug 09 02:56:55 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-213f0c84-2aac-471b-a84c-76a6aebbd53e tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance ccc96aeb-4de7-4511-a668-e3dee04e22f5 from the live domain config. {{(pid=72351) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2513}}
Aug 09 02:56:55 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-213f0c84-2aac-471b-a84c-76a6aebbd53e tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 02:56:55 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:56:55 user nova-compute[72351]:   <source dev="/dev/sdb"/>
Aug 09 02:56:55 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 02:56:55 user nova-compute[72351]:   <serial>7d4f3c7d-7d2c-480f-8bc1-fe2ed2e58ecf</serial>
Aug 09 02:56:55 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Aug 09 02:56:55 user nova-compute[72351]: </disk>
Aug 09 02:56:55 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 02:56:56 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Received event <DeviceRemovedEvent: 1691549816.4872692, ccc96aeb-4de7-4511-a668-e3dee04e22f5 => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=72351) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2360}}
Aug 09 02:56:56 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-213f0c84-2aac-471b-a84c-76a6aebbd53e tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance ccc96aeb-4de7-4511-a668-e3dee04e22f5 {{(pid=72351) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2589}}
Aug 09 02:56:56 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-213f0c84-2aac-471b-a84c-76a6aebbd53e tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Successfully detached device vdb from instance ccc96aeb-4de7-4511-a668-e3dee04e22f5 from the live domain config.
Aug 09 02:56:56 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-213f0c84-2aac-471b-a84c-76a6aebbd53e tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 02:56:56 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-213f0c84-2aac-471b-a84c-76a6aebbd53e tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:7d4f3c7d-7d2c-480f-8bc1-fe2ed2e58ecf'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'qankzYOy', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb'}, None), 'kwargs': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:56:56 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-213f0c84-2aac-471b-a84c-76a6aebbd53e tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:56:56 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-213f0c84-2aac-471b-a84c-76a6aebbd53e tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:56:56 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-213f0c84-2aac-471b-a84c-76a6aebbd53e tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:56:56 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-213f0c84-2aac-471b-a84c-76a6aebbd53e tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:7d4f3c7d-7d2c-480f-8bc1-fe2ed2e58ecf', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 02:56:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:56:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:56:56 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ceb38227-e949-4bf9-99df-b6bc2d63ec5f]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:7d4f3c7d-7d2c-480f-8bc1-fe2ed2e58ecf\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:b0933701-a6a9-44db-8e5e-be76f4c96de6\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:56:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:56:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:56:56 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[220e3677-1ab8-4715-97e3-027ce4a5813e]: (4, ('tcp: [62] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7d4f3c7d-7d2c-480f-8bc1-fe2ed2e58ecf (non-flash)\ntcp: [63] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b0933701-a6a9-44db-8e5e-be76f4c96de6 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:56:56 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-213f0c84-2aac-471b-a84c-76a6aebbd53e tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [62] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7d4f3c7d-7d2c-480f-8bc1-fe2ed2e58ecf (non-flash)
Aug 09 02:56:56 user nova-compute[72351]: tcp: [63] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b0933701-a6a9-44db-8e5e-be76f4c96de6 (non-flash)
Aug 09 02:56:56 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:56:56 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-213f0c84-2aac-471b-a84c-76a6aebbd53e tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsi session list stdout=tcp: [62] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7d4f3c7d-7d2c-480f-8bc1-fe2ed2e58ecf (non-flash)
Aug 09 02:56:56 user nova-compute[72351]: tcp: [63] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b0933701-a6a9-44db-8e5e-be76f4c96de6 (non-flash)
Aug 09 02:56:56 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:56:56 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-213f0c84-2aac-471b-a84c-76a6aebbd53e tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fcfbc52c280>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:7d4f3c7d-7d2c-480f-8bc1-fe2ed2e58ecf'): ({'sdb'}, set())}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 02:56:56 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-213f0c84-2aac-471b-a84c-76a6aebbd53e tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Removing single pathed devices sdb {{(pid=72351) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Aug 09 02:56:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:56:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:56:56 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[788aeab1-b4e7-4322-8582-a4e3e513141e]: (4, ('path checker states:\nup                  2\n\npaths: 0\nbusy: False\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:56:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:56:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:56:56 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[09514553-4d52-4815-9aa3-3b732599778b]: (4, ('ok\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:56:56 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-213f0c84-2aac-471b-a84c-76a6aebbd53e tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Flushing IO for device /dev/sdb {{(pid=72351) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Aug 09 02:56:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:56:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.013s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:56:56 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[1d1f8280-d610-4bf9-b794-8f529f406d7f]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:56:56 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-213f0c84-2aac-471b-a84c-76a6aebbd53e tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=72351) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Aug 09 02:56:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:56:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.038s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:56:56 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[960d60c8-6839-45c2-9c6f-79be0f32ca86]: (4, ('1', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:56:56 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-213f0c84-2aac-471b-a84c-76a6aebbd53e tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Aug 09 02:56:56 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-213f0c84-2aac-471b-a84c-76a6aebbd53e tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] SCSI volumes sdb have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Aug 09 02:56:56 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-213f0c84-2aac-471b-a84c-76a6aebbd53e tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:7d4f3c7d-7d2c-480f-8bc1-fe2ed2e58ecf')] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 02:56:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7d4f3c7d-7d2c-480f-8bc1-fe2ed2e58ecf -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:56:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7d4f3c7d-7d2c-480f-8bc1-fe2ed2e58ecf -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:56:56 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[2a5438e5-4aaa-455a-9e40-668e016b5111]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:56:56 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-213f0c84-2aac-471b-a84c-76a6aebbd53e tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:56:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7d4f3c7d-7d2c-480f-8bc1-fe2ed2e58ecf -p 172.16.0.220:3260 --logout {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:56:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7d4f3c7d-7d2c-480f-8bc1-fe2ed2e58ecf -p 172.16.0.220:3260 --logout" returned: 0 in 0.058s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:56:56 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[fae1df64-471a-40ad-9ccf-3169d2a9f1ed]: (4, ('Logging out of session [sid: 62, target: iqn.2010-10.org.openstack:7d4f3c7d-7d2c-480f-8bc1-fe2ed2e58ecf, portal: 172.16.0.220,3260]\nLogout of [sid: 62, target: iqn.2010-10.org.openstack:7d4f3c7d-7d2c-480f-8bc1-fe2ed2e58ecf, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:56:56 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-213f0c84-2aac-471b-a84c-76a6aebbd53e tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 62, target: iqn.2010-10.org.openstack:7d4f3c7d-7d2c-480f-8bc1-fe2ed2e58ecf, portal: 172.16.0.220,3260]
Aug 09 02:56:56 user nova-compute[72351]: Logout of [sid: 62, target: iqn.2010-10.org.openstack:7d4f3c7d-7d2c-480f-8bc1-fe2ed2e58ecf, portal: 172.16.0.220,3260] successful.
Aug 09 02:56:56 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:56:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7d4f3c7d-7d2c-480f-8bc1-fe2ed2e58ecf -p 172.16.0.220:3260 --op delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:56:56 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7d4f3c7d-7d2c-480f-8bc1-fe2ed2e58ecf -p 172.16.0.220:3260 --op delete" returned: 0 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:56:56 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[8804eb3a-f2dc-4c92-9112-c618b6aa2ab7]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:56:56 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-213f0c84-2aac-471b-a84c-76a6aebbd53e tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:56:56 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-213f0c84-2aac-471b-a84c-76a6aebbd53e tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.268s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:56:56 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-213f0c84-2aac-471b-a84c-76a6aebbd53e tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] <== disconnect_volume: return (269ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:56:56 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-213f0c84-2aac-471b-a84c-76a6aebbd53e tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 02:56:57 user nova-compute[72351]: DEBUG nova.objects.instance [None req-213f0c84-2aac-471b-a84c-76a6aebbd53e tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lazy-loading 'flavor' on Instance uuid ccc96aeb-4de7-4511-a668-e3dee04e22f5 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:56:57 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-213f0c84-2aac-471b-a84c-76a6aebbd53e tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "ccc96aeb-4de7-4511-a668-e3dee04e22f5" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.637s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:56:57 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:56:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1976dc93-92f1-4a1a-80c5-5313aecc07fb tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Acquiring lock "ccc96aeb-4de7-4511-a668-e3dee04e22f5" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:56:58 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1976dc93-92f1-4a1a-80c5-5313aecc07fb tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "ccc96aeb-4de7-4511-a668-e3dee04e22f5" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:56:58 user nova-compute[72351]: INFO nova.compute.manager [None req-1976dc93-92f1-4a1a-80c5-5313aecc07fb tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Detaching volume b0933701-a6a9-44db-8e5e-be76f4c96de6
Aug 09 02:56:58 user nova-compute[72351]: INFO nova.virt.block_device [None req-1976dc93-92f1-4a1a-80c5-5313aecc07fb tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Attempting to driver detach volume b0933701-a6a9-44db-8e5e-be76f4c96de6 from mountpoint /dev/vdc
Aug 09 02:56:58 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-1976dc93-92f1-4a1a-80c5-5313aecc07fb tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Attempting to detach device vdc from instance ccc96aeb-4de7-4511-a668-e3dee04e22f5 from the persistent domain config. {{(pid=72351) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2477}}
Aug 09 02:56:58 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-1976dc93-92f1-4a1a-80c5-5313aecc07fb tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 02:56:58 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:56:58 user nova-compute[72351]:   <source dev="/dev/sdc"/>
Aug 09 02:56:58 user nova-compute[72351]:   <target dev="vdc" bus="virtio"/>
Aug 09 02:56:58 user nova-compute[72351]:   <serial>b0933701-a6a9-44db-8e5e-be76f4c96de6</serial>
Aug 09 02:56:58 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x08" function="0x0"/>
Aug 09 02:56:58 user nova-compute[72351]: </disk>
Aug 09 02:56:58 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 02:56:58 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-1976dc93-92f1-4a1a-80c5-5313aecc07fb tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Successfully detached device vdc from instance ccc96aeb-4de7-4511-a668-e3dee04e22f5 from the persistent domain config.
Aug 09 02:56:58 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-1976dc93-92f1-4a1a-80c5-5313aecc07fb tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] (1/8): Attempting to detach device vdc with device alias virtio-disk2 from instance ccc96aeb-4de7-4511-a668-e3dee04e22f5 from the live domain config. {{(pid=72351) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2513}}
Aug 09 02:56:58 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-1976dc93-92f1-4a1a-80c5-5313aecc07fb tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 02:56:58 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 02:56:58 user nova-compute[72351]:   <source dev="/dev/sdc"/>
Aug 09 02:56:58 user nova-compute[72351]:   <target dev="vdc" bus="virtio"/>
Aug 09 02:56:58 user nova-compute[72351]:   <serial>b0933701-a6a9-44db-8e5e-be76f4c96de6</serial>
Aug 09 02:56:58 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x08" function="0x0"/>
Aug 09 02:56:58 user nova-compute[72351]: </disk>
Aug 09 02:56:58 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 02:56:58 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Received event <DeviceRemovedEvent: 1691549818.7645001, ccc96aeb-4de7-4511-a668-e3dee04e22f5 => virtio-disk2> from libvirt while the driver is waiting for it; dispatched. {{(pid=72351) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2360}}
Aug 09 02:56:58 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-1976dc93-92f1-4a1a-80c5-5313aecc07fb tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Start waiting for the detach event from libvirt for device vdc with device alias virtio-disk2 for instance ccc96aeb-4de7-4511-a668-e3dee04e22f5 {{(pid=72351) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2589}}
Aug 09 02:56:58 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-1976dc93-92f1-4a1a-80c5-5313aecc07fb tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Successfully detached device vdc from instance ccc96aeb-4de7-4511-a668-e3dee04e22f5 from the live domain config.
Aug 09 02:56:58 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-1976dc93-92f1-4a1a-80c5-5313aecc07fb tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 02:56:58 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1976dc93-92f1-4a1a-80c5-5313aecc07fb tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:b0933701-a6a9-44db-8e5e-be76f4c96de6'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'ReiFGxhn', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc'}, None), 'kwargs': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 02:56:58 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-1976dc93-92f1-4a1a-80c5-5313aecc07fb tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 02:56:58 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-1976dc93-92f1-4a1a-80c5-5313aecc07fb tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 02:56:58 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-1976dc93-92f1-4a1a-80c5-5313aecc07fb tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 02:56:58 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1976dc93-92f1-4a1a-80c5-5313aecc07fb tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:b0933701-a6a9-44db-8e5e-be76f4c96de6', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 02:56:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:56:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:56:58 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d23c7282-7846-4ddc-8276-548042d309be]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:b0933701-a6a9-44db-8e5e-be76f4c96de6\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:56:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:56:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:56:58 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[37f3cc19-95ac-4932-8c9f-d6d0d21d1bc1]: (4, ('tcp: [63] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b0933701-a6a9-44db-8e5e-be76f4c96de6 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:56:58 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1976dc93-92f1-4a1a-80c5-5313aecc07fb tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [63] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b0933701-a6a9-44db-8e5e-be76f4c96de6 (non-flash)
Aug 09 02:56:58 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 02:56:58 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1976dc93-92f1-4a1a-80c5-5313aecc07fb tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsi session list stdout=tcp: [63] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b0933701-a6a9-44db-8e5e-be76f4c96de6 (non-flash)
Aug 09 02:56:58 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 02:56:58 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1976dc93-92f1-4a1a-80c5-5313aecc07fb tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fcfbc552cb0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:b0933701-a6a9-44db-8e5e-be76f4c96de6'): ({'sdc'}, set())}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 02:56:58 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-1976dc93-92f1-4a1a-80c5-5313aecc07fb tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Removing single pathed devices sdc {{(pid=72351) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Aug 09 02:56:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:56:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:56:58 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[116a87b3-d047-4be0-95d2-c5b6064bcd0e]: (4, ('path checker states:\nup                  1\n\npaths: 0\nbusy: False\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:56:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdc {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:56:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdc" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:56:58 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[230bc0f2-366f-4045-a252-d29853ccac90]: (4, ('ok\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:56:58 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-1976dc93-92f1-4a1a-80c5-5313aecc07fb tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Flushing IO for device /dev/sdc {{(pid=72351) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Aug 09 02:56:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdc {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:56:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdc" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:56:58 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[cca02cb7-015c-4cc9-a6d5-dc0487f79a7f]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:56:58 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-1976dc93-92f1-4a1a-80c5-5313aecc07fb tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Remove SCSI device /dev/sdc with /sys/block/sdc/device/delete {{(pid=72351) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Aug 09 02:56:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdc/device/delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:56:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdc/device/delete" returned: 0 in 0.039s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:56:58 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[0441c349-a322-436e-981e-41a02d60e1eb]: (4, ('1', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:56:58 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-1976dc93-92f1-4a1a-80c5-5313aecc07fb tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Checking to see if SCSI volumes sdc have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Aug 09 02:56:58 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-1976dc93-92f1-4a1a-80c5-5313aecc07fb tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] SCSI volumes sdc have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Aug 09 02:56:58 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1976dc93-92f1-4a1a-80c5-5313aecc07fb tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:b0933701-a6a9-44db-8e5e-be76f4c96de6')] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 02:56:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b0933701-a6a9-44db-8e5e-be76f4c96de6 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:56:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b0933701-a6a9-44db-8e5e-be76f4c96de6 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:56:58 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[816d59a3-ab4f-4a15-9d96-198e52c3103b]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:56:58 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1976dc93-92f1-4a1a-80c5-5313aecc07fb tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:56:58 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b0933701-a6a9-44db-8e5e-be76f4c96de6 -p 172.16.0.220:3260 --logout {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:56:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b0933701-a6a9-44db-8e5e-be76f4c96de6 -p 172.16.0.220:3260 --logout" returned: 0 in 0.060s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:56:59 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e085f691-ffc2-4609-99a0-0e4fd66acf63]: (4, ('Logging out of session [sid: 63, target: iqn.2010-10.org.openstack:b0933701-a6a9-44db-8e5e-be76f4c96de6, portal: 172.16.0.220,3260]\nLogout of [sid: 63, target: iqn.2010-10.org.openstack:b0933701-a6a9-44db-8e5e-be76f4c96de6, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:56:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1976dc93-92f1-4a1a-80c5-5313aecc07fb tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 63, target: iqn.2010-10.org.openstack:b0933701-a6a9-44db-8e5e-be76f4c96de6, portal: 172.16.0.220,3260]
Aug 09 02:56:59 user nova-compute[72351]: Logout of [sid: 63, target: iqn.2010-10.org.openstack:b0933701-a6a9-44db-8e5e-be76f4c96de6, portal: 172.16.0.220,3260] successful.
Aug 09 02:56:59 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:56:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b0933701-a6a9-44db-8e5e-be76f4c96de6 -p 172.16.0.220:3260 --op delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 02:56:59 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b0933701-a6a9-44db-8e5e-be76f4c96de6 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.025s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 02:56:59 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[3a55b11a-8f86-403f-8fd9-d2f19dcc7147]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 02:56:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1976dc93-92f1-4a1a-80c5-5313aecc07fb tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 02:56:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-1976dc93-92f1-4a1a-80c5-5313aecc07fb tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.276s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 02:56:59 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1976dc93-92f1-4a1a-80c5-5313aecc07fb tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] <== disconnect_volume: return (277ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 02:56:59 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-1976dc93-92f1-4a1a-80c5-5313aecc07fb tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 02:56:59 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:56:59 user nova-compute[72351]: DEBUG nova.objects.instance [None req-1976dc93-92f1-4a1a-80c5-5313aecc07fb tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lazy-loading 'flavor' on Instance uuid ccc96aeb-4de7-4511-a668-e3dee04e22f5 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:56:59 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1976dc93-92f1-4a1a-80c5-5313aecc07fb tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "ccc96aeb-4de7-4511-a668-e3dee04e22f5" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.643s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:57:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-532c3cb8-fa21-46f1-8559-a206a1f4d037 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Acquiring lock "ccc96aeb-4de7-4511-a668-e3dee04e22f5" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:57:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-532c3cb8-fa21-46f1-8559-a206a1f4d037 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "ccc96aeb-4de7-4511-a668-e3dee04e22f5" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:57:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-532c3cb8-fa21-46f1-8559-a206a1f4d037 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Acquiring lock "ccc96aeb-4de7-4511-a668-e3dee04e22f5-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:57:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-532c3cb8-fa21-46f1-8559-a206a1f4d037 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "ccc96aeb-4de7-4511-a668-e3dee04e22f5-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:57:00 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-532c3cb8-fa21-46f1-8559-a206a1f4d037 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "ccc96aeb-4de7-4511-a668-e3dee04e22f5-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:57:00 user nova-compute[72351]: INFO nova.compute.manager [None req-532c3cb8-fa21-46f1-8559-a206a1f4d037 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Terminating instance
Aug 09 02:57:00 user nova-compute[72351]: DEBUG nova.compute.manager [None req-532c3cb8-fa21-46f1-8559-a206a1f4d037 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Start destroying the instance on the hypervisor. {{(pid=72351) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3124}}
Aug 09 02:57:00 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:57:01 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:57:01 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:57:01 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:57:01 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:57:01 user nova-compute[72351]: DEBUG nova.compute.manager [req-0de13601-50c5-4856-8c77-14c9be036e15 req-ce196bcd-86df-49bb-beb6-0066447f70b1 service nova] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Received event network-vif-unplugged-07ee9224-5c7b-4362-82a4-54731362c757 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:57:01 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-0de13601-50c5-4856-8c77-14c9be036e15 req-ce196bcd-86df-49bb-beb6-0066447f70b1 service nova] Acquiring lock "ccc96aeb-4de7-4511-a668-e3dee04e22f5-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:57:01 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-0de13601-50c5-4856-8c77-14c9be036e15 req-ce196bcd-86df-49bb-beb6-0066447f70b1 service nova] Lock "ccc96aeb-4de7-4511-a668-e3dee04e22f5-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:57:01 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-0de13601-50c5-4856-8c77-14c9be036e15 req-ce196bcd-86df-49bb-beb6-0066447f70b1 service nova] Lock "ccc96aeb-4de7-4511-a668-e3dee04e22f5-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:57:01 user nova-compute[72351]: DEBUG nova.compute.manager [req-0de13601-50c5-4856-8c77-14c9be036e15 req-ce196bcd-86df-49bb-beb6-0066447f70b1 service nova] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] No waiting events found dispatching network-vif-unplugged-07ee9224-5c7b-4362-82a4-54731362c757 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:57:01 user nova-compute[72351]: DEBUG nova.compute.manager [req-0de13601-50c5-4856-8c77-14c9be036e15 req-ce196bcd-86df-49bb-beb6-0066447f70b1 service nova] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Received event network-vif-unplugged-07ee9224-5c7b-4362-82a4-54731362c757 for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 02:57:01 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Instance destroyed successfully.
Aug 09 02:57:01 user nova-compute[72351]: DEBUG nova.objects.instance [None req-532c3cb8-fa21-46f1-8559-a206a1f4d037 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lazy-loading 'resources' on Instance uuid ccc96aeb-4de7-4511-a668-e3dee04e22f5 {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 02:57:01 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-532c3cb8-fa21-46f1-8559-a206a1f4d037 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T02:54:34Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachVolumeTestJSON-server-552342611',display_name='tempest-AttachVolumeTestJSON-server-552342611',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumetestjson-server-552342611',id=51,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBIUwHXpH2fJ3g3wNlmi7NgH+LwCmDsQaQHWhrNQiMNmpopZzyLyOCqErm6qlmfocX44iiHaCxZQak7p6ZXLZG4tz3AK9MrpUOgwFC/D5lYRrszrhgIM72DjhVn9gCRfaoQ==',key_name='tempest-keypair-1729558399',keypairs=<?>,launch_index=0,launched_at=2023-08-09T02:54:41Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='ec9861e0882e4065ab99ad3f74d6b25f',ramdisk_id='',reservation_id='r-i5wzpbxv',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeTestJSON-1725368857',owner_user_name='tempest-AttachVolumeTestJSON-1725368857-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T02:54:41Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='a2bc49dd654f4beca4b8eb5a49cd350d',uuid=ccc96aeb-4de7-4511-a668-e3dee04e22f5,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "07ee9224-5c7b-4362-82a4-54731362c757", "address": "fa:16:3e:a4:6f:41", "network": {"id": "dba57c3d-6efd-4057-8053-1c6a92bb24f8", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-568585773-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.182", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ec9861e0882e4065ab99ad3f74d6b25f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap07ee9224-5c", "ovs_interfaceid": "07ee9224-5c7b-4362-82a4-54731362c757", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 02:57:01 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-532c3cb8-fa21-46f1-8559-a206a1f4d037 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Converting VIF {"id": "07ee9224-5c7b-4362-82a4-54731362c757", "address": "fa:16:3e:a4:6f:41", "network": {"id": "dba57c3d-6efd-4057-8053-1c6a92bb24f8", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-568585773-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.182", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ec9861e0882e4065ab99ad3f74d6b25f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap07ee9224-5c", "ovs_interfaceid": "07ee9224-5c7b-4362-82a4-54731362c757", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 02:57:01 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-532c3cb8-fa21-46f1-8559-a206a1f4d037 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:a4:6f:41,bridge_name='br-int',has_traffic_filtering=True,id=07ee9224-5c7b-4362-82a4-54731362c757,network=Network(dba57c3d-6efd-4057-8053-1c6a92bb24f8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap07ee9224-5c') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 02:57:01 user nova-compute[72351]: DEBUG os_vif [None req-532c3cb8-fa21-46f1-8559-a206a1f4d037 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:a4:6f:41,bridge_name='br-int',has_traffic_filtering=True,id=07ee9224-5c7b-4362-82a4-54731362c757,network=Network(dba57c3d-6efd-4057-8053-1c6a92bb24f8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap07ee9224-5c') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 02:57:01 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:57:01 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap07ee9224-5c, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 02:57:01 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:57:01 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:57:01 user nova-compute[72351]: INFO os_vif [None req-532c3cb8-fa21-46f1-8559-a206a1f4d037 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:a4:6f:41,bridge_name='br-int',has_traffic_filtering=True,id=07ee9224-5c7b-4362-82a4-54731362c757,network=Network(dba57c3d-6efd-4057-8053-1c6a92bb24f8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap07ee9224-5c')
Aug 09 02:57:01 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-532c3cb8-fa21-46f1-8559-a206a1f4d037 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Deleting instance files /opt/stack/data/nova/instances/ccc96aeb-4de7-4511-a668-e3dee04e22f5_del
Aug 09 02:57:01 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-532c3cb8-fa21-46f1-8559-a206a1f4d037 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Deletion of /opt/stack/data/nova/instances/ccc96aeb-4de7-4511-a668-e3dee04e22f5_del complete
Aug 09 02:57:01 user nova-compute[72351]: INFO nova.compute.manager [None req-532c3cb8-fa21-46f1-8559-a206a1f4d037 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Took 0.86 seconds to destroy the instance on the hypervisor.
Aug 09 02:57:01 user nova-compute[72351]: DEBUG oslo.service.loopingcall [None req-532c3cb8-fa21-46f1-8559-a206a1f4d037 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72351) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Aug 09 02:57:01 user nova-compute[72351]: DEBUG nova.compute.manager [-] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Deallocating network for instance {{(pid=72351) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2263}}
Aug 09 02:57:01 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] deallocate_for_instance() {{(pid=72351) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1795}}
Aug 09 02:57:02 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:57:02 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Took 1.06 seconds to deallocate network for instance.
Aug 09 02:57:02 user nova-compute[72351]: DEBUG nova.compute.manager [req-f308b2aa-7ee8-423b-a627-0695cb5d6a29 req-5839063f-5a31-414f-976e-73e702a8ce32 service nova] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Received event network-vif-deleted-07ee9224-5c7b-4362-82a4-54731362c757 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:57:02 user nova-compute[72351]: INFO nova.compute.manager [req-f308b2aa-7ee8-423b-a627-0695cb5d6a29 req-5839063f-5a31-414f-976e-73e702a8ce32 service nova] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Neutron deleted interface 07ee9224-5c7b-4362-82a4-54731362c757; detaching it from the instance and deleting it from the info cache
Aug 09 02:57:02 user nova-compute[72351]: DEBUG nova.network.neutron [req-f308b2aa-7ee8-423b-a627-0695cb5d6a29 req-5839063f-5a31-414f-976e-73e702a8ce32 service nova] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 02:57:02 user nova-compute[72351]: DEBUG nova.compute.manager [req-f308b2aa-7ee8-423b-a627-0695cb5d6a29 req-5839063f-5a31-414f-976e-73e702a8ce32 service nova] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Detach interface failed, port_id=07ee9224-5c7b-4362-82a4-54731362c757, reason: Instance ccc96aeb-4de7-4511-a668-e3dee04e22f5 could not be found. {{(pid=72351) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10866}}
Aug 09 02:57:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-532c3cb8-fa21-46f1-8559-a206a1f4d037 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:57:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-532c3cb8-fa21-46f1-8559-a206a1f4d037 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:57:02 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-532c3cb8-fa21-46f1-8559-a206a1f4d037 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:57:02 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-532c3cb8-fa21-46f1-8559-a206a1f4d037 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:57:03 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-532c3cb8-fa21-46f1-8559-a206a1f4d037 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.120s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:57:03 user nova-compute[72351]: INFO nova.scheduler.client.report [None req-532c3cb8-fa21-46f1-8559-a206a1f4d037 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Deleted allocations for instance ccc96aeb-4de7-4511-a668-e3dee04e22f5
Aug 09 02:57:03 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-532c3cb8-fa21-46f1-8559-a206a1f4d037 tempest-AttachVolumeTestJSON-1725368857 tempest-AttachVolumeTestJSON-1725368857-project-member] Lock "ccc96aeb-4de7-4511-a668-e3dee04e22f5" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.246s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:57:03 user nova-compute[72351]: DEBUG nova.compute.manager [req-72c71675-4bb2-47bd-9851-6a3c90e191fe req-165a351d-a71a-40c7-a556-5776129ebb1a service nova] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Received event network-vif-plugged-07ee9224-5c7b-4362-82a4-54731362c757 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 02:57:03 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-72c71675-4bb2-47bd-9851-6a3c90e191fe req-165a351d-a71a-40c7-a556-5776129ebb1a service nova] Acquiring lock "ccc96aeb-4de7-4511-a668-e3dee04e22f5-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:57:03 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-72c71675-4bb2-47bd-9851-6a3c90e191fe req-165a351d-a71a-40c7-a556-5776129ebb1a service nova] Lock "ccc96aeb-4de7-4511-a668-e3dee04e22f5-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:57:03 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-72c71675-4bb2-47bd-9851-6a3c90e191fe req-165a351d-a71a-40c7-a556-5776129ebb1a service nova] Lock "ccc96aeb-4de7-4511-a668-e3dee04e22f5-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:57:03 user nova-compute[72351]: DEBUG nova.compute.manager [req-72c71675-4bb2-47bd-9851-6a3c90e191fe req-165a351d-a71a-40c7-a556-5776129ebb1a service nova] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] No waiting events found dispatching network-vif-plugged-07ee9224-5c7b-4362-82a4-54731362c757 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 02:57:03 user nova-compute[72351]: WARNING nova.compute.manager [req-72c71675-4bb2-47bd-9851-6a3c90e191fe req-165a351d-a71a-40c7-a556-5776129ebb1a service nova] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Received unexpected event network-vif-plugged-07ee9224-5c7b-4362-82a4-54731362c757 for instance with vm_state deleted and task_state None.
Aug 09 02:57:06 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:57:11 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:57:16 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691549821.6855147, ccc96aeb-4de7-4511-a668-e3dee04e22f5 => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 02:57:16 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] VM Stopped (Lifecycle Event)
Aug 09 02:57:16 user nova-compute[72351]: DEBUG nova.compute.manager [None req-3e4c004b-deb9-4e74-8cb3-83a98076356f None None] [instance: ccc96aeb-4de7-4511-a668-e3dee04e22f5] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 02:57:16 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:57:16 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:57:16 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 02:57:16 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:57:16 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:57:16 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:57:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:57:22 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:57:25 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:57:25 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Starting heal instance info cache {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9842}}
Aug 09 02:57:25 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Rebuilding the list of instances to heal {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9846}}
Aug 09 02:57:25 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Didn't find any instances for network info cache update. {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9928}}
Aug 09 02:57:25 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:57:25 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72351) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10461}}
Aug 09 02:57:26 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:57:26 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:57:30 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:57:30 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:57:31 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:57:31 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:57:31 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:57:31 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:57:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:57:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:57:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:57:31 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Auditing locally available compute resources for user (node: user) {{(pid=72351) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Aug 09 02:57:32 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:57:32 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:57:32 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Hypervisor/Node resource view: name=user free_ram=8956MB free_disk=25.22469711303711GB free_vcpus=12 pci_devices=[{"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72351) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Aug 09 02:57:32 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:57:32 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:57:32 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Total usable vcpus: 12, total allocated vcpus: 0 {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Aug 09 02:57:32 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Final resource view: name=user phys_ram=16011MB used_ram=512MB phys_disk=40GB used_disk=0GB total_vcpus=12 used_vcpus=0 pci_stats=[] {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Aug 09 02:57:32 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:57:32 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:57:32 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Compute_service record updated for user:user {{(pid=72351) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Aug 09 02:57:32 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.137s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:57:36 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:57:39 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:57:41 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:57:46 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:57:46 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:57:46 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 02:57:46 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:57:46 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:57:46 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:57:51 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:57:51 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:57:56 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:58:01 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:58:06 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:58:11 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:58:11 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:58:16 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:58:16 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:58:16 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5001 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 02:58:16 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:58:16 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:58:16 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:58:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:58:22 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:58:26 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:58:26 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:58:26 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Starting heal instance info cache {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9842}}
Aug 09 02:58:26 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Rebuilding the list of instances to heal {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9846}}
Aug 09 02:58:26 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Didn't find any instances for network info cache update. {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9928}}
Aug 09 02:58:26 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:58:27 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:58:27 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72351) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10461}}
Aug 09 02:58:30 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:58:31 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:58:31 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:58:32 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:58:32 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:58:32 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:58:32 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:58:32 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:58:32 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:58:32 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Auditing locally available compute resources for user (node: user) {{(pid=72351) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Aug 09 02:58:33 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:58:33 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:58:33 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Hypervisor/Node resource view: name=user free_ram=8949MB free_disk=25.22281265258789GB free_vcpus=12 pci_devices=[{"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72351) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Aug 09 02:58:33 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:58:33 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:58:33 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Total usable vcpus: 12, total allocated vcpus: 0 {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Aug 09 02:58:33 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Final resource view: name=user phys_ram=16011MB used_ram=512MB phys_disk=40GB used_disk=0GB total_vcpus=12 used_vcpus=0 pci_stats=[] {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Aug 09 02:58:33 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:58:33 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:58:33 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Compute_service record updated for user:user {{(pid=72351) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Aug 09 02:58:33 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.132s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:58:36 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:58:39 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._cleanup_expired_console_auth_tokens {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:58:41 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:58:46 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:58:48 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._run_pending_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:58:48 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Cleaning up deleted instances {{(pid=72351) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11129}}
Aug 09 02:58:48 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] There are 0 instances to clean {{(pid=72351) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11138}}
Aug 09 02:58:51 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:58:56 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:58:57 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._cleanup_incomplete_migrations {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:58:57 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Cleaning up deleted instances with incomplete migration  {{(pid=72351) _cleanup_incomplete_migrations /opt/stack/nova/nova/compute/manager.py:11167}}
Aug 09 02:59:01 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:59:01 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:59:01 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 02:59:01 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:59:01 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:59:01 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:59:06 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:59:06 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:59:06 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5001 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 02:59:06 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:59:06 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:59:06 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:59:11 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:59:16 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:59:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:59:22 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:59:26 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:59:26 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:59:26 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 02:59:26 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:59:26 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:59:26 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:59:26 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:59:26 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Starting heal instance info cache {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9842}}
Aug 09 02:59:26 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Rebuilding the list of instances to heal {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9846}}
Aug 09 02:59:26 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Didn't find any instances for network info cache update. {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9928}}
Aug 09 02:59:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:59:27 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:59:27 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:59:27 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72351) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10461}}
Aug 09 02:59:30 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:59:31 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:59:31 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:59:32 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:59:32 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:59:32 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:59:32 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:59:32 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:59:32 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Auditing locally available compute resources for user (node: user) {{(pid=72351) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Aug 09 02:59:33 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:59:33 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 02:59:33 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Hypervisor/Node resource view: name=user free_ram=8950MB free_disk=25.220901489257812GB free_vcpus=12 pci_devices=[{"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72351) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Aug 09 02:59:33 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 02:59:33 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 02:59:33 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Total usable vcpus: 12, total allocated vcpus: 0 {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Aug 09 02:59:33 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Final resource view: name=user phys_ram=16011MB used_ram=512MB phys_disk=40GB used_disk=0GB total_vcpus=12 used_vcpus=0 pci_stats=[] {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Aug 09 02:59:33 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Refreshing inventories for resource provider 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:804}}
Aug 09 02:59:33 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Updating ProviderTree inventory for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 from _refresh_and_get_inventory using data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) _refresh_and_get_inventory /opt/stack/nova/nova/scheduler/client/report.py:768}}
Aug 09 02:59:33 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Updating inventory in ProviderTree for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 with inventory: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:176}}
Aug 09 02:59:33 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Refreshing aggregate associations for resource provider 8eae9b37-1236-4e6d-91a5-b650de27b009, aggregates: None {{(pid=72351) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:813}}
Aug 09 02:59:33 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Refreshing trait associations for resource provider 8eae9b37-1236-4e6d-91a5-b650de27b009, traits: COMPUTE_VIOMMU_MODEL_INTEL,COMPUTE_STORAGE_BUS_IDE,COMPUTE_GRAPHICS_MODEL_NONE,COMPUTE_NET_VIF_MODEL_E1000E,COMPUTE_IMAGE_TYPE_AKI,COMPUTE_VOLUME_ATTACH_WITH_TAG,COMPUTE_NET_VIF_MODEL_VMXNET3,COMPUTE_SOCKET_PCI_NUMA_AFFINITY,COMPUTE_ACCELERATORS,HW_CPU_X86_SSE41,COMPUTE_NET_ATTACH_INTERFACE_WITH_TAG,COMPUTE_GRAPHICS_MODEL_VGA,COMPUTE_GRAPHICS_MODEL_QXL,COMPUTE_VIOMMU_MODEL_AUTO,COMPUTE_SECURITY_UEFI_SECURE_BOOT,COMPUTE_NET_VIF_MODEL_E1000,COMPUTE_TRUSTED_CERTS,COMPUTE_VOLUME_MULTI_ATTACH,COMPUTE_STORAGE_BUS_FDC,COMPUTE_VOLUME_EXTEND,COMPUTE_GRAPHICS_MODEL_CIRRUS,COMPUTE_GRAPHICS_MODEL_VMVGA,COMPUTE_GRAPHICS_MODEL_BOCHS,COMPUTE_DEVICE_TAGGING,COMPUTE_STORAGE_BUS_SCSI,COMPUTE_STORAGE_BUS_VIRTIO,HW_CPU_X86_SSSE3,HW_CPU_X86_SSE2,COMPUTE_IMAGE_TYPE_AMI,COMPUTE_STORAGE_BUS_USB,COMPUTE_NET_VIF_MODEL_PCNET,COMPUTE_NET_VIF_MODEL_VIRTIO,COMPUTE_IMAGE_TYPE_ISO,HW_CPU_X86_SSE42,HW_CPU_X86_SSE,COMPUTE_IMAGE_TYPE_ARI,COMPUTE_IMAGE_TYPE_RAW,COMPUTE_IMAGE_TYPE_QCOW2,COMPUTE_NET_VIF_MODEL_NE2K_PCI,COMPUTE_NODE,HW_CPU_X86_MMX,COMPUTE_NET_VIF_MODEL_RTL8139,COMPUTE_GRAPHICS_MODEL_VIRTIO,COMPUTE_NET_VIF_MODEL_SPAPR_VLAN,COMPUTE_STORAGE_BUS_SATA,COMPUTE_NET_ATTACH_INTERFACE,COMPUTE_RESCUE_BFV {{(pid=72351) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:825}}
Aug 09 02:59:33 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 02:59:33 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 02:59:33 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Compute_service record updated for user:user {{(pid=72351) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Aug 09 02:59:33 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.454s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 02:59:35 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:59:36 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:59:41 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:59:41 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:59:41 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 02:59:41 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:59:41 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:59:41 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:59:41 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 02:59:46 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:59:51 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:59:56 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 02:59:56 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 02:59:56 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5001 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 02:59:56 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:59:56 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 02:59:56 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:00:01 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:00:06 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:00:11 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:00:11 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:00:11 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 03:00:11 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:00:11 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:00:11 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:00:16 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:00:21 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:00:22 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:00:23 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:00:26 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:00:26 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:00:27 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:00:27 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Starting heal instance info cache {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9842}}
Aug 09 03:00:27 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Rebuilding the list of instances to heal {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9846}}
Aug 09 03:00:27 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Didn't find any instances for network info cache update. {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9928}}
Aug 09 03:00:28 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:00:28 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:00:28 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72351) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10461}}
Aug 09 03:00:29 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:00:31 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:00:31 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:00:31 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:00:32 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:00:34 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:00:34 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:00:34 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:00:34 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 03:00:34 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 03:00:34 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 03:00:34 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Auditing locally available compute resources for user (node: user) {{(pid=72351) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Aug 09 03:00:34 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:00:34 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 03:00:34 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 03:00:34 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Hypervisor/Node resource view: name=user free_ram=8617MB free_disk=25.208091735839844GB free_vcpus=12 pci_devices=[{"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72351) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Aug 09 03:00:34 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 03:00:34 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 03:00:34 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Total usable vcpus: 12, total allocated vcpus: 0 {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Aug 09 03:00:34 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Final resource view: name=user phys_ram=16011MB used_ram=512MB phys_disk=40GB used_disk=0GB total_vcpus=12 used_vcpus=0 pci_stats=[] {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Aug 09 03:00:34 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 03:00:34 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 03:00:34 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Compute_service record updated for user:user {{(pid=72351) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Aug 09 03:00:34 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.187s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 03:00:35 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:00:36 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:00:37 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:00:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:00:41 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Acquiring lock "56b80e5d-b7f4-4780-987e-5def6d299f3d" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 03:00:41 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "56b80e5d-b7f4-4780-987e-5def6d299f3d" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 03:00:41 user nova-compute[72351]: DEBUG nova.compute.manager [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Starting instance... {{(pid=72351) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2406}}
Aug 09 03:00:41 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:00:41 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 03:00:41 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 03:00:41 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=72351) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Aug 09 03:00:41 user nova-compute[72351]: INFO nova.compute.claims [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Claim successful on node user
Aug 09 03:00:42 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 03:00:42 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 03:00:42 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.206s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 03:00:42 user nova-compute[72351]: DEBUG nova.compute.manager [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Start building networks asynchronously for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2803}}
Aug 09 03:00:42 user nova-compute[72351]: DEBUG nova.compute.manager [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Allocating IP information in the background. {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1956}}
Aug 09 03:00:42 user nova-compute[72351]: DEBUG nova.network.neutron [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] allocate_for_instance() {{(pid=72351) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Aug 09 03:00:42 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Aug 09 03:00:42 user nova-compute[72351]: DEBUG nova.compute.manager [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Start building block device mappings for instance. {{(pid=72351) _build_resources /opt/stack/nova/nova/compute/manager.py:2838}}
Aug 09 03:00:42 user nova-compute[72351]: DEBUG nova.policy [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '44f56f4289eb4eb79f4751ef02264af8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a636abaff2944499911a14ffb2846b48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72351) authorize /opt/stack/nova/nova/policy.py:203}}
Aug 09 03:00:42 user nova-compute[72351]: DEBUG nova.compute.manager [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Start spawning the instance on the hypervisor. {{(pid=72351) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2612}}
Aug 09 03:00:42 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Creating instance directory {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4717}}
Aug 09 03:00:42 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Creating image(s)
Aug 09 03:00:42 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Acquiring lock "/opt/stack/data/nova/instances/56b80e5d-b7f4-4780-987e-5def6d299f3d/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 03:00:42 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "/opt/stack/data/nova/instances/56b80e5d-b7f4-4780-987e-5def6d299f3d/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 03:00:42 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "/opt/stack/data/nova/instances/56b80e5d-b7f4-4780-987e-5def6d299f3d/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 03:00:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:00:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.162s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:00:42 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Acquiring lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 03:00:42 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 03:00:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:00:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.160s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:00:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/56b80e5d-b7f4-4780-987e-5def6d299f3d/disk 1073741824 {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:00:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd,backing_fmt=raw /opt/stack/data/nova/instances/56b80e5d-b7f4-4780-987e-5def6d299f3d/disk 1073741824" returned: 0 in 0.060s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:00:42 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "f54924f2669dce52d08d283dcdc864ffcb0b07cd" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.228s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 03:00:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:00:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f54924f2669dce52d08d283dcdc864ffcb0b07cd --force-share --output=json" returned: 0 in 0.168s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:00:42 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Checking if we can resize image /opt/stack/data/nova/instances/56b80e5d-b7f4-4780-987e-5def6d299f3d/disk. size=1073741824 {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Aug 09 03:00:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56b80e5d-b7f4-4780-987e-5def6d299f3d/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:00:43 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56b80e5d-b7f4-4780-987e-5def6d299f3d/disk --force-share --output=json" returned: 0 in 0.156s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:00:43 user nova-compute[72351]: DEBUG nova.virt.disk.api [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Cannot resize image /opt/stack/data/nova/instances/56b80e5d-b7f4-4780-987e-5def6d299f3d/disk to a smaller size. {{(pid=72351) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Aug 09 03:00:43 user nova-compute[72351]: DEBUG nova.objects.instance [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lazy-loading 'migration_context' on Instance uuid 56b80e5d-b7f4-4780-987e-5def6d299f3d {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 03:00:43 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Created local disks {{(pid=72351) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4851}}
Aug 09 03:00:43 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Ensure instance console log exists: /opt/stack/data/nova/instances/56b80e5d-b7f4-4780-987e-5def6d299f3d/console.log {{(pid=72351) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4603}}
Aug 09 03:00:43 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 03:00:43 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 03:00:43 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 03:00:43 user nova-compute[72351]: DEBUG nova.network.neutron [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Successfully created port: c69c5267-24fa-40f1-8f76-b4f2ec1ced93 {{(pid=72351) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Aug 09 03:00:44 user nova-compute[72351]: DEBUG nova.network.neutron [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Successfully updated port: c69c5267-24fa-40f1-8f76-b4f2ec1ced93 {{(pid=72351) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Aug 09 03:00:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Acquiring lock "refresh_cache-56b80e5d-b7f4-4780-987e-5def6d299f3d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 03:00:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Acquired lock "refresh_cache-56b80e5d-b7f4-4780-987e-5def6d299f3d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 03:00:44 user nova-compute[72351]: DEBUG nova.network.neutron [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Building network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2002}}
Aug 09 03:00:44 user nova-compute[72351]: DEBUG nova.network.neutron [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Instance cache missing network info. {{(pid=72351) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3315}}
Aug 09 03:00:44 user nova-compute[72351]: DEBUG nova.compute.manager [req-c441e629-a729-4687-a8ab-d76398270283 req-26fe917d-45f3-45de-8457-584d3e0cdb4e service nova] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Received event network-changed-c69c5267-24fa-40f1-8f76-b4f2ec1ced93 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 03:00:44 user nova-compute[72351]: DEBUG nova.compute.manager [req-c441e629-a729-4687-a8ab-d76398270283 req-26fe917d-45f3-45de-8457-584d3e0cdb4e service nova] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Refreshing instance network info cache due to event network-changed-c69c5267-24fa-40f1-8f76-b4f2ec1ced93. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 03:00:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c441e629-a729-4687-a8ab-d76398270283 req-26fe917d-45f3-45de-8457-584d3e0cdb4e service nova] Acquiring lock "refresh_cache-56b80e5d-b7f4-4780-987e-5def6d299f3d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 03:00:44 user nova-compute[72351]: DEBUG nova.network.neutron [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Updating instance_info_cache with network_info: [{"id": "c69c5267-24fa-40f1-8f76-b4f2ec1ced93", "address": "fa:16:3e:ea:18:dd", "network": {"id": "45ee1620-bd54-4ca8-ab3a-9222782aea6f", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-173232413-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a636abaff2944499911a14ffb2846b48", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc69c5267-24", "ovs_interfaceid": "c69c5267-24fa-40f1-8f76-b4f2ec1ced93", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 03:00:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Releasing lock "refresh_cache-56b80e5d-b7f4-4780-987e-5def6d299f3d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 03:00:44 user nova-compute[72351]: DEBUG nova.compute.manager [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Instance network_info: |[{"id": "c69c5267-24fa-40f1-8f76-b4f2ec1ced93", "address": "fa:16:3e:ea:18:dd", "network": {"id": "45ee1620-bd54-4ca8-ab3a-9222782aea6f", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-173232413-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a636abaff2944499911a14ffb2846b48", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc69c5267-24", "ovs_interfaceid": "c69c5267-24fa-40f1-8f76-b4f2ec1ced93", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=72351) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1971}}
Aug 09 03:00:44 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c441e629-a729-4687-a8ab-d76398270283 req-26fe917d-45f3-45de-8457-584d3e0cdb4e service nova] Acquired lock "refresh_cache-56b80e5d-b7f4-4780-987e-5def6d299f3d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 03:00:44 user nova-compute[72351]: DEBUG nova.network.neutron [req-c441e629-a729-4687-a8ab-d76398270283 req-26fe917d-45f3-45de-8457-584d3e0cdb4e service nova] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Refreshing network info cache for port c69c5267-24fa-40f1-8f76-b4f2ec1ced93 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 03:00:44 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Start _get_guest_xml network_info=[{"id": "c69c5267-24fa-40f1-8f76-b4f2ec1ced93", "address": "fa:16:3e:ea:18:dd", "network": {"id": "45ee1620-bd54-4ca8-ab3a-9222782aea6f", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-173232413-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a636abaff2944499911a14ffb2846b48", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc69c5267-24", "ovs_interfaceid": "c69c5267-24fa-40f1-8f76-b4f2ec1ced93", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'size': 0, 'device_type': 'disk', 'encryption_secret_uuid': None, 'guest_format': None, 'encryption_options': None, 'disk_bus': 'virtio', 'encrypted': False, 'encryption_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'image_id': '18ccda7f-31d3-4978-ab9d-4ed2f74eff94'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7553}}
Aug 09 03:00:44 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 03:00:44 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 03:00:44 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1650}}
Aug 09 03:00:44 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] CPU controller missing on host. {{(pid=72351) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1660}}
Aug 09 03:00:44 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1669}}
Aug 09 03:00:44 user nova-compute[72351]: DEBUG nova.virt.libvirt.host [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] CPU controller found on host. {{(pid=72351) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1676}}
Aug 09 03:00:44 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=72351) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5390}}
Aug 09 03:00:44 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-08-09T02:13:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-08-09T02:13:15Z,direct_url=<?>,disk_format='qcow2',id=18ccda7f-31d3-4978-ab9d-4ed2f74eff94,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='f9af963f7c574a7ab2deedfcf953a666',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-08-09T02:13:16Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Aug 09 03:00:44 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Flavor limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Aug 09 03:00:44 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Image limits 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Aug 09 03:00:44 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Flavor pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Aug 09 03:00:44 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Image pref 0:0:0 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Aug 09 03:00:44 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Aug 09 03:00:44 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Aug 09 03:00:44 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Aug 09 03:00:44 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Got 1 possible topologies {{(pid=72351) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Aug 09 03:00:44 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Aug 09 03:00:44 user nova-compute[72351]: DEBUG nova.virt.hardware [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=72351) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Aug 09 03:00:44 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T03:00:41Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-SnapshotDataIntegrityTests-server-987829355',display_name='tempest-SnapshotDataIntegrityTests-server-987829355',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-snapshotdataintegritytests-server-987829355',id=52,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNotwPL2HAojEvWq4iWFaRF6K2ttAaXMNCv5PqTAvmXQtQ2oZUb0Y5aQuvrV3ERKKWu0WcMHTZ53M0AXKuTVceZAdA/FjnYInOjduGzbBvX/kPOV2Ht34221e9wQ+LuGig==',key_name='tempest-keypair-756037336',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='a636abaff2944499911a14ffb2846b48',ramdisk_id='',reservation_id='r-v2t8deh2',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-SnapshotDataIntegrityTests-248167435',owner_user_name='tempest-SnapshotDataIntegrityTests-248167435-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T03:00:42Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='44f56f4289eb4eb79f4751ef02264af8',uuid=56b80e5d-b7f4-4780-987e-5def6d299f3d,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "c69c5267-24fa-40f1-8f76-b4f2ec1ced93", "address": "fa:16:3e:ea:18:dd", "network": {"id": "45ee1620-bd54-4ca8-ab3a-9222782aea6f", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-173232413-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a636abaff2944499911a14ffb2846b48", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc69c5267-24", "ovs_interfaceid": "c69c5267-24fa-40f1-8f76-b4f2ec1ced93", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=72351) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Aug 09 03:00:44 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Converting VIF {"id": "c69c5267-24fa-40f1-8f76-b4f2ec1ced93", "address": "fa:16:3e:ea:18:dd", "network": {"id": "45ee1620-bd54-4ca8-ab3a-9222782aea6f", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-173232413-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a636abaff2944499911a14ffb2846b48", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc69c5267-24", "ovs_interfaceid": "c69c5267-24fa-40f1-8f76-b4f2ec1ced93", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 03:00:44 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:ea:18:dd,bridge_name='br-int',has_traffic_filtering=True,id=c69c5267-24fa-40f1-8f76-b4f2ec1ced93,network=Network(45ee1620-bd54-4ca8-ab3a-9222782aea6f),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc69c5267-24') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 03:00:44 user nova-compute[72351]: DEBUG nova.objects.instance [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lazy-loading 'pci_devices' on Instance uuid 56b80e5d-b7f4-4780-987e-5def6d299f3d {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 03:00:44 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] End _get_guest_xml xml=<domain type="kvm">
Aug 09 03:00:44 user nova-compute[72351]:   <uuid>56b80e5d-b7f4-4780-987e-5def6d299f3d</uuid>
Aug 09 03:00:44 user nova-compute[72351]:   <name>instance-00000034</name>
Aug 09 03:00:44 user nova-compute[72351]:   <memory>524288</memory>
Aug 09 03:00:44 user nova-compute[72351]:   <vcpu>1</vcpu>
Aug 09 03:00:44 user nova-compute[72351]:   <metadata>
Aug 09 03:00:44 user nova-compute[72351]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Aug 09 03:00:44 user nova-compute[72351]:       <nova:package version="0.0.0"/>
Aug 09 03:00:44 user nova-compute[72351]:       <nova:name>tempest-SnapshotDataIntegrityTests-server-987829355</nova:name>
Aug 09 03:00:44 user nova-compute[72351]:       <nova:creationTime>2023-08-09 03:00:44</nova:creationTime>
Aug 09 03:00:44 user nova-compute[72351]:       <nova:flavor name="m1.tiny">
Aug 09 03:00:44 user nova-compute[72351]:         <nova:memory>512</nova:memory>
Aug 09 03:00:44 user nova-compute[72351]:         <nova:disk>1</nova:disk>
Aug 09 03:00:44 user nova-compute[72351]:         <nova:swap>0</nova:swap>
Aug 09 03:00:44 user nova-compute[72351]:         <nova:ephemeral>0</nova:ephemeral>
Aug 09 03:00:44 user nova-compute[72351]:         <nova:vcpus>1</nova:vcpus>
Aug 09 03:00:44 user nova-compute[72351]:       </nova:flavor>
Aug 09 03:00:44 user nova-compute[72351]:       <nova:owner>
Aug 09 03:00:44 user nova-compute[72351]:         <nova:user uuid="44f56f4289eb4eb79f4751ef02264af8">tempest-SnapshotDataIntegrityTests-248167435-project-member</nova:user>
Aug 09 03:00:44 user nova-compute[72351]:         <nova:project uuid="a636abaff2944499911a14ffb2846b48">tempest-SnapshotDataIntegrityTests-248167435</nova:project>
Aug 09 03:00:44 user nova-compute[72351]:       </nova:owner>
Aug 09 03:00:44 user nova-compute[72351]:       <nova:root type="image" uuid="18ccda7f-31d3-4978-ab9d-4ed2f74eff94"/>
Aug 09 03:00:44 user nova-compute[72351]:       <nova:ports>
Aug 09 03:00:44 user nova-compute[72351]:         <nova:port uuid="c69c5267-24fa-40f1-8f76-b4f2ec1ced93">
Aug 09 03:00:44 user nova-compute[72351]:           <nova:ip type="fixed" address="10.0.0.10" ipVersion="4"/>
Aug 09 03:00:44 user nova-compute[72351]:         </nova:port>
Aug 09 03:00:44 user nova-compute[72351]:       </nova:ports>
Aug 09 03:00:44 user nova-compute[72351]:     </nova:instance>
Aug 09 03:00:44 user nova-compute[72351]:   </metadata>
Aug 09 03:00:44 user nova-compute[72351]:   <sysinfo type="smbios">
Aug 09 03:00:44 user nova-compute[72351]:     <system>
Aug 09 03:00:44 user nova-compute[72351]:       <entry name="manufacturer">OpenStack Foundation</entry>
Aug 09 03:00:44 user nova-compute[72351]:       <entry name="product">OpenStack Nova</entry>
Aug 09 03:00:44 user nova-compute[72351]:       <entry name="version">0.0.0</entry>
Aug 09 03:00:44 user nova-compute[72351]:       <entry name="serial">56b80e5d-b7f4-4780-987e-5def6d299f3d</entry>
Aug 09 03:00:44 user nova-compute[72351]:       <entry name="uuid">56b80e5d-b7f4-4780-987e-5def6d299f3d</entry>
Aug 09 03:00:44 user nova-compute[72351]:       <entry name="family">Virtual Machine</entry>
Aug 09 03:00:44 user nova-compute[72351]:     </system>
Aug 09 03:00:44 user nova-compute[72351]:   </sysinfo>
Aug 09 03:00:44 user nova-compute[72351]:   <os>
Aug 09 03:00:44 user nova-compute[72351]:     <type arch="x86_64" machine="pc">hvm</type>
Aug 09 03:00:44 user nova-compute[72351]:     <boot dev="hd"/>
Aug 09 03:00:44 user nova-compute[72351]:     <smbios mode="sysinfo"/>
Aug 09 03:00:44 user nova-compute[72351]:   </os>
Aug 09 03:00:44 user nova-compute[72351]:   <features>
Aug 09 03:00:44 user nova-compute[72351]:     <acpi/>
Aug 09 03:00:44 user nova-compute[72351]:     <apic/>
Aug 09 03:00:44 user nova-compute[72351]:     <vmcoreinfo/>
Aug 09 03:00:44 user nova-compute[72351]:   </features>
Aug 09 03:00:44 user nova-compute[72351]:   <clock offset="utc">
Aug 09 03:00:44 user nova-compute[72351]:     <timer name="pit" tickpolicy="delay"/>
Aug 09 03:00:44 user nova-compute[72351]:     <timer name="rtc" tickpolicy="catchup"/>
Aug 09 03:00:44 user nova-compute[72351]:     <timer name="hpet" present="no"/>
Aug 09 03:00:44 user nova-compute[72351]:   </clock>
Aug 09 03:00:44 user nova-compute[72351]:   <cpu mode="custom" match="exact">
Aug 09 03:00:44 user nova-compute[72351]:     <model>Nehalem</model>
Aug 09 03:00:44 user nova-compute[72351]:     <topology sockets="1" cores="1" threads="1"/>
Aug 09 03:00:44 user nova-compute[72351]:   </cpu>
Aug 09 03:00:44 user nova-compute[72351]:   <devices>
Aug 09 03:00:44 user nova-compute[72351]:     <disk type="file" device="disk">
Aug 09 03:00:44 user nova-compute[72351]:       <driver name="qemu" type="qcow2" cache="none"/>
Aug 09 03:00:44 user nova-compute[72351]:       <source file="/opt/stack/data/nova/instances/56b80e5d-b7f4-4780-987e-5def6d299f3d/disk"/>
Aug 09 03:00:44 user nova-compute[72351]:       <target dev="vda" bus="virtio"/>
Aug 09 03:00:44 user nova-compute[72351]:     </disk>
Aug 09 03:00:44 user nova-compute[72351]:     <interface type="ethernet">
Aug 09 03:00:44 user nova-compute[72351]:       <mac address="fa:16:3e:ea:18:dd"/>
Aug 09 03:00:44 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 03:00:44 user nova-compute[72351]:       <mtu size="1442"/>
Aug 09 03:00:44 user nova-compute[72351]:       <target dev="tapc69c5267-24"/>
Aug 09 03:00:44 user nova-compute[72351]:     </interface>
Aug 09 03:00:44 user nova-compute[72351]:     <serial type="pty">
Aug 09 03:00:44 user nova-compute[72351]:       <log file="/opt/stack/data/nova/instances/56b80e5d-b7f4-4780-987e-5def6d299f3d/console.log" append="off"/>
Aug 09 03:00:44 user nova-compute[72351]:     </serial>
Aug 09 03:00:44 user nova-compute[72351]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Aug 09 03:00:44 user nova-compute[72351]:     <video>
Aug 09 03:00:44 user nova-compute[72351]:       <model type="virtio"/>
Aug 09 03:00:44 user nova-compute[72351]:     </video>
Aug 09 03:00:44 user nova-compute[72351]:     <rng model="virtio">
Aug 09 03:00:44 user nova-compute[72351]:       <backend model="random">/dev/urandom</backend>
Aug 09 03:00:44 user nova-compute[72351]:     </rng>
Aug 09 03:00:44 user nova-compute[72351]:     <controller type="usb" index="0" model="none"/>
Aug 09 03:00:44 user nova-compute[72351]:     <memballoon model="virtio">
Aug 09 03:00:44 user nova-compute[72351]:       <stats period="10"/>
Aug 09 03:00:44 user nova-compute[72351]:     </memballoon>
Aug 09 03:00:44 user nova-compute[72351]:   </devices>
Aug 09 03:00:44 user nova-compute[72351]: </domain>
Aug 09 03:00:44 user nova-compute[72351]:  {{(pid=72351) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7559}}
Aug 09 03:00:44 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T03:00:41Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-SnapshotDataIntegrityTests-server-987829355',display_name='tempest-SnapshotDataIntegrityTests-server-987829355',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-snapshotdataintegritytests-server-987829355',id=52,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNotwPL2HAojEvWq4iWFaRF6K2ttAaXMNCv5PqTAvmXQtQ2oZUb0Y5aQuvrV3ERKKWu0WcMHTZ53M0AXKuTVceZAdA/FjnYInOjduGzbBvX/kPOV2Ht34221e9wQ+LuGig==',key_name='tempest-keypair-756037336',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='a636abaff2944499911a14ffb2846b48',ramdisk_id='',reservation_id='r-v2t8deh2',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-SnapshotDataIntegrityTests-248167435',owner_user_name='tempest-SnapshotDataIntegrityTests-248167435-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-08-09T03:00:42Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='44f56f4289eb4eb79f4751ef02264af8',uuid=56b80e5d-b7f4-4780-987e-5def6d299f3d,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "c69c5267-24fa-40f1-8f76-b4f2ec1ced93", "address": "fa:16:3e:ea:18:dd", "network": {"id": "45ee1620-bd54-4ca8-ab3a-9222782aea6f", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-173232413-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a636abaff2944499911a14ffb2846b48", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc69c5267-24", "ovs_interfaceid": "c69c5267-24fa-40f1-8f76-b4f2ec1ced93", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Aug 09 03:00:44 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Converting VIF {"id": "c69c5267-24fa-40f1-8f76-b4f2ec1ced93", "address": "fa:16:3e:ea:18:dd", "network": {"id": "45ee1620-bd54-4ca8-ab3a-9222782aea6f", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-173232413-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a636abaff2944499911a14ffb2846b48", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc69c5267-24", "ovs_interfaceid": "c69c5267-24fa-40f1-8f76-b4f2ec1ced93", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 03:00:44 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:ea:18:dd,bridge_name='br-int',has_traffic_filtering=True,id=c69c5267-24fa-40f1-8f76-b4f2ec1ced93,network=Network(45ee1620-bd54-4ca8-ab3a-9222782aea6f),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc69c5267-24') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 03:00:44 user nova-compute[72351]: DEBUG os_vif [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:ea:18:dd,bridge_name='br-int',has_traffic_filtering=True,id=c69c5267-24fa-40f1-8f76-b4f2ec1ced93,network=Network(45ee1620-bd54-4ca8-ab3a-9222782aea6f),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc69c5267-24') {{(pid=72351) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Aug 09 03:00:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:00:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 03:00:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Aug 09 03:00:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:00:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapc69c5267-24, may_exist=True, interface_attrs={}) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 03:00:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapc69c5267-24, col_values=(('external_ids', {'iface-id': 'c69c5267-24fa-40f1-8f76-b4f2ec1ced93', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:ea:18:dd', 'vm-uuid': '56b80e5d-b7f4-4780-987e-5def6d299f3d'}),), if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 03:00:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:00:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:00:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:00:44 user nova-compute[72351]: INFO os_vif [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:ea:18:dd,bridge_name='br-int',has_traffic_filtering=True,id=c69c5267-24fa-40f1-8f76-b4f2ec1ced93,network=Network(45ee1620-bd54-4ca8-ab3a-9222782aea6f),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc69c5267-24')
Aug 09 03:00:44 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 03:00:44 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] No VIF found with MAC fa:16:3e:ea:18:dd, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 03:00:45 user nova-compute[72351]: DEBUG nova.network.neutron [req-c441e629-a729-4687-a8ab-d76398270283 req-26fe917d-45f3-45de-8457-584d3e0cdb4e service nova] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Updated VIF entry in instance network info cache for port c69c5267-24fa-40f1-8f76-b4f2ec1ced93. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 03:00:45 user nova-compute[72351]: DEBUG nova.network.neutron [req-c441e629-a729-4687-a8ab-d76398270283 req-26fe917d-45f3-45de-8457-584d3e0cdb4e service nova] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Updating instance_info_cache with network_info: [{"id": "c69c5267-24fa-40f1-8f76-b4f2ec1ced93", "address": "fa:16:3e:ea:18:dd", "network": {"id": "45ee1620-bd54-4ca8-ab3a-9222782aea6f", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-173232413-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a636abaff2944499911a14ffb2846b48", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc69c5267-24", "ovs_interfaceid": "c69c5267-24fa-40f1-8f76-b4f2ec1ced93", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 03:00:45 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-c441e629-a729-4687-a8ab-d76398270283 req-26fe917d-45f3-45de-8457-584d3e0cdb4e service nova] Releasing lock "refresh_cache-56b80e5d-b7f4-4780-987e-5def6d299f3d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 03:00:46 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:00:46 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:00:46 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:00:46 user nova-compute[72351]: DEBUG nova.compute.manager [req-734bd5d0-7a52-4d50-adb9-a39f02b363d7 req-2c2e03d8-5607-4c19-9c26-227e1aa8100e service nova] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Received event network-vif-plugged-c69c5267-24fa-40f1-8f76-b4f2ec1ced93 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 03:00:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-734bd5d0-7a52-4d50-adb9-a39f02b363d7 req-2c2e03d8-5607-4c19-9c26-227e1aa8100e service nova] Acquiring lock "56b80e5d-b7f4-4780-987e-5def6d299f3d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 03:00:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-734bd5d0-7a52-4d50-adb9-a39f02b363d7 req-2c2e03d8-5607-4c19-9c26-227e1aa8100e service nova] Lock "56b80e5d-b7f4-4780-987e-5def6d299f3d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 03:00:46 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-734bd5d0-7a52-4d50-adb9-a39f02b363d7 req-2c2e03d8-5607-4c19-9c26-227e1aa8100e service nova] Lock "56b80e5d-b7f4-4780-987e-5def6d299f3d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 03:00:46 user nova-compute[72351]: DEBUG nova.compute.manager [req-734bd5d0-7a52-4d50-adb9-a39f02b363d7 req-2c2e03d8-5607-4c19-9c26-227e1aa8100e service nova] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] No waiting events found dispatching network-vif-plugged-c69c5267-24fa-40f1-8f76-b4f2ec1ced93 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 03:00:46 user nova-compute[72351]: WARNING nova.compute.manager [req-734bd5d0-7a52-4d50-adb9-a39f02b363d7 req-2c2e03d8-5607-4c19-9c26-227e1aa8100e service nova] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Received unexpected event network-vif-plugged-c69c5267-24fa-40f1-8f76-b4f2ec1ced93 for instance with vm_state building and task_state spawning.
Aug 09 03:00:46 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:00:46 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:00:46 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:00:46 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:00:48 user nova-compute[72351]: DEBUG nova.compute.manager [req-1c8a71f6-74ae-4692-a185-6cfdd265ccbe req-7db0475e-19b6-45c2-a2ae-79ce416021da service nova] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Received event network-vif-plugged-c69c5267-24fa-40f1-8f76-b4f2ec1ced93 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 03:00:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-1c8a71f6-74ae-4692-a185-6cfdd265ccbe req-7db0475e-19b6-45c2-a2ae-79ce416021da service nova] Acquiring lock "56b80e5d-b7f4-4780-987e-5def6d299f3d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 03:00:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-1c8a71f6-74ae-4692-a185-6cfdd265ccbe req-7db0475e-19b6-45c2-a2ae-79ce416021da service nova] Lock "56b80e5d-b7f4-4780-987e-5def6d299f3d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 03:00:48 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-1c8a71f6-74ae-4692-a185-6cfdd265ccbe req-7db0475e-19b6-45c2-a2ae-79ce416021da service nova] Lock "56b80e5d-b7f4-4780-987e-5def6d299f3d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 03:00:48 user nova-compute[72351]: DEBUG nova.compute.manager [req-1c8a71f6-74ae-4692-a185-6cfdd265ccbe req-7db0475e-19b6-45c2-a2ae-79ce416021da service nova] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] No waiting events found dispatching network-vif-plugged-c69c5267-24fa-40f1-8f76-b4f2ec1ced93 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 03:00:48 user nova-compute[72351]: WARNING nova.compute.manager [req-1c8a71f6-74ae-4692-a185-6cfdd265ccbe req-7db0475e-19b6-45c2-a2ae-79ce416021da service nova] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Received unexpected event network-vif-plugged-c69c5267-24fa-40f1-8f76-b4f2ec1ced93 for instance with vm_state building and task_state spawning.
Aug 09 03:00:48 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691550048.971866, 56b80e5d-b7f4-4780-987e-5def6d299f3d => Resumed> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 03:00:48 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] VM Resumed (Lifecycle Event)
Aug 09 03:00:48 user nova-compute[72351]: DEBUG nova.compute.manager [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Instance event wait completed in 0 seconds for  {{(pid=72351) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Aug 09 03:00:48 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Guest created on hypervisor {{(pid=72351) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4411}}
Aug 09 03:00:48 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Instance spawned successfully.
Aug 09 03:00:48 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:894}}
Aug 09 03:00:48 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 03:00:48 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 03:00:49 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Found default for hw_cdrom_bus of ide {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 03:00:49 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Found default for hw_disk_bus of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 03:00:49 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Found default for hw_input_bus of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 03:00:49 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Found default for hw_pointer_model of None {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 03:00:49 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Found default for hw_video_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 03:00:49 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Found default for hw_vif_model of virtio {{(pid=72351) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:923}}
Aug 09 03:00:49 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 03:00:49 user nova-compute[72351]: DEBUG nova.virt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Emitting event <LifecycleEvent: 1691550048.9756129, 56b80e5d-b7f4-4780-987e-5def6d299f3d => Started> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 03:00:49 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] VM Started (Lifecycle Event)
Aug 09 03:00:49 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 03:00:49 user nova-compute[72351]: DEBUG nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=72351) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1398}}
Aug 09 03:00:49 user nova-compute[72351]: INFO nova.compute.manager [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] During sync_power_state the instance has a pending task (spawning). Skip.
Aug 09 03:00:49 user nova-compute[72351]: INFO nova.compute.manager [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Took 6.73 seconds to spawn the instance on the hypervisor.
Aug 09 03:00:49 user nova-compute[72351]: DEBUG nova.compute.manager [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 03:00:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:00:49 user nova-compute[72351]: INFO nova.compute.manager [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Took 7.46 seconds to build instance.
Aug 09 03:00:49 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-16a9b767-7e8a-4364-b9ef-e7c18dfca101 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "56b80e5d-b7f4-4780-987e-5def6d299f3d" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 7.578s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 03:00:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:00:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:00:59 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:00:59 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:01:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:01:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:01:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:01:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:01:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:01:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:01:19 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:01:19 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:01:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:01:24 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:01:27 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:01:27 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Starting heal instance info cache {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9842}}
Aug 09 03:01:27 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Rebuilding the list of instances to heal {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9846}}
Aug 09 03:01:27 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "refresh_cache-56b80e5d-b7f4-4780-987e-5def6d299f3d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 03:01:27 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquired lock "refresh_cache-56b80e5d-b7f4-4780-987e-5def6d299f3d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 03:01:27 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Forcefully refreshing network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1996}}
Aug 09 03:01:27 user nova-compute[72351]: DEBUG nova.objects.instance [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lazy-loading 'info_cache' on Instance uuid 56b80e5d-b7f4-4780-987e-5def6d299f3d {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 03:01:28 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Updating instance_info_cache with network_info: [{"id": "c69c5267-24fa-40f1-8f76-b4f2ec1ced93", "address": "fa:16:3e:ea:18:dd", "network": {"id": "45ee1620-bd54-4ca8-ab3a-9222782aea6f", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-173232413-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a636abaff2944499911a14ffb2846b48", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc69c5267-24", "ovs_interfaceid": "c69c5267-24fa-40f1-8f76-b4f2ec1ced93", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 03:01:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Releasing lock "refresh_cache-56b80e5d-b7f4-4780-987e-5def6d299f3d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 03:01:28 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Updated the network info_cache for instance {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9913}}
Aug 09 03:01:28 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:01:28 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72351) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10461}}
Aug 09 03:01:29 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:01:29 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:01:29 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5003 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 03:01:29 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:01:29 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:01:29 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:01:30 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:01:32 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:01:34 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:01:34 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:01:34 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:01:34 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:01:34 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 03:01:34 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 03:01:34 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 03:01:34 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Auditing locally available compute resources for user (node: user) {{(pid=72351) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Aug 09 03:01:35 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56b80e5d-b7f4-4780-987e-5def6d299f3d/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:01:35 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56b80e5d-b7f4-4780-987e-5def6d299f3d/disk --force-share --output=json" returned: 0 in 0.165s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:01:35 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56b80e5d-b7f4-4780-987e-5def6d299f3d/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:01:35 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56b80e5d-b7f4-4780-987e-5def6d299f3d/disk --force-share --output=json" returned: 0 in 0.159s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:01:35 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 03:01:35 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 03:01:35 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Hypervisor/Node resource view: name=user free_ram=8613MB free_disk=25.174747467041016GB free_vcpus=11 pci_devices=[{"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72351) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Aug 09 03:01:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 03:01:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 03:01:35 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 56b80e5d-b7f4-4780-987e-5def6d299f3d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 03:01:35 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Total usable vcpus: 12, total allocated vcpus: 1 {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Aug 09 03:01:35 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Final resource view: name=user phys_ram=16011MB used_ram=1024MB phys_disk=40GB used_disk=1GB total_vcpus=12 used_vcpus=1 pci_stats=[] {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Aug 09 03:01:35 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 03:01:35 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 03:01:35 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Compute_service record updated for user:user {{(pid=72351) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Aug 09 03:01:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.212s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 03:01:36 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:01:36 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:01:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:01:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:01:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5003 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 03:01:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:01:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:01:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:01:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:01:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:01:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5003 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 03:01:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:01:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:01:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:01:45 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:01:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:01:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:01:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 03:01:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:01:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:01:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:01:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:01:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:01:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 03:01:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:01:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:01:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:01:59 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:02:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:02:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:02:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5003 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 03:02:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:02:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:02:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:02:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:02:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:02:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:02:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 03:02:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:02:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:02:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:02:19 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:02:19 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:02:19 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 03:02:19 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:02:19 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:02:19 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:02:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:02:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:02:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 03:02:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:02:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:02:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:02:26 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:02:27 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:02:27 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Starting heal instance info cache {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9842}}
Aug 09 03:02:27 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Rebuilding the list of instances to heal {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9846}}
Aug 09 03:02:27 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "refresh_cache-56b80e5d-b7f4-4780-987e-5def6d299f3d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 03:02:27 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquired lock "refresh_cache-56b80e5d-b7f4-4780-987e-5def6d299f3d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 03:02:27 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Forcefully refreshing network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1996}}
Aug 09 03:02:27 user nova-compute[72351]: DEBUG nova.objects.instance [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lazy-loading 'info_cache' on Instance uuid 56b80e5d-b7f4-4780-987e-5def6d299f3d {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 03:02:28 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Updating instance_info_cache with network_info: [{"id": "c69c5267-24fa-40f1-8f76-b4f2ec1ced93", "address": "fa:16:3e:ea:18:dd", "network": {"id": "45ee1620-bd54-4ca8-ab3a-9222782aea6f", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-173232413-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a636abaff2944499911a14ffb2846b48", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc69c5267-24", "ovs_interfaceid": "c69c5267-24fa-40f1-8f76-b4f2ec1ced93", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 03:02:28 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Releasing lock "refresh_cache-56b80e5d-b7f4-4780-987e-5def6d299f3d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 03:02:28 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Updated the network info_cache for instance {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9913}}
Aug 09 03:02:29 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:02:29 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:02:29 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5003 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 03:02:29 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:02:29 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:02:29 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:02:30 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:02:30 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72351) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10461}}
Aug 09 03:02:32 user nova-compute[72351]: DEBUG nova.compute.manager [req-84517146-43d1-48d3-9a2e-3ac0907bf54b req-a6dd92a3-7239-4709-98a9-af1110ca8fe7 service nova] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Received event network-changed-c69c5267-24fa-40f1-8f76-b4f2ec1ced93 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 03:02:32 user nova-compute[72351]: DEBUG nova.compute.manager [req-84517146-43d1-48d3-9a2e-3ac0907bf54b req-a6dd92a3-7239-4709-98a9-af1110ca8fe7 service nova] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Refreshing instance network info cache due to event network-changed-c69c5267-24fa-40f1-8f76-b4f2ec1ced93. {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11037}}
Aug 09 03:02:32 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-84517146-43d1-48d3-9a2e-3ac0907bf54b req-a6dd92a3-7239-4709-98a9-af1110ca8fe7 service nova] Acquiring lock "refresh_cache-56b80e5d-b7f4-4780-987e-5def6d299f3d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 03:02:32 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-84517146-43d1-48d3-9a2e-3ac0907bf54b req-a6dd92a3-7239-4709-98a9-af1110ca8fe7 service nova] Acquired lock "refresh_cache-56b80e5d-b7f4-4780-987e-5def6d299f3d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 03:02:32 user nova-compute[72351]: DEBUG nova.network.neutron [req-84517146-43d1-48d3-9a2e-3ac0907bf54b req-a6dd92a3-7239-4709-98a9-af1110ca8fe7 service nova] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Refreshing network info cache for port c69c5267-24fa-40f1-8f76-b4f2ec1ced93 {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1999}}
Aug 09 03:02:32 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:02:32 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:02:33 user nova-compute[72351]: DEBUG nova.network.neutron [req-84517146-43d1-48d3-9a2e-3ac0907bf54b req-a6dd92a3-7239-4709-98a9-af1110ca8fe7 service nova] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Updated VIF entry in instance network info cache for port c69c5267-24fa-40f1-8f76-b4f2ec1ced93. {{(pid=72351) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3474}}
Aug 09 03:02:33 user nova-compute[72351]: DEBUG nova.network.neutron [req-84517146-43d1-48d3-9a2e-3ac0907bf54b req-a6dd92a3-7239-4709-98a9-af1110ca8fe7 service nova] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Updating instance_info_cache with network_info: [{"id": "c69c5267-24fa-40f1-8f76-b4f2ec1ced93", "address": "fa:16:3e:ea:18:dd", "network": {"id": "45ee1620-bd54-4ca8-ab3a-9222782aea6f", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-173232413-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.86", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a636abaff2944499911a14ffb2846b48", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc69c5267-24", "ovs_interfaceid": "c69c5267-24fa-40f1-8f76-b4f2ec1ced93", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 03:02:33 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-84517146-43d1-48d3-9a2e-3ac0907bf54b req-a6dd92a3-7239-4709-98a9-af1110ca8fe7 service nova] Releasing lock "refresh_cache-56b80e5d-b7f4-4780-987e-5def6d299f3d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 03:02:34 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:02:34 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:02:34 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:02:36 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:02:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 03:02:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 03:02:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 03:02:36 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Auditing locally available compute resources for user (node: user) {{(pid=72351) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Aug 09 03:02:37 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56b80e5d-b7f4-4780-987e-5def6d299f3d/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:02:37 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56b80e5d-b7f4-4780-987e-5def6d299f3d/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:02:37 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56b80e5d-b7f4-4780-987e-5def6d299f3d/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:02:37 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56b80e5d-b7f4-4780-987e-5def6d299f3d/disk --force-share --output=json" returned: 0 in 0.217s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:02:37 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f6da9ec8-d942-4fc5-91ec-64af11b7cf01 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Acquiring lock "56b80e5d-b7f4-4780-987e-5def6d299f3d" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 03:02:37 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f6da9ec8-d942-4fc5-91ec-64af11b7cf01 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "56b80e5d-b7f4-4780-987e-5def6d299f3d" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 03:02:37 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 03:02:37 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 03:02:37 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Hypervisor/Node resource view: name=user free_ram=8786MB free_disk=25.167686462402344GB free_vcpus=11 pci_devices=[{"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72351) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Aug 09 03:02:37 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 03:02:37 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 03:02:37 user nova-compute[72351]: DEBUG nova.objects.instance [None req-f6da9ec8-d942-4fc5-91ec-64af11b7cf01 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lazy-loading 'flavor' on Instance uuid 56b80e5d-b7f4-4780-987e-5def6d299f3d {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 03:02:37 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f6da9ec8-d942-4fc5-91ec-64af11b7cf01 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "56b80e5d-b7f4-4780-987e-5def6d299f3d" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.068s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 03:02:37 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 56b80e5d-b7f4-4780-987e-5def6d299f3d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 03:02:37 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Total usable vcpus: 12, total allocated vcpus: 1 {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Aug 09 03:02:37 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Final resource view: name=user phys_ram=16011MB used_ram=1024MB phys_disk=40GB used_disk=1GB total_vcpus=12 used_vcpus=1 pci_stats=[] {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Aug 09 03:02:38 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 03:02:38 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 03:02:38 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Compute_service record updated for user:user {{(pid=72351) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Aug 09 03:02:38 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.218s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 03:02:38 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f6da9ec8-d942-4fc5-91ec-64af11b7cf01 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Acquiring lock "56b80e5d-b7f4-4780-987e-5def6d299f3d" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 03:02:38 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f6da9ec8-d942-4fc5-91ec-64af11b7cf01 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "56b80e5d-b7f4-4780-987e-5def6d299f3d" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 03:02:38 user nova-compute[72351]: INFO nova.compute.manager [None req-f6da9ec8-d942-4fc5-91ec-64af11b7cf01 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Attaching volume d42522d0-d07f-4000-bbb5-745318e48605 to /dev/vdb
Aug 09 03:02:38 user nova-compute[72351]: DEBUG os_brick.utils [None req-f6da9ec8-d942-4fc5-91ec-64af11b7cf01 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 03:02:38 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:02:38 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.016s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:02:38 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[9dbae692-6b83-4a99-95c0-f263cff3fa0c]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:a52269602ff4\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:02:38 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:02:38 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:02:38 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[c58a7183-ecde-4b75-ae4b-bf3c7ab37383]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:02:38 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:02:38 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:02:38 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[94df26b2-822c-431a-8c8d-f2d2ce9c505c]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:02:38 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[c49c854b-f972-417e-a5ea-f5936c0b082a]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:02:38 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-f6da9ec8-d942-4fc5-91ec-64af11b7cf01 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Running cmd (subprocess): nvme version {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:02:38 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-f6da9ec8-d942-4fc5-91ec-64af11b7cf01 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] 'nvme version' failed. Not Retrying. {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 03:02:38 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.nvmeof [None req-f6da9ec8-d942-4fc5-91ec-64af11b7cf01 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] nvme not present on system {{(pid=72351) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Aug 09 03:02:38 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:02:38 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 03:02:38 user nova-compute[72351]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Aug 09 03:02:38 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[9ed1d926-9124-4d74-ad4a-eb0953631ec2]: (4, '') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:02:38 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-f6da9ec8-d942-4fc5-91ec-64af11b7cf01 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72351) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Aug 09 03:02:38 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-f6da9ec8-d942-4fc5-91ec-64af11b7cf01 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Aug 09 03:02:38 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-f6da9ec8-d942-4fc5-91ec-64af11b7cf01 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] LIGHTOS: no hostnqn found. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Aug 09 03:02:38 user nova-compute[72351]: DEBUG os_brick.utils [None req-f6da9ec8-d942-4fc5-91ec-64af11b7cf01 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] <== get_connector_properties: return (164ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:a52269602ff4', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 03:02:38 user nova-compute[72351]: DEBUG nova.virt.block_device [None req-f6da9ec8-d942-4fc5-91ec-64af11b7cf01 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Updating existing volume attachment record: 71f8efe6-1e00-47ae-ac00-b786f5da6c06 {{(pid=72351) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Aug 09 03:02:39 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:02:39 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:02:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:02:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:02:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:02:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:02:45 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-f6da9ec8-d942-4fc5-91ec-64af11b7cf01 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 03:02:45 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-f6da9ec8-d942-4fc5-91ec-64af11b7cf01 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:d42522d0-d07f-4000-bbb5-745318e48605'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'uUspTfAX', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 03:02:45 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-f6da9ec8-d942-4fc5-91ec-64af11b7cf01 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 03:02:45 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-f6da9ec8-d942-4fc5-91ec-64af11b7cf01 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 03:02:45 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-f6da9ec8-d942-4fc5-91ec-64af11b7cf01 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 03:02:45 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-f6da9ec8-d942-4fc5-91ec-64af11b7cf01 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 03:02:45 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-f6da9ec8-d942-4fc5-91ec-64af11b7cf01 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:d42522d0-d07f-4000-bbb5-745318e48605" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 03:02:45 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-f6da9ec8-d942-4fc5-91ec-64af11b7cf01 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:d42522d0-d07f-4000-bbb5-745318e48605" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 03:02:45 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-f6da9ec8-d942-4fc5-91ec-64af11b7cf01 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 03:02:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d42522d0-d07f-4000-bbb5-745318e48605 -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:02:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d42522d0-d07f-4000-bbb5-745318e48605 -p 172.16.0.220:3260" returned: 21 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:02:45 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[7672a6a1-54fc-456f-b568-67f2df0000cd]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:02:45 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-f6da9ec8-d942-4fc5-91ec-64af11b7cf01 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Aug 09 03:02:45 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 03:02:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d42522d0-d07f-4000-bbb5-745318e48605 -p 172.16.0.220:3260 --interface default --op new {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:02:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d42522d0-d07f-4000-bbb5-745318e48605 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:02:45 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ee3c0f42-bcc9-4051-8d43-e2406316c22b]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:d42522d0-d07f-4000-bbb5-745318e48605] added\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:02:45 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-f6da9ec8-d942-4fc5-91ec-64af11b7cf01 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:d42522d0-d07f-4000-bbb5-745318e48605] added
Aug 09 03:02:45 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 03:02:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d42522d0-d07f-4000-bbb5-745318e48605 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:02:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d42522d0-d07f-4000-bbb5-745318e48605 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:02:45 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[bc9e81cc-7c94-48c2-9511-0cbca003a054]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:02:45 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-f6da9ec8-d942-4fc5-91ec-64af11b7cf01 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 03:02:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d42522d0-d07f-4000-bbb5-745318e48605 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:02:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d42522d0-d07f-4000-bbb5-745318e48605 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:02:45 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[47cd5d8d-80a8-40a4-83d7-308bd51347ad]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:02:45 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-f6da9ec8-d942-4fc5-91ec-64af11b7cf01 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 03:02:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d42522d0-d07f-4000-bbb5-745318e48605 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v uUspTfAX {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:02:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d42522d0-d07f-4000-bbb5-745318e48605 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v uUspTfAX" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:02:45 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ac2c1188-26a4-4e01-b85d-389afc51f2e8]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:02:45 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-f6da9ec8-d942-4fc5-91ec-64af11b7cf01 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'uUspTfAX'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 03:02:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d42522d0-d07f-4000-bbb5-745318e48605 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:02:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d42522d0-d07f-4000-bbb5-745318e48605 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:02:45 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[4660ae38-a218-4d01-9566-a6a29f6e119d]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:02:45 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-f6da9ec8-d942-4fc5-91ec-64af11b7cf01 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 03:02:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:02:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 21 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:02:45 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e509991f-9f3b-4103-a15c-597b7187c868]: (4, ('', 'iscsiadm: No active sessions.\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:02:45 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-f6da9ec8-d942-4fc5-91ec-64af11b7cf01 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsiadm ('-m', 'session'): stdout= stderr=iscsiadm: No active sessions.
Aug 09 03:02:45 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 03:02:45 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-f6da9ec8-d942-4fc5-91ec-64af11b7cf01 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsi session list stdout= stderr=iscsiadm: No active sessions.
Aug 09 03:02:45 user nova-compute[72351]:  {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 03:02:45 user nova-compute[72351]: WARNING os_brick.initiator.connectors.iscsi [None req-f6da9ec8-d942-4fc5-91ec-64af11b7cf01 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsiadm stderr output when getting sessions: iscsiadm: No active sessions.
Aug 09 03:02:45 user nova-compute[72351]: 
Aug 09 03:02:45 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d42522d0-d07f-4000-bbb5-745318e48605 -p 172.16.0.220:3260 --login {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:02:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d42522d0-d07f-4000-bbb5-745318e48605 -p 172.16.0.220:3260 --login" returned: 0 in 0.045s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:02:46 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[c2752556-8bb9-47d5-96ba-232e10156a18]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:d42522d0-d07f-4000-bbb5-745318e48605, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:d42522d0-d07f-4000-bbb5-745318e48605, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:02:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-f6da9ec8-d942-4fc5-91ec-64af11b7cf01 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:d42522d0-d07f-4000-bbb5-745318e48605, portal: 172.16.0.220,3260]
Aug 09 03:02:46 user nova-compute[72351]: Login to [iface: default, target: iqn.2010-10.org.openstack:d42522d0-d07f-4000-bbb5-745318e48605, portal: 172.16.0.220,3260] successful.
Aug 09 03:02:46 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 03:02:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d42522d0-d07f-4000-bbb5-745318e48605 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:02:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d42522d0-d07f-4000-bbb5-745318e48605 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.024s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:02:46 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[67faa468-c471-4e9a-8cc0-41a091015d57]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:02:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-f6da9ec8-d942-4fc5-91ec-64af11b7cf01 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 03:02:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:02:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:02:46 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[fde76c83-cf13-4f94-a71f-88d73fa8affe]: (4, ('tcp: [67] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d42522d0-d07f-4000-bbb5-745318e48605 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:02:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-f6da9ec8-d942-4fc5-91ec-64af11b7cf01 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [67] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d42522d0-d07f-4000-bbb5-745318e48605 (non-flash)
Aug 09 03:02:46 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 03:02:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-f6da9ec8-d942-4fc5-91ec-64af11b7cf01 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsi session list stdout=tcp: [67] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d42522d0-d07f-4000-bbb5-745318e48605 (non-flash)
Aug 09 03:02:46 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 03:02:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-f6da9ec8-d942-4fc5-91ec-64af11b7cf01 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:d42522d0-d07f-4000-bbb5-745318e48605" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.257s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 03:02:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-f6da9ec8-d942-4fc5-91ec-64af11b7cf01 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 03:02:46 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-f6da9ec8-d942-4fc5-91ec-64af11b7cf01 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] HCTL ('33', '-', '-', 0) found on session 67 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 03:02:46 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-f6da9ec8-d942-4fc5-91ec-64af11b7cf01 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 03:02:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:02:46 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:02:46 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[1d020c12-a5e7-4e73-86be-35884e4fe61f]: (4, ('- - 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:02:46 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-f6da9ec8-d942-4fc5-91ec-64af11b7cf01 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Searching for a device in session 67 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 03:02:46 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-f6da9ec8-d942-4fc5-91ec-64af11b7cf01 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Connected to sdb using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'uUspTfAX', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 03:02:47 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-f6da9ec8-d942-4fc5-91ec-64af11b7cf01 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.288s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 03:02:47 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-f6da9ec8-d942-4fc5-91ec-64af11b7cf01 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] <== connect_volume: return (1290ms) {'type': 'block', 'scsi_wwn': '23634376461656538', 'path': '/dev/sdb'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 03:02:47 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-f6da9ec8-d942-4fc5-91ec-64af11b7cf01 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23634376461656538', 'path': '/dev/sdb'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 03:02:47 user nova-compute[72351]: DEBUG nova.objects.instance [None req-f6da9ec8-d942-4fc5-91ec-64af11b7cf01 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lazy-loading 'flavor' on Instance uuid 56b80e5d-b7f4-4780-987e-5def6d299f3d {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 03:02:47 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-f6da9ec8-d942-4fc5-91ec-64af11b7cf01 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] attach device xml: <disk type="block" device="disk">
Aug 09 03:02:47 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 03:02:47 user nova-compute[72351]:   <source dev="/dev/sdb"/>
Aug 09 03:02:47 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 03:02:47 user nova-compute[72351]:   <serial>d42522d0-d07f-4000-bbb5-745318e48605</serial>
Aug 09 03:02:47 user nova-compute[72351]: </disk>
Aug 09 03:02:47 user nova-compute[72351]:  {{(pid=72351) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Aug 09 03:02:47 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-f6da9ec8-d942-4fc5-91ec-64af11b7cf01 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 03:02:47 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-f6da9ec8-d942-4fc5-91ec-64af11b7cf01 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 03:02:47 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-f6da9ec8-d942-4fc5-91ec-64af11b7cf01 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] No VIF found with MAC fa:16:3e:ea:18:dd, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 03:02:47 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-f6da9ec8-d942-4fc5-91ec-64af11b7cf01 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "56b80e5d-b7f4-4780-987e-5def6d299f3d" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 9.619s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 03:02:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:02:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:02:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:02:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:02:59 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:02:59 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:03:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:03:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:03:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:03:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:03:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 03:03:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:03:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:03:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:03:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1235c670-d24a-45f0-9c70-9c1090344f97 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Acquiring lock "56b80e5d-b7f4-4780-987e-5def6d299f3d" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 03:03:10 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1235c670-d24a-45f0-9c70-9c1090344f97 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "56b80e5d-b7f4-4780-987e-5def6d299f3d" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 03:03:10 user nova-compute[72351]: INFO nova.compute.manager [None req-1235c670-d24a-45f0-9c70-9c1090344f97 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Detaching volume d42522d0-d07f-4000-bbb5-745318e48605
Aug 09 03:03:10 user nova-compute[72351]: INFO nova.virt.block_device [None req-1235c670-d24a-45f0-9c70-9c1090344f97 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Attempting to driver detach volume d42522d0-d07f-4000-bbb5-745318e48605 from mountpoint /dev/vdb
Aug 09 03:03:10 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-1235c670-d24a-45f0-9c70-9c1090344f97 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Attempting to detach device vdb from instance 56b80e5d-b7f4-4780-987e-5def6d299f3d from the persistent domain config. {{(pid=72351) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2477}}
Aug 09 03:03:10 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-1235c670-d24a-45f0-9c70-9c1090344f97 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 03:03:10 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 03:03:10 user nova-compute[72351]:   <source dev="/dev/sdb"/>
Aug 09 03:03:10 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 03:03:10 user nova-compute[72351]:   <serial>d42522d0-d07f-4000-bbb5-745318e48605</serial>
Aug 09 03:03:10 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Aug 09 03:03:10 user nova-compute[72351]: </disk>
Aug 09 03:03:10 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 03:03:10 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-1235c670-d24a-45f0-9c70-9c1090344f97 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Successfully detached device vdb from instance 56b80e5d-b7f4-4780-987e-5def6d299f3d from the persistent domain config.
Aug 09 03:03:10 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-1235c670-d24a-45f0-9c70-9c1090344f97 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 56b80e5d-b7f4-4780-987e-5def6d299f3d from the live domain config. {{(pid=72351) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2513}}
Aug 09 03:03:10 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-1235c670-d24a-45f0-9c70-9c1090344f97 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 03:03:10 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 03:03:10 user nova-compute[72351]:   <source dev="/dev/sdb"/>
Aug 09 03:03:10 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 03:03:10 user nova-compute[72351]:   <serial>d42522d0-d07f-4000-bbb5-745318e48605</serial>
Aug 09 03:03:10 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Aug 09 03:03:10 user nova-compute[72351]: </disk>
Aug 09 03:03:10 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 03:03:10 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Received event <DeviceRemovedEvent: 1691550190.7745411, 56b80e5d-b7f4-4780-987e-5def6d299f3d => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=72351) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2360}}
Aug 09 03:03:10 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-1235c670-d24a-45f0-9c70-9c1090344f97 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 56b80e5d-b7f4-4780-987e-5def6d299f3d {{(pid=72351) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2589}}
Aug 09 03:03:10 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-1235c670-d24a-45f0-9c70-9c1090344f97 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Successfully detached device vdb from instance 56b80e5d-b7f4-4780-987e-5def6d299f3d from the live domain config.
Aug 09 03:03:10 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-1235c670-d24a-45f0-9c70-9c1090344f97 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 03:03:10 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1235c670-d24a-45f0-9c70-9c1090344f97 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:d42522d0-d07f-4000-bbb5-745318e48605'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'uUspTfAX', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb'}, None), 'kwargs': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 03:03:10 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-1235c670-d24a-45f0-9c70-9c1090344f97 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 03:03:10 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-1235c670-d24a-45f0-9c70-9c1090344f97 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 03:03:10 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-1235c670-d24a-45f0-9c70-9c1090344f97 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 03:03:10 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1235c670-d24a-45f0-9c70-9c1090344f97 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:d42522d0-d07f-4000-bbb5-745318e48605', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 03:03:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:03:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:03:10 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[eacad252-decd-4cff-b169-3974cd55c33f]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:d42522d0-d07f-4000-bbb5-745318e48605\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:03:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:03:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:03:10 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a2065411-7c8e-4ca9-857c-be1c66edf06b]: (4, ('tcp: [67] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d42522d0-d07f-4000-bbb5-745318e48605 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:03:10 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1235c670-d24a-45f0-9c70-9c1090344f97 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [67] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d42522d0-d07f-4000-bbb5-745318e48605 (non-flash)
Aug 09 03:03:10 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 03:03:10 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1235c670-d24a-45f0-9c70-9c1090344f97 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsi session list stdout=tcp: [67] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d42522d0-d07f-4000-bbb5-745318e48605 (non-flash)
Aug 09 03:03:10 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 03:03:10 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1235c670-d24a-45f0-9c70-9c1090344f97 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fcfdc1bd630>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:d42522d0-d07f-4000-bbb5-745318e48605'): ({'sdb'}, set())}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 03:03:10 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-1235c670-d24a-45f0-9c70-9c1090344f97 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Removing single pathed devices sdb {{(pid=72351) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Aug 09 03:03:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:03:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:03:10 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[9c76017b-1226-4191-b121-e775cc288462]: (4, ('path checker states:\nup                  1\n\npaths: 0\nbusy: False\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:03:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:03:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:03:10 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[b5ad786b-964e-4100-a477-8de4839f301f]: (4, ('ok\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:03:10 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-1235c670-d24a-45f0-9c70-9c1090344f97 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Flushing IO for device /dev/sdb {{(pid=72351) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Aug 09 03:03:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:03:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:03:10 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[dec369d0-0478-49b6-bcfe-685d6ff4ec0d]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:03:10 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-1235c670-d24a-45f0-9c70-9c1090344f97 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=72351) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Aug 09 03:03:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:03:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.036s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:03:10 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e8adfe39-53af-45b8-9e7e-f53704fcbe80]: (4, ('1', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:03:10 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-1235c670-d24a-45f0-9c70-9c1090344f97 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Aug 09 03:03:10 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-1235c670-d24a-45f0-9c70-9c1090344f97 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] SCSI volumes sdb have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Aug 09 03:03:10 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1235c670-d24a-45f0-9c70-9c1090344f97 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:d42522d0-d07f-4000-bbb5-745318e48605')] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 03:03:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d42522d0-d07f-4000-bbb5-745318e48605 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:03:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d42522d0-d07f-4000-bbb5-745318e48605 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:03:10 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[528fb995-1a13-491b-95da-bc3f707ba316]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:03:10 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1235c670-d24a-45f0-9c70-9c1090344f97 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 03:03:10 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d42522d0-d07f-4000-bbb5-745318e48605 -p 172.16.0.220:3260 --logout {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:03:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d42522d0-d07f-4000-bbb5-745318e48605 -p 172.16.0.220:3260 --logout" returned: 0 in 0.072s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:03:11 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[2904b779-db7a-4910-9ef0-118ebaa8281f]: (4, ('Logging out of session [sid: 67, target: iqn.2010-10.org.openstack:d42522d0-d07f-4000-bbb5-745318e48605, portal: 172.16.0.220,3260]\nLogout of [sid: 67, target: iqn.2010-10.org.openstack:d42522d0-d07f-4000-bbb5-745318e48605, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:03:11 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1235c670-d24a-45f0-9c70-9c1090344f97 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 67, target: iqn.2010-10.org.openstack:d42522d0-d07f-4000-bbb5-745318e48605, portal: 172.16.0.220,3260]
Aug 09 03:03:11 user nova-compute[72351]: Logout of [sid: 67, target: iqn.2010-10.org.openstack:d42522d0-d07f-4000-bbb5-745318e48605, portal: 172.16.0.220,3260] successful.
Aug 09 03:03:11 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 03:03:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d42522d0-d07f-4000-bbb5-745318e48605 -p 172.16.0.220:3260 --op delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:03:11 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d42522d0-d07f-4000-bbb5-745318e48605 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:03:11 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[8c4d320c-13f3-4394-b4d5-10209285d141]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:03:11 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1235c670-d24a-45f0-9c70-9c1090344f97 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 03:03:11 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-1235c670-d24a-45f0-9c70-9c1090344f97 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.286s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 03:03:11 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1235c670-d24a-45f0-9c70-9c1090344f97 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] <== disconnect_volume: return (287ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 03:03:11 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-1235c670-d24a-45f0-9c70-9c1090344f97 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 03:03:11 user nova-compute[72351]: DEBUG nova.objects.instance [None req-1235c670-d24a-45f0-9c70-9c1090344f97 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lazy-loading 'flavor' on Instance uuid 56b80e5d-b7f4-4780-987e-5def6d299f3d {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 03:03:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1235c670-d24a-45f0-9c70-9c1090344f97 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "56b80e5d-b7f4-4780-987e-5def6d299f3d" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.534s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 03:03:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:03:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:03:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 03:03:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:03:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:03:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:03:19 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:03:19 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:03:19 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5003 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 03:03:19 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:03:19 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:03:19 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:03:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:03:27 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:03:29 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:03:29 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Starting heal instance info cache {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9842}}
Aug 09 03:03:29 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Rebuilding the list of instances to heal {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9846}}
Aug 09 03:03:29 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:03:29 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:03:29 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 03:03:29 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:03:29 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:03:29 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:03:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "refresh_cache-56b80e5d-b7f4-4780-987e-5def6d299f3d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 03:03:29 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquired lock "refresh_cache-56b80e5d-b7f4-4780-987e-5def6d299f3d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 03:03:29 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Forcefully refreshing network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1996}}
Aug 09 03:03:29 user nova-compute[72351]: DEBUG nova.objects.instance [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lazy-loading 'info_cache' on Instance uuid 56b80e5d-b7f4-4780-987e-5def6d299f3d {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 03:03:30 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Updating instance_info_cache with network_info: [{"id": "c69c5267-24fa-40f1-8f76-b4f2ec1ced93", "address": "fa:16:3e:ea:18:dd", "network": {"id": "45ee1620-bd54-4ca8-ab3a-9222782aea6f", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-173232413-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.86", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a636abaff2944499911a14ffb2846b48", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc69c5267-24", "ovs_interfaceid": "c69c5267-24fa-40f1-8f76-b4f2ec1ced93", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 03:03:30 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Releasing lock "refresh_cache-56b80e5d-b7f4-4780-987e-5def6d299f3d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 03:03:30 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Updated the network info_cache for instance {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9913}}
Aug 09 03:03:32 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:03:32 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72351) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10461}}
Aug 09 03:03:33 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:03:33 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:03:34 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:03:34 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:03:37 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:03:37 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:03:37 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 03:03:37 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 03:03:37 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 03:03:37 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Auditing locally available compute resources for user (node: user) {{(pid=72351) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Aug 09 03:03:38 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56b80e5d-b7f4-4780-987e-5def6d299f3d/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:03:38 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56b80e5d-b7f4-4780-987e-5def6d299f3d/disk --force-share --output=json" returned: 0 in 0.146s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:03:38 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56b80e5d-b7f4-4780-987e-5def6d299f3d/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:03:38 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56b80e5d-b7f4-4780-987e-5def6d299f3d/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:03:38 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 03:03:38 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 03:03:38 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Hypervisor/Node resource view: name=user free_ram=8772MB free_disk=25.158607482910156GB free_vcpus=11 pci_devices=[{"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72351) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Aug 09 03:03:38 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 03:03:38 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 03:03:38 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 56b80e5d-b7f4-4780-987e-5def6d299f3d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 03:03:38 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Total usable vcpus: 12, total allocated vcpus: 1 {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Aug 09 03:03:38 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Final resource view: name=user phys_ram=16011MB used_ram=1024MB phys_disk=40GB used_disk=1GB total_vcpus=12 used_vcpus=1 pci_stats=[] {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Aug 09 03:03:38 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 03:03:38 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 03:03:38 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Compute_service record updated for user:user {{(pid=72351) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Aug 09 03:03:38 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.164s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 03:03:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:03:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:03:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5003 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 03:03:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:03:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:03:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:03:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:03:40 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:03:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4997-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:03:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:03:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 03:03:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:03:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:03:44 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:03:49 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._cleanup_expired_console_auth_tokens {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:03:49 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:03:50 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:03:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:03:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:03:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5004 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 03:03:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:03:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:03:54 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:03:58 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._run_pending_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:03:58 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Cleaning up deleted instances {{(pid=72351) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11129}}
Aug 09 03:03:58 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] There are 0 instances to clean {{(pid=72351) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11138}}
Aug 09 03:03:59 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:03:59 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:03:59 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 03:03:59 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:03:59 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:03:59 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:04:02 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._cleanup_incomplete_migrations {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:04:02 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Cleaning up deleted instances with incomplete migration  {{(pid=72351) _cleanup_incomplete_migrations /opt/stack/nova/nova/compute/manager.py:11167}}
Aug 09 03:04:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:04:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e0291e02-e5ad-47d2-bb1c-2bbac7e5aa23 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Acquiring lock "56b80e5d-b7f4-4780-987e-5def6d299f3d" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 03:04:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e0291e02-e5ad-47d2-bb1c-2bbac7e5aa23 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "56b80e5d-b7f4-4780-987e-5def6d299f3d" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 03:04:04 user nova-compute[72351]: DEBUG nova.objects.instance [None req-e0291e02-e5ad-47d2-bb1c-2bbac7e5aa23 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lazy-loading 'flavor' on Instance uuid 56b80e5d-b7f4-4780-987e-5def6d299f3d {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 03:04:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e0291e02-e5ad-47d2-bb1c-2bbac7e5aa23 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "56b80e5d-b7f4-4780-987e-5def6d299f3d" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.057s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 03:04:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e0291e02-e5ad-47d2-bb1c-2bbac7e5aa23 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Acquiring lock "56b80e5d-b7f4-4780-987e-5def6d299f3d" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 03:04:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e0291e02-e5ad-47d2-bb1c-2bbac7e5aa23 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "56b80e5d-b7f4-4780-987e-5def6d299f3d" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 03:04:04 user nova-compute[72351]: INFO nova.compute.manager [None req-e0291e02-e5ad-47d2-bb1c-2bbac7e5aa23 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Attaching volume 9af4ce59-6302-4dcc-8bb9-4178d26b47bc to /dev/vdb
Aug 09 03:04:04 user nova-compute[72351]: DEBUG os_brick.utils [None req-e0291e02-e5ad-47d2-bb1c-2bbac7e5aa23 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 03:04:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:04:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.013s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:04:04 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[8de8dbb7-7339-4ac5-a598-569eb0ee5c32]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:a52269602ff4\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:04:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:04:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.016s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:04:04 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a28f4df5-cae7-4925-8b36-b57bf844079f]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:04:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:04:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.016s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:04:04 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[886b0746-7320-415d-95ad-802bf19634ed]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:04:04 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[5fd1759a-7070-45db-848b-4c0b51c8fd55]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:04:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-e0291e02-e5ad-47d2-bb1c-2bbac7e5aa23 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Running cmd (subprocess): nvme version {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:04:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-e0291e02-e5ad-47d2-bb1c-2bbac7e5aa23 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] 'nvme version' failed. Not Retrying. {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 03:04:04 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.nvmeof [None req-e0291e02-e5ad-47d2-bb1c-2bbac7e5aa23 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] nvme not present on system {{(pid=72351) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Aug 09 03:04:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:04:04 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 03:04:04 user nova-compute[72351]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Aug 09 03:04:04 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[520784a4-f07e-4fde-b6aa-cf3eb5d30989]: (4, '') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:04:04 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-e0291e02-e5ad-47d2-bb1c-2bbac7e5aa23 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72351) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Aug 09 03:04:04 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-e0291e02-e5ad-47d2-bb1c-2bbac7e5aa23 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Aug 09 03:04:04 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-e0291e02-e5ad-47d2-bb1c-2bbac7e5aa23 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] LIGHTOS: no hostnqn found. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Aug 09 03:04:04 user nova-compute[72351]: DEBUG os_brick.utils [None req-e0291e02-e5ad-47d2-bb1c-2bbac7e5aa23 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] <== get_connector_properties: return (122ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:a52269602ff4', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 03:04:04 user nova-compute[72351]: DEBUG nova.virt.block_device [None req-e0291e02-e5ad-47d2-bb1c-2bbac7e5aa23 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Updating existing volume attachment record: 92139822-be08-4f59-a3c0-e3c979833425 {{(pid=72351) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Aug 09 03:04:04 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:04:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:04:09 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:04:12 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-e0291e02-e5ad-47d2-bb1c-2bbac7e5aa23 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 03:04:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e0291e02-e5ad-47d2-bb1c-2bbac7e5aa23 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:9af4ce59-6302-4dcc-8bb9-4178d26b47bc'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'jDQtImZl', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 03:04:12 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-e0291e02-e5ad-47d2-bb1c-2bbac7e5aa23 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 03:04:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-e0291e02-e5ad-47d2-bb1c-2bbac7e5aa23 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 03:04:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-e0291e02-e5ad-47d2-bb1c-2bbac7e5aa23 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 03:04:12 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-e0291e02-e5ad-47d2-bb1c-2bbac7e5aa23 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 03:04:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-e0291e02-e5ad-47d2-bb1c-2bbac7e5aa23 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:9af4ce59-6302-4dcc-8bb9-4178d26b47bc" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 03:04:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-e0291e02-e5ad-47d2-bb1c-2bbac7e5aa23 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:9af4ce59-6302-4dcc-8bb9-4178d26b47bc" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 03:04:12 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-e0291e02-e5ad-47d2-bb1c-2bbac7e5aa23 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 03:04:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9af4ce59-6302-4dcc-8bb9-4178d26b47bc -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:04:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9af4ce59-6302-4dcc-8bb9-4178d26b47bc -p 172.16.0.220:3260" returned: 21 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:04:12 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[149b4be7-3ef4-49d7-bd13-8f5212edd32a]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:04:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e0291e02-e5ad-47d2-bb1c-2bbac7e5aa23 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Aug 09 03:04:12 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 03:04:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9af4ce59-6302-4dcc-8bb9-4178d26b47bc -p 172.16.0.220:3260 --interface default --op new {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:04:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9af4ce59-6302-4dcc-8bb9-4178d26b47bc -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:04:12 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[69cf022d-54b5-4353-a098-f1fe39287a8e]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:9af4ce59-6302-4dcc-8bb9-4178d26b47bc] added\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:04:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e0291e02-e5ad-47d2-bb1c-2bbac7e5aa23 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:9af4ce59-6302-4dcc-8bb9-4178d26b47bc] added
Aug 09 03:04:12 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 03:04:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9af4ce59-6302-4dcc-8bb9-4178d26b47bc -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:04:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9af4ce59-6302-4dcc-8bb9-4178d26b47bc -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.030s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:04:12 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[97a96c53-34bc-4d60-a9fa-4c3f01398b14]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:04:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e0291e02-e5ad-47d2-bb1c-2bbac7e5aa23 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 03:04:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9af4ce59-6302-4dcc-8bb9-4178d26b47bc -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:04:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9af4ce59-6302-4dcc-8bb9-4178d26b47bc -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.025s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:04:12 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[81bc3913-3511-47ae-9694-886254dfbf33]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:04:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e0291e02-e5ad-47d2-bb1c-2bbac7e5aa23 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 03:04:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9af4ce59-6302-4dcc-8bb9-4178d26b47bc -p 172.16.0.220:3260 --op update -n node.session.auth.username -v jDQtImZl {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:04:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9af4ce59-6302-4dcc-8bb9-4178d26b47bc -p 172.16.0.220:3260 --op update -n node.session.auth.username -v jDQtImZl" returned: 0 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:04:12 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[0df85daa-b0d3-43b1-b6c1-10489712d42e]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:04:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e0291e02-e5ad-47d2-bb1c-2bbac7e5aa23 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'jDQtImZl'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 03:04:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9af4ce59-6302-4dcc-8bb9-4178d26b47bc -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:04:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9af4ce59-6302-4dcc-8bb9-4178d26b47bc -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:04:12 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[ca6338f2-3669-48c9-904c-d042c802eed9]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:04:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e0291e02-e5ad-47d2-bb1c-2bbac7e5aa23 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 03:04:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:04:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 21 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:04:12 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[4f5ff1e8-140e-42da-9d1d-ba84f472e8a6]: (4, ('', 'iscsiadm: No active sessions.\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:04:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e0291e02-e5ad-47d2-bb1c-2bbac7e5aa23 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsiadm ('-m', 'session'): stdout= stderr=iscsiadm: No active sessions.
Aug 09 03:04:12 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 03:04:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e0291e02-e5ad-47d2-bb1c-2bbac7e5aa23 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsi session list stdout= stderr=iscsiadm: No active sessions.
Aug 09 03:04:12 user nova-compute[72351]:  {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 03:04:12 user nova-compute[72351]: WARNING os_brick.initiator.connectors.iscsi [None req-e0291e02-e5ad-47d2-bb1c-2bbac7e5aa23 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsiadm stderr output when getting sessions: iscsiadm: No active sessions.
Aug 09 03:04:12 user nova-compute[72351]: 
Aug 09 03:04:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9af4ce59-6302-4dcc-8bb9-4178d26b47bc -p 172.16.0.220:3260 --login {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:04:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9af4ce59-6302-4dcc-8bb9-4178d26b47bc -p 172.16.0.220:3260 --login" returned: 0 in 0.051s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:04:12 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[9fdc1509-cf34-4513-b171-62e05d2bf182]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:9af4ce59-6302-4dcc-8bb9-4178d26b47bc, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:9af4ce59-6302-4dcc-8bb9-4178d26b47bc, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:04:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e0291e02-e5ad-47d2-bb1c-2bbac7e5aa23 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:9af4ce59-6302-4dcc-8bb9-4178d26b47bc, portal: 172.16.0.220,3260]
Aug 09 03:04:12 user nova-compute[72351]: Login to [iface: default, target: iqn.2010-10.org.openstack:9af4ce59-6302-4dcc-8bb9-4178d26b47bc, portal: 172.16.0.220,3260] successful.
Aug 09 03:04:12 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 03:04:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9af4ce59-6302-4dcc-8bb9-4178d26b47bc -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:04:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9af4ce59-6302-4dcc-8bb9-4178d26b47bc -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:04:12 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a4a15a48-66b0-45aa-b0b3-d349e718f890]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:04:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e0291e02-e5ad-47d2-bb1c-2bbac7e5aa23 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 03:04:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:04:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:04:12 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[9e1cb8fa-46a6-4e59-9df5-f25d23b87792]: (4, ('tcp: [68] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9af4ce59-6302-4dcc-8bb9-4178d26b47bc (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:04:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e0291e02-e5ad-47d2-bb1c-2bbac7e5aa23 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [68] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9af4ce59-6302-4dcc-8bb9-4178d26b47bc (non-flash)
Aug 09 03:04:12 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 03:04:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e0291e02-e5ad-47d2-bb1c-2bbac7e5aa23 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsi session list stdout=tcp: [68] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9af4ce59-6302-4dcc-8bb9-4178d26b47bc (non-flash)
Aug 09 03:04:12 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 03:04:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-e0291e02-e5ad-47d2-bb1c-2bbac7e5aa23 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:9af4ce59-6302-4dcc-8bb9-4178d26b47bc" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.291s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 03:04:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e0291e02-e5ad-47d2-bb1c-2bbac7e5aa23 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 03:04:12 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-e0291e02-e5ad-47d2-bb1c-2bbac7e5aa23 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] HCTL ('33', '-', '-', 0) found on session 68 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 03:04:12 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-e0291e02-e5ad-47d2-bb1c-2bbac7e5aa23 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 03:04:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:04:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:04:12 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d2065000-91ee-4bd0-96be-20db76a4b793]: (4, ('- - 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:04:12 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-e0291e02-e5ad-47d2-bb1c-2bbac7e5aa23 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Searching for a device in session 68 and hctl ['33', '*', '*', 0] yield: None {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 03:04:13 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-e0291e02-e5ad-47d2-bb1c-2bbac7e5aa23 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Searching for a device in session 68 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 03:04:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e0291e02-e5ad-47d2-bb1c-2bbac7e5aa23 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Connected to sdb using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'jDQtImZl', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 03:04:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-e0291e02-e5ad-47d2-bb1c-2bbac7e5aa23 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.322s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 03:04:13 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-e0291e02-e5ad-47d2-bb1c-2bbac7e5aa23 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] <== connect_volume: return (1324ms) {'type': 'block', 'scsi_wwn': '23937636466613437', 'path': '/dev/sdb'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 03:04:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-e0291e02-e5ad-47d2-bb1c-2bbac7e5aa23 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23937636466613437', 'path': '/dev/sdb'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 03:04:13 user nova-compute[72351]: DEBUG nova.objects.instance [None req-e0291e02-e5ad-47d2-bb1c-2bbac7e5aa23 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lazy-loading 'flavor' on Instance uuid 56b80e5d-b7f4-4780-987e-5def6d299f3d {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 03:04:13 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-e0291e02-e5ad-47d2-bb1c-2bbac7e5aa23 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] attach device xml: <disk type="block" device="disk">
Aug 09 03:04:13 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 03:04:13 user nova-compute[72351]:   <source dev="/dev/sdb"/>
Aug 09 03:04:13 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 03:04:13 user nova-compute[72351]:   <serial>9af4ce59-6302-4dcc-8bb9-4178d26b47bc</serial>
Aug 09 03:04:13 user nova-compute[72351]: </disk>
Aug 09 03:04:13 user nova-compute[72351]:  {{(pid=72351) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Aug 09 03:04:14 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-e0291e02-e5ad-47d2-bb1c-2bbac7e5aa23 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 03:04:14 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-e0291e02-e5ad-47d2-bb1c-2bbac7e5aa23 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 03:04:14 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-e0291e02-e5ad-47d2-bb1c-2bbac7e5aa23 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] No VIF found with MAC fa:16:3e:ea:18:dd, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 03:04:14 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e0291e02-e5ad-47d2-bb1c-2bbac7e5aa23 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "56b80e5d-b7f4-4780-987e-5def6d299f3d" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 9.579s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 03:04:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:04:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:04:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 03:04:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:04:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:04:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:04:16 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-fa02311b-751c-408a-8ae6-76e331f383aa tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Acquiring lock "56b80e5d-b7f4-4780-987e-5def6d299f3d" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 03:04:16 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-fa02311b-751c-408a-8ae6-76e331f383aa tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "56b80e5d-b7f4-4780-987e-5def6d299f3d" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 03:04:16 user nova-compute[72351]: INFO nova.compute.manager [None req-fa02311b-751c-408a-8ae6-76e331f383aa tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Detaching volume 9af4ce59-6302-4dcc-8bb9-4178d26b47bc
Aug 09 03:04:16 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._sync_power_states {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:04:17 user nova-compute[72351]: INFO nova.virt.block_device [None req-fa02311b-751c-408a-8ae6-76e331f383aa tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Attempting to driver detach volume 9af4ce59-6302-4dcc-8bb9-4178d26b47bc from mountpoint /dev/vdb
Aug 09 03:04:17 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Triggering sync for uuid 56b80e5d-b7f4-4780-987e-5def6d299f3d {{(pid=72351) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10252}}
Aug 09 03:04:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "56b80e5d-b7f4-4780-987e-5def6d299f3d" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 03:04:17 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-fa02311b-751c-408a-8ae6-76e331f383aa tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Attempting to detach device vdb from instance 56b80e5d-b7f4-4780-987e-5def6d299f3d from the persistent domain config. {{(pid=72351) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2477}}
Aug 09 03:04:17 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-fa02311b-751c-408a-8ae6-76e331f383aa tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 03:04:17 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 03:04:17 user nova-compute[72351]:   <source dev="/dev/sdb"/>
Aug 09 03:04:17 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 03:04:17 user nova-compute[72351]:   <serial>9af4ce59-6302-4dcc-8bb9-4178d26b47bc</serial>
Aug 09 03:04:17 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Aug 09 03:04:17 user nova-compute[72351]: </disk>
Aug 09 03:04:17 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 03:04:17 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-fa02311b-751c-408a-8ae6-76e331f383aa tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Successfully detached device vdb from instance 56b80e5d-b7f4-4780-987e-5def6d299f3d from the persistent domain config.
Aug 09 03:04:17 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-fa02311b-751c-408a-8ae6-76e331f383aa tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 56b80e5d-b7f4-4780-987e-5def6d299f3d from the live domain config. {{(pid=72351) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2513}}
Aug 09 03:04:17 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-fa02311b-751c-408a-8ae6-76e331f383aa tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 03:04:17 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 03:04:17 user nova-compute[72351]:   <source dev="/dev/sdb"/>
Aug 09 03:04:17 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 03:04:17 user nova-compute[72351]:   <serial>9af4ce59-6302-4dcc-8bb9-4178d26b47bc</serial>
Aug 09 03:04:17 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Aug 09 03:04:17 user nova-compute[72351]: </disk>
Aug 09 03:04:17 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 03:04:17 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Received event <DeviceRemovedEvent: 1691550257.4846199, 56b80e5d-b7f4-4780-987e-5def6d299f3d => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=72351) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2360}}
Aug 09 03:04:17 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-fa02311b-751c-408a-8ae6-76e331f383aa tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 56b80e5d-b7f4-4780-987e-5def6d299f3d {{(pid=72351) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2589}}
Aug 09 03:04:17 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-fa02311b-751c-408a-8ae6-76e331f383aa tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Successfully detached device vdb from instance 56b80e5d-b7f4-4780-987e-5def6d299f3d from the live domain config.
Aug 09 03:04:17 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-fa02311b-751c-408a-8ae6-76e331f383aa tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 03:04:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa02311b-751c-408a-8ae6-76e331f383aa tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:9af4ce59-6302-4dcc-8bb9-4178d26b47bc'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'jDQtImZl', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb'}, None), 'kwargs': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 03:04:17 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-fa02311b-751c-408a-8ae6-76e331f383aa tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 03:04:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-fa02311b-751c-408a-8ae6-76e331f383aa tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 03:04:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-fa02311b-751c-408a-8ae6-76e331f383aa tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 03:04:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa02311b-751c-408a-8ae6-76e331f383aa tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:9af4ce59-6302-4dcc-8bb9-4178d26b47bc', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 03:04:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:04:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:04:17 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[9be92031-d719-425e-9b33-c374115d274f]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:9af4ce59-6302-4dcc-8bb9-4178d26b47bc\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:04:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:04:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:04:17 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[6b8fe00a-11f8-4282-ad93-c152f5b41037]: (4, ('tcp: [68] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9af4ce59-6302-4dcc-8bb9-4178d26b47bc (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:04:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa02311b-751c-408a-8ae6-76e331f383aa tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [68] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9af4ce59-6302-4dcc-8bb9-4178d26b47bc (non-flash)
Aug 09 03:04:17 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 03:04:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa02311b-751c-408a-8ae6-76e331f383aa tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsi session list stdout=tcp: [68] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9af4ce59-6302-4dcc-8bb9-4178d26b47bc (non-flash)
Aug 09 03:04:17 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 03:04:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa02311b-751c-408a-8ae6-76e331f383aa tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fcfdc18b010>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:9af4ce59-6302-4dcc-8bb9-4178d26b47bc'): ({'sdb'}, set())}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 03:04:17 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-fa02311b-751c-408a-8ae6-76e331f383aa tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Removing single pathed devices sdb {{(pid=72351) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Aug 09 03:04:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:04:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:04:17 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[5c01f474-0225-4ec8-a6fc-66b9ba31fe2e]: (4, ('path checker states:\nup                  1\n\npaths: 0\nbusy: False\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:04:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:04:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.024s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:04:17 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[2a151a97-4976-4c11-a60a-4e5b36a7b49b]: (4, ('ok\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:04:17 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-fa02311b-751c-408a-8ae6-76e331f383aa tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Flushing IO for device /dev/sdb {{(pid=72351) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Aug 09 03:04:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:04:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:04:17 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[5c3b42b0-6cb6-44a8-9563-e1aae401a325]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:04:17 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-fa02311b-751c-408a-8ae6-76e331f383aa tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=72351) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Aug 09 03:04:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:04:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.048s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:04:17 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[44b89f82-c32c-4e45-85d7-736544203dd9]: (4, ('1', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:04:17 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-fa02311b-751c-408a-8ae6-76e331f383aa tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Aug 09 03:04:17 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-fa02311b-751c-408a-8ae6-76e331f383aa tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] SCSI volumes sdb have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Aug 09 03:04:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa02311b-751c-408a-8ae6-76e331f383aa tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:9af4ce59-6302-4dcc-8bb9-4178d26b47bc')] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 03:04:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9af4ce59-6302-4dcc-8bb9-4178d26b47bc -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:04:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9af4ce59-6302-4dcc-8bb9-4178d26b47bc -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:04:17 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[77b02ca3-022d-413f-9128-8bbc71af540d]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:04:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa02311b-751c-408a-8ae6-76e331f383aa tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 03:04:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9af4ce59-6302-4dcc-8bb9-4178d26b47bc -p 172.16.0.220:3260 --logout {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:04:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9af4ce59-6302-4dcc-8bb9-4178d26b47bc -p 172.16.0.220:3260 --logout" returned: 0 in 0.062s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:04:17 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[1e690ca6-221c-43bc-86e2-f4768ceec83f]: (4, ('Logging out of session [sid: 68, target: iqn.2010-10.org.openstack:9af4ce59-6302-4dcc-8bb9-4178d26b47bc, portal: 172.16.0.220,3260]\nLogout of [sid: 68, target: iqn.2010-10.org.openstack:9af4ce59-6302-4dcc-8bb9-4178d26b47bc, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:04:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa02311b-751c-408a-8ae6-76e331f383aa tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 68, target: iqn.2010-10.org.openstack:9af4ce59-6302-4dcc-8bb9-4178d26b47bc, portal: 172.16.0.220,3260]
Aug 09 03:04:17 user nova-compute[72351]: Logout of [sid: 68, target: iqn.2010-10.org.openstack:9af4ce59-6302-4dcc-8bb9-4178d26b47bc, portal: 172.16.0.220,3260] successful.
Aug 09 03:04:17 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 03:04:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9af4ce59-6302-4dcc-8bb9-4178d26b47bc -p 172.16.0.220:3260 --op delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:04:17 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9af4ce59-6302-4dcc-8bb9-4178d26b47bc -p 172.16.0.220:3260 --op delete" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:04:17 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[1e2c590e-9d9b-4e80-8398-c28a1cb94620]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:04:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa02311b-751c-408a-8ae6-76e331f383aa tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 03:04:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-fa02311b-751c-408a-8ae6-76e331f383aa tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.291s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 03:04:17 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa02311b-751c-408a-8ae6-76e331f383aa tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] <== disconnect_volume: return (292ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 03:04:17 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-fa02311b-751c-408a-8ae6-76e331f383aa tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 03:04:18 user nova-compute[72351]: DEBUG nova.objects.instance [None req-fa02311b-751c-408a-8ae6-76e331f383aa tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lazy-loading 'flavor' on Instance uuid 56b80e5d-b7f4-4780-987e-5def6d299f3d {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 03:04:18 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-fa02311b-751c-408a-8ae6-76e331f383aa tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "56b80e5d-b7f4-4780-987e-5def6d299f3d" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.619s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 03:04:18 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "56b80e5d-b7f4-4780-987e-5def6d299f3d" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 1.541s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 03:04:18 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "56b80e5d-b7f4-4780-987e-5def6d299f3d" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.035s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 03:04:19 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:04:19 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:04:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:04:27 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:04:29 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:04:29 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:04:29 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 03:04:29 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:04:29 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:04:29 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:04:31 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:04:31 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Starting heal instance info cache {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9842}}
Aug 09 03:04:31 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Rebuilding the list of instances to heal {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9846}}
Aug 09 03:04:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "refresh_cache-56b80e5d-b7f4-4780-987e-5def6d299f3d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 03:04:31 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquired lock "refresh_cache-56b80e5d-b7f4-4780-987e-5def6d299f3d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 03:04:31 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Forcefully refreshing network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1996}}
Aug 09 03:04:31 user nova-compute[72351]: DEBUG nova.objects.instance [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lazy-loading 'info_cache' on Instance uuid 56b80e5d-b7f4-4780-987e-5def6d299f3d {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 03:04:32 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Updating instance_info_cache with network_info: [{"id": "c69c5267-24fa-40f1-8f76-b4f2ec1ced93", "address": "fa:16:3e:ea:18:dd", "network": {"id": "45ee1620-bd54-4ca8-ab3a-9222782aea6f", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-173232413-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.86", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a636abaff2944499911a14ffb2846b48", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc69c5267-24", "ovs_interfaceid": "c69c5267-24fa-40f1-8f76-b4f2ec1ced93", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 03:04:32 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Releasing lock "refresh_cache-56b80e5d-b7f4-4780-987e-5def6d299f3d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 03:04:32 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Updated the network info_cache for instance {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9913}}
Aug 09 03:04:32 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:04:32 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72351) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10461}}
Aug 09 03:04:34 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:04:34 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:04:35 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:04:35 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 03:04:35 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:04:35 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:04:35 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:04:35 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:04:35 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:04:38 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:04:39 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:04:39 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 03:04:39 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 03:04:39 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 03:04:39 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Auditing locally available compute resources for user (node: user) {{(pid=72351) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Aug 09 03:04:40 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4997-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:04:40 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:04:40 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 03:04:40 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:04:40 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:04:40 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:04:40 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56b80e5d-b7f4-4780-987e-5def6d299f3d/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:04:40 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56b80e5d-b7f4-4780-987e-5def6d299f3d/disk --force-share --output=json" returned: 0 in 0.140s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:04:40 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56b80e5d-b7f4-4780-987e-5def6d299f3d/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:04:40 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56b80e5d-b7f4-4780-987e-5def6d299f3d/disk --force-share --output=json" returned: 0 in 0.132s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:04:40 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 03:04:40 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 03:04:40 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Hypervisor/Node resource view: name=user free_ram=8775MB free_disk=25.156631469726562GB free_vcpus=11 pci_devices=[{"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72351) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Aug 09 03:04:40 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 03:04:40 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 03:04:41 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 56b80e5d-b7f4-4780-987e-5def6d299f3d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 03:04:41 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Total usable vcpus: 12, total allocated vcpus: 1 {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Aug 09 03:04:41 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Final resource view: name=user phys_ram=16011MB used_ram=1024MB phys_disk=40GB used_disk=1GB total_vcpus=12 used_vcpus=1 pci_stats=[] {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Aug 09 03:04:41 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Refreshing inventories for resource provider 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:804}}
Aug 09 03:04:41 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Updating ProviderTree inventory for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 from _refresh_and_get_inventory using data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) _refresh_and_get_inventory /opt/stack/nova/nova/scheduler/client/report.py:768}}
Aug 09 03:04:41 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Updating inventory in ProviderTree for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 with inventory: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:176}}
Aug 09 03:04:41 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Refreshing aggregate associations for resource provider 8eae9b37-1236-4e6d-91a5-b650de27b009, aggregates: None {{(pid=72351) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:813}}
Aug 09 03:04:41 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Refreshing trait associations for resource provider 8eae9b37-1236-4e6d-91a5-b650de27b009, traits: COMPUTE_VIOMMU_MODEL_INTEL,COMPUTE_STORAGE_BUS_IDE,COMPUTE_GRAPHICS_MODEL_NONE,COMPUTE_NET_VIF_MODEL_E1000E,COMPUTE_IMAGE_TYPE_AKI,COMPUTE_VOLUME_ATTACH_WITH_TAG,COMPUTE_NET_VIF_MODEL_VMXNET3,COMPUTE_SOCKET_PCI_NUMA_AFFINITY,COMPUTE_ACCELERATORS,HW_CPU_X86_SSE41,COMPUTE_NET_ATTACH_INTERFACE_WITH_TAG,COMPUTE_GRAPHICS_MODEL_VGA,COMPUTE_GRAPHICS_MODEL_QXL,COMPUTE_VIOMMU_MODEL_AUTO,COMPUTE_SECURITY_UEFI_SECURE_BOOT,COMPUTE_NET_VIF_MODEL_E1000,COMPUTE_TRUSTED_CERTS,COMPUTE_VOLUME_MULTI_ATTACH,COMPUTE_STORAGE_BUS_FDC,COMPUTE_VOLUME_EXTEND,COMPUTE_GRAPHICS_MODEL_CIRRUS,COMPUTE_GRAPHICS_MODEL_VMVGA,COMPUTE_GRAPHICS_MODEL_BOCHS,COMPUTE_DEVICE_TAGGING,COMPUTE_STORAGE_BUS_SCSI,COMPUTE_STORAGE_BUS_VIRTIO,HW_CPU_X86_SSSE3,HW_CPU_X86_SSE2,COMPUTE_IMAGE_TYPE_AMI,COMPUTE_STORAGE_BUS_USB,COMPUTE_NET_VIF_MODEL_PCNET,COMPUTE_NET_VIF_MODEL_VIRTIO,COMPUTE_IMAGE_TYPE_ISO,HW_CPU_X86_SSE42,HW_CPU_X86_SSE,COMPUTE_IMAGE_TYPE_ARI,COMPUTE_IMAGE_TYPE_RAW,COMPUTE_IMAGE_TYPE_QCOW2,COMPUTE_NET_VIF_MODEL_NE2K_PCI,COMPUTE_NODE,HW_CPU_X86_MMX,COMPUTE_NET_VIF_MODEL_RTL8139,COMPUTE_GRAPHICS_MODEL_VIRTIO,COMPUTE_NET_VIF_MODEL_SPAPR_VLAN,COMPUTE_STORAGE_BUS_SATA,COMPUTE_NET_ATTACH_INTERFACE,COMPUTE_RESCUE_BFV {{(pid=72351) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:825}}
Aug 09 03:04:41 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 03:04:41 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 03:04:41 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Compute_service record updated for user:user {{(pid=72351) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Aug 09 03:04:41 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.502s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 03:04:43 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:04:45 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4996-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:04:50 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:04:50 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:04:50 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5003 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 03:04:50 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:04:50 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:04:50 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:04:55 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4997-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:04:55 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:04:55 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5003 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 03:04:55 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:04:55 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:04:55 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:05:00 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4997-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:05:00 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:05:00 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5003 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 03:05:00 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:05:00 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:05:00 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:05:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4997-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:05:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:05:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5003 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 03:05:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:05:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:05:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:05:10 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4997-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:05:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-997a5514-c4e1-4ae0-bdf6-3b50432f4588 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Acquiring lock "56b80e5d-b7f4-4780-987e-5def6d299f3d" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 03:05:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-997a5514-c4e1-4ae0-bdf6-3b50432f4588 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "56b80e5d-b7f4-4780-987e-5def6d299f3d" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 03:05:11 user nova-compute[72351]: DEBUG nova.objects.instance [None req-997a5514-c4e1-4ae0-bdf6-3b50432f4588 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lazy-loading 'flavor' on Instance uuid 56b80e5d-b7f4-4780-987e-5def6d299f3d {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 03:05:11 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-997a5514-c4e1-4ae0-bdf6-3b50432f4588 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "56b80e5d-b7f4-4780-987e-5def6d299f3d" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.055s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 03:05:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-997a5514-c4e1-4ae0-bdf6-3b50432f4588 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Acquiring lock "56b80e5d-b7f4-4780-987e-5def6d299f3d" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 03:05:12 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-997a5514-c4e1-4ae0-bdf6-3b50432f4588 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "56b80e5d-b7f4-4780-987e-5def6d299f3d" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 03:05:12 user nova-compute[72351]: INFO nova.compute.manager [None req-997a5514-c4e1-4ae0-bdf6-3b50432f4588 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Attaching volume 5464ce1a-4377-4ef0-a2f1-bcd89b0dd2f6 to /dev/vdb
Aug 09 03:05:12 user nova-compute[72351]: DEBUG os_brick.utils [None req-997a5514-c4e1-4ae0-bdf6-3b50432f4588 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 03:05:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:05:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.016s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:05:12 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[0d6eba98-fb5c-4df9-b739-368fd7f410e7]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:a52269602ff4\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:05:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:05:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.016s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:05:12 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[49d2ac91-02da-449d-8f0b-45b049cd3d4b]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:05:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:05:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:05:12 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[bd75fff8-c40d-49f3-89a9-a3a5fabb1649]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:05:12 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[7ac9db4d-f07b-44cc-99ec-14828618c645]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:05:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-997a5514-c4e1-4ae0-bdf6-3b50432f4588 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Running cmd (subprocess): nvme version {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:05:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-997a5514-c4e1-4ae0-bdf6-3b50432f4588 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] 'nvme version' failed. Not Retrying. {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 03:05:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.nvmeof [None req-997a5514-c4e1-4ae0-bdf6-3b50432f4588 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] nvme not present on system {{(pid=72351) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Aug 09 03:05:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:05:12 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 03:05:12 user nova-compute[72351]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Aug 09 03:05:12 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[893a95da-40bb-40e6-80bd-d2a55aeb18bc]: (4, '') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:05:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-997a5514-c4e1-4ae0-bdf6-3b50432f4588 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72351) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Aug 09 03:05:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-997a5514-c4e1-4ae0-bdf6-3b50432f4588 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Aug 09 03:05:12 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-997a5514-c4e1-4ae0-bdf6-3b50432f4588 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] LIGHTOS: no hostnqn found. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Aug 09 03:05:12 user nova-compute[72351]: DEBUG os_brick.utils [None req-997a5514-c4e1-4ae0-bdf6-3b50432f4588 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] <== get_connector_properties: return (127ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:a52269602ff4', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 03:05:12 user nova-compute[72351]: DEBUG nova.virt.block_device [None req-997a5514-c4e1-4ae0-bdf6-3b50432f4588 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Updating existing volume attachment record: 3b1e25a8-c076-4c5f-a704-f509a8b682b1 {{(pid=72351) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Aug 09 03:05:14 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:05:15 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:05:19 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-997a5514-c4e1-4ae0-bdf6-3b50432f4588 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 03:05:19 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-997a5514-c4e1-4ae0-bdf6-3b50432f4588 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:5464ce1a-4377-4ef0-a2f1-bcd89b0dd2f6'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'WEtyoCSj', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 03:05:19 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-997a5514-c4e1-4ae0-bdf6-3b50432f4588 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 03:05:19 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-997a5514-c4e1-4ae0-bdf6-3b50432f4588 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 03:05:19 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-997a5514-c4e1-4ae0-bdf6-3b50432f4588 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 03:05:19 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-997a5514-c4e1-4ae0-bdf6-3b50432f4588 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 03:05:19 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-997a5514-c4e1-4ae0-bdf6-3b50432f4588 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:5464ce1a-4377-4ef0-a2f1-bcd89b0dd2f6" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 03:05:19 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-997a5514-c4e1-4ae0-bdf6-3b50432f4588 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:5464ce1a-4377-4ef0-a2f1-bcd89b0dd2f6" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 03:05:19 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-997a5514-c4e1-4ae0-bdf6-3b50432f4588 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 03:05:19 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5464ce1a-4377-4ef0-a2f1-bcd89b0dd2f6 -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:05:19 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5464ce1a-4377-4ef0-a2f1-bcd89b0dd2f6 -p 172.16.0.220:3260" returned: 21 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:05:19 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[02ae0578-60ea-4160-a605-5c549aa2d7cb]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:05:19 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-997a5514-c4e1-4ae0-bdf6-3b50432f4588 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Aug 09 03:05:19 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 03:05:19 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5464ce1a-4377-4ef0-a2f1-bcd89b0dd2f6 -p 172.16.0.220:3260 --interface default --op new {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:05:19 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5464ce1a-4377-4ef0-a2f1-bcd89b0dd2f6 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:05:19 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[aeab605f-e4d8-4918-a8fb-837a0ab0e19b]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:5464ce1a-4377-4ef0-a2f1-bcd89b0dd2f6] added\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:05:19 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-997a5514-c4e1-4ae0-bdf6-3b50432f4588 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:5464ce1a-4377-4ef0-a2f1-bcd89b0dd2f6] added
Aug 09 03:05:19 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 03:05:19 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5464ce1a-4377-4ef0-a2f1-bcd89b0dd2f6 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:05:19 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5464ce1a-4377-4ef0-a2f1-bcd89b0dd2f6 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.031s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:05:19 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[920eb75e-f5b4-42bb-a070-80af45a113a4]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:05:19 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-997a5514-c4e1-4ae0-bdf6-3b50432f4588 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 03:05:19 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5464ce1a-4377-4ef0-a2f1-bcd89b0dd2f6 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:05:19 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5464ce1a-4377-4ef0-a2f1-bcd89b0dd2f6 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:05:19 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[af52b3e9-eee4-44fe-9b30-a23331f0f07b]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:05:19 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-997a5514-c4e1-4ae0-bdf6-3b50432f4588 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 03:05:19 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5464ce1a-4377-4ef0-a2f1-bcd89b0dd2f6 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v WEtyoCSj {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:05:19 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5464ce1a-4377-4ef0-a2f1-bcd89b0dd2f6 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v WEtyoCSj" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:05:19 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[85447851-33ca-4259-9a29-f5112c75c040]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:05:19 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-997a5514-c4e1-4ae0-bdf6-3b50432f4588 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'WEtyoCSj'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 03:05:19 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5464ce1a-4377-4ef0-a2f1-bcd89b0dd2f6 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:05:19 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5464ce1a-4377-4ef0-a2f1-bcd89b0dd2f6 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.021s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:05:19 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[63d87119-e121-46ab-a5f7-c42eb3d062a1]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:05:19 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-997a5514-c4e1-4ae0-bdf6-3b50432f4588 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 03:05:19 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:05:19 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 21 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:05:19 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[b791fddc-7292-4b60-8ba2-9e938ee2e24e]: (4, ('', 'iscsiadm: No active sessions.\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:05:19 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-997a5514-c4e1-4ae0-bdf6-3b50432f4588 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsiadm ('-m', 'session'): stdout= stderr=iscsiadm: No active sessions.
Aug 09 03:05:19 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 03:05:19 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-997a5514-c4e1-4ae0-bdf6-3b50432f4588 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsi session list stdout= stderr=iscsiadm: No active sessions.
Aug 09 03:05:19 user nova-compute[72351]:  {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 03:05:19 user nova-compute[72351]: WARNING os_brick.initiator.connectors.iscsi [None req-997a5514-c4e1-4ae0-bdf6-3b50432f4588 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsiadm stderr output when getting sessions: iscsiadm: No active sessions.
Aug 09 03:05:19 user nova-compute[72351]: 
Aug 09 03:05:20 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5464ce1a-4377-4ef0-a2f1-bcd89b0dd2f6 -p 172.16.0.220:3260 --login {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:05:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:05:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:05:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 03:05:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:05:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:05:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:05:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:05:20 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5464ce1a-4377-4ef0-a2f1-bcd89b0dd2f6 -p 172.16.0.220:3260 --login" returned: 0 in 0.059s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:05:20 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[6c133005-2aed-4289-a7f2-e74b41e85647]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:5464ce1a-4377-4ef0-a2f1-bcd89b0dd2f6, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:5464ce1a-4377-4ef0-a2f1-bcd89b0dd2f6, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:05:20 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-997a5514-c4e1-4ae0-bdf6-3b50432f4588 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:5464ce1a-4377-4ef0-a2f1-bcd89b0dd2f6, portal: 172.16.0.220,3260]
Aug 09 03:05:20 user nova-compute[72351]: Login to [iface: default, target: iqn.2010-10.org.openstack:5464ce1a-4377-4ef0-a2f1-bcd89b0dd2f6, portal: 172.16.0.220,3260] successful.
Aug 09 03:05:20 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 03:05:20 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5464ce1a-4377-4ef0-a2f1-bcd89b0dd2f6 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:05:20 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5464ce1a-4377-4ef0-a2f1-bcd89b0dd2f6 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.023s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:05:20 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[91ee619a-76f4-47ee-91dd-9ea78bc8e2e1]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:05:20 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-997a5514-c4e1-4ae0-bdf6-3b50432f4588 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 03:05:20 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:05:20 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:05:20 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[a2ef5c81-a6e1-4711-92d9-13d74734ad7b]: (4, ('tcp: [69] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5464ce1a-4377-4ef0-a2f1-bcd89b0dd2f6 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:05:20 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-997a5514-c4e1-4ae0-bdf6-3b50432f4588 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [69] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5464ce1a-4377-4ef0-a2f1-bcd89b0dd2f6 (non-flash)
Aug 09 03:05:20 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 03:05:20 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-997a5514-c4e1-4ae0-bdf6-3b50432f4588 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsi session list stdout=tcp: [69] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5464ce1a-4377-4ef0-a2f1-bcd89b0dd2f6 (non-flash)
Aug 09 03:05:20 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 03:05:20 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-997a5514-c4e1-4ae0-bdf6-3b50432f4588 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:5464ce1a-4377-4ef0-a2f1-bcd89b0dd2f6" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.289s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 03:05:20 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-997a5514-c4e1-4ae0-bdf6-3b50432f4588 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 03:05:20 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-997a5514-c4e1-4ae0-bdf6-3b50432f4588 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] HCTL ('33', '-', '-', 0) found on session 69 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 03:05:20 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-997a5514-c4e1-4ae0-bdf6-3b50432f4588 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 03:05:20 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:05:20 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:05:20 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[049f0195-b4b2-4e2d-88b7-c9fe3f5a71b1]: (4, ('- - 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:05:20 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-997a5514-c4e1-4ae0-bdf6-3b50432f4588 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Searching for a device in session 69 and hctl ['33', '*', '*', 0] yield: None {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 03:05:21 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-997a5514-c4e1-4ae0-bdf6-3b50432f4588 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Searching for a device in session 69 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 03:05:21 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-997a5514-c4e1-4ae0-bdf6-3b50432f4588 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Connected to sdb using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'WEtyoCSj', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 03:05:21 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-997a5514-c4e1-4ae0-bdf6-3b50432f4588 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.322s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 03:05:21 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-997a5514-c4e1-4ae0-bdf6-3b50432f4588 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] <== connect_volume: return (1324ms) {'type': 'block', 'scsi_wwn': '26531333661363038', 'path': '/dev/sdb'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 03:05:21 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-997a5514-c4e1-4ae0-bdf6-3b50432f4588 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26531333661363038', 'path': '/dev/sdb'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 03:05:21 user nova-compute[72351]: DEBUG nova.objects.instance [None req-997a5514-c4e1-4ae0-bdf6-3b50432f4588 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lazy-loading 'flavor' on Instance uuid 56b80e5d-b7f4-4780-987e-5def6d299f3d {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 03:05:21 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-997a5514-c4e1-4ae0-bdf6-3b50432f4588 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] attach device xml: <disk type="block" device="disk">
Aug 09 03:05:21 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 03:05:21 user nova-compute[72351]:   <source dev="/dev/sdb"/>
Aug 09 03:05:21 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 03:05:21 user nova-compute[72351]:   <serial>5464ce1a-4377-4ef0-a2f1-bcd89b0dd2f6</serial>
Aug 09 03:05:21 user nova-compute[72351]: </disk>
Aug 09 03:05:21 user nova-compute[72351]:  {{(pid=72351) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Aug 09 03:05:21 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-997a5514-c4e1-4ae0-bdf6-3b50432f4588 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 03:05:21 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-997a5514-c4e1-4ae0-bdf6-3b50432f4588 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 03:05:21 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-997a5514-c4e1-4ae0-bdf6-3b50432f4588 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] No VIF found with MAC fa:16:3e:ea:18:dd, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 03:05:21 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-997a5514-c4e1-4ae0-bdf6-3b50432f4588 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "56b80e5d-b7f4-4780-987e-5def6d299f3d" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 9.629s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 03:05:23 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-3eb17030-3070-4fde-8d04-019433cbf1db tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Acquiring lock "56b80e5d-b7f4-4780-987e-5def6d299f3d" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 03:05:23 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-3eb17030-3070-4fde-8d04-019433cbf1db tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "56b80e5d-b7f4-4780-987e-5def6d299f3d" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 03:05:23 user nova-compute[72351]: INFO nova.compute.manager [None req-3eb17030-3070-4fde-8d04-019433cbf1db tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Detaching volume 5464ce1a-4377-4ef0-a2f1-bcd89b0dd2f6
Aug 09 03:05:23 user nova-compute[72351]: INFO nova.virt.block_device [None req-3eb17030-3070-4fde-8d04-019433cbf1db tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Attempting to driver detach volume 5464ce1a-4377-4ef0-a2f1-bcd89b0dd2f6 from mountpoint /dev/vdb
Aug 09 03:05:23 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-3eb17030-3070-4fde-8d04-019433cbf1db tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Attempting to detach device vdb from instance 56b80e5d-b7f4-4780-987e-5def6d299f3d from the persistent domain config. {{(pid=72351) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2477}}
Aug 09 03:05:23 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-3eb17030-3070-4fde-8d04-019433cbf1db tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 03:05:23 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 03:05:23 user nova-compute[72351]:   <source dev="/dev/sdb"/>
Aug 09 03:05:23 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 03:05:23 user nova-compute[72351]:   <serial>5464ce1a-4377-4ef0-a2f1-bcd89b0dd2f6</serial>
Aug 09 03:05:23 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Aug 09 03:05:23 user nova-compute[72351]: </disk>
Aug 09 03:05:23 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 03:05:23 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-3eb17030-3070-4fde-8d04-019433cbf1db tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Successfully detached device vdb from instance 56b80e5d-b7f4-4780-987e-5def6d299f3d from the persistent domain config.
Aug 09 03:05:23 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-3eb17030-3070-4fde-8d04-019433cbf1db tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 56b80e5d-b7f4-4780-987e-5def6d299f3d from the live domain config. {{(pid=72351) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2513}}
Aug 09 03:05:23 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-3eb17030-3070-4fde-8d04-019433cbf1db tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 03:05:23 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 03:05:23 user nova-compute[72351]:   <source dev="/dev/sdb"/>
Aug 09 03:05:23 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 03:05:23 user nova-compute[72351]:   <serial>5464ce1a-4377-4ef0-a2f1-bcd89b0dd2f6</serial>
Aug 09 03:05:23 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Aug 09 03:05:23 user nova-compute[72351]: </disk>
Aug 09 03:05:23 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 03:05:23 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Received event <DeviceRemovedEvent: 1691550323.9238207, 56b80e5d-b7f4-4780-987e-5def6d299f3d => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=72351) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2360}}
Aug 09 03:05:23 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-3eb17030-3070-4fde-8d04-019433cbf1db tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 56b80e5d-b7f4-4780-987e-5def6d299f3d {{(pid=72351) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2589}}
Aug 09 03:05:23 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-3eb17030-3070-4fde-8d04-019433cbf1db tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Successfully detached device vdb from instance 56b80e5d-b7f4-4780-987e-5def6d299f3d from the live domain config.
Aug 09 03:05:23 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-3eb17030-3070-4fde-8d04-019433cbf1db tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 03:05:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-3eb17030-3070-4fde-8d04-019433cbf1db tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:5464ce1a-4377-4ef0-a2f1-bcd89b0dd2f6'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'WEtyoCSj', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb'}, None), 'kwargs': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 03:05:23 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-3eb17030-3070-4fde-8d04-019433cbf1db tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 03:05:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-3eb17030-3070-4fde-8d04-019433cbf1db tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 03:05:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-3eb17030-3070-4fde-8d04-019433cbf1db tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 03:05:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-3eb17030-3070-4fde-8d04-019433cbf1db tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:5464ce1a-4377-4ef0-a2f1-bcd89b0dd2f6', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 03:05:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:05:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:05:23 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[2ff57047-9d17-4524-a2e2-b6ecebd0cb38]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:5464ce1a-4377-4ef0-a2f1-bcd89b0dd2f6\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:05:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:05:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:05:23 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e4571120-934c-4280-8fd7-14edc12b173f]: (4, ('tcp: [69] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5464ce1a-4377-4ef0-a2f1-bcd89b0dd2f6 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:05:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-3eb17030-3070-4fde-8d04-019433cbf1db tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [69] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5464ce1a-4377-4ef0-a2f1-bcd89b0dd2f6 (non-flash)
Aug 09 03:05:23 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 03:05:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-3eb17030-3070-4fde-8d04-019433cbf1db tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsi session list stdout=tcp: [69] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5464ce1a-4377-4ef0-a2f1-bcd89b0dd2f6 (non-flash)
Aug 09 03:05:23 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 03:05:23 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-3eb17030-3070-4fde-8d04-019433cbf1db tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fcfdc18bbe0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:5464ce1a-4377-4ef0-a2f1-bcd89b0dd2f6'): ({'sdb'}, set())}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 03:05:23 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-3eb17030-3070-4fde-8d04-019433cbf1db tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Removing single pathed devices sdb {{(pid=72351) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Aug 09 03:05:23 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:05:24 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.024s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:05:24 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[922ee353-4881-4fa6-b463-4ac30452f931]: (4, ('path checker states:\nup                  1\n\npaths: 0\nbusy: False\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:05:24 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:05:24 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:05:24 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[9e4420ff-c348-4c91-b322-cafddae9de60]: (4, ('ok\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:05:24 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-3eb17030-3070-4fde-8d04-019433cbf1db tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Flushing IO for device /dev/sdb {{(pid=72351) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Aug 09 03:05:24 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:05:24 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:05:24 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[7c564df9-a418-45e7-ba82-c5cc99d1cbc8]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:05:24 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-3eb17030-3070-4fde-8d04-019433cbf1db tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=72351) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Aug 09 03:05:24 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:05:24 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.050s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:05:24 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[df5cdae2-68dc-4d6e-ba0f-e4a0e33d0932]: (4, ('1', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:05:24 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-3eb17030-3070-4fde-8d04-019433cbf1db tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Aug 09 03:05:24 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-3eb17030-3070-4fde-8d04-019433cbf1db tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] SCSI volumes sdb have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Aug 09 03:05:24 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-3eb17030-3070-4fde-8d04-019433cbf1db tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:5464ce1a-4377-4ef0-a2f1-bcd89b0dd2f6')] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 03:05:24 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5464ce1a-4377-4ef0-a2f1-bcd89b0dd2f6 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:05:24 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5464ce1a-4377-4ef0-a2f1-bcd89b0dd2f6 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:05:24 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[7d044400-7486-4ca4-8ed8-e17d432e0126]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:05:24 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-3eb17030-3070-4fde-8d04-019433cbf1db tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 03:05:24 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5464ce1a-4377-4ef0-a2f1-bcd89b0dd2f6 -p 172.16.0.220:3260 --logout {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:05:24 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5464ce1a-4377-4ef0-a2f1-bcd89b0dd2f6 -p 172.16.0.220:3260 --logout" returned: 0 in 0.069s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:05:24 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[9f17592a-338f-4e41-a6cc-ee46ea4de499]: (4, ('Logging out of session [sid: 69, target: iqn.2010-10.org.openstack:5464ce1a-4377-4ef0-a2f1-bcd89b0dd2f6, portal: 172.16.0.220,3260]\nLogout of [sid: 69, target: iqn.2010-10.org.openstack:5464ce1a-4377-4ef0-a2f1-bcd89b0dd2f6, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:05:24 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-3eb17030-3070-4fde-8d04-019433cbf1db tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 69, target: iqn.2010-10.org.openstack:5464ce1a-4377-4ef0-a2f1-bcd89b0dd2f6, portal: 172.16.0.220,3260]
Aug 09 03:05:24 user nova-compute[72351]: Logout of [sid: 69, target: iqn.2010-10.org.openstack:5464ce1a-4377-4ef0-a2f1-bcd89b0dd2f6, portal: 172.16.0.220,3260] successful.
Aug 09 03:05:24 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 03:05:24 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5464ce1a-4377-4ef0-a2f1-bcd89b0dd2f6 -p 172.16.0.220:3260 --op delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:05:24 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5464ce1a-4377-4ef0-a2f1-bcd89b0dd2f6 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:05:24 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d6013f53-b94d-4685-8d62-b84ea86a993c]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:05:24 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-3eb17030-3070-4fde-8d04-019433cbf1db tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 03:05:24 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-3eb17030-3070-4fde-8d04-019433cbf1db tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.304s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 03:05:24 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-3eb17030-3070-4fde-8d04-019433cbf1db tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] <== disconnect_volume: return (306ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 03:05:24 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-3eb17030-3070-4fde-8d04-019433cbf1db tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 03:05:24 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:05:24 user nova-compute[72351]: DEBUG nova.objects.instance [None req-3eb17030-3070-4fde-8d04-019433cbf1db tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lazy-loading 'flavor' on Instance uuid 56b80e5d-b7f4-4780-987e-5def6d299f3d {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 03:05:24 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-3eb17030-3070-4fde-8d04-019433cbf1db tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "56b80e5d-b7f4-4780-987e-5def6d299f3d" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.635s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 03:05:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:05:29 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:05:30 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:05:30 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:05:30 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5003 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 03:05:30 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:05:30 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:05:30 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:05:32 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:05:32 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72351) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10461}}
Aug 09 03:05:33 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:05:33 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Starting heal instance info cache {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9842}}
Aug 09 03:05:33 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Rebuilding the list of instances to heal {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9846}}
Aug 09 03:05:33 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "refresh_cache-56b80e5d-b7f4-4780-987e-5def6d299f3d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 03:05:33 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquired lock "refresh_cache-56b80e5d-b7f4-4780-987e-5def6d299f3d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 03:05:33 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Forcefully refreshing network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1996}}
Aug 09 03:05:33 user nova-compute[72351]: DEBUG nova.objects.instance [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lazy-loading 'info_cache' on Instance uuid 56b80e5d-b7f4-4780-987e-5def6d299f3d {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 03:05:34 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Updating instance_info_cache with network_info: [{"id": "c69c5267-24fa-40f1-8f76-b4f2ec1ced93", "address": "fa:16:3e:ea:18:dd", "network": {"id": "45ee1620-bd54-4ca8-ab3a-9222782aea6f", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-173232413-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.86", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a636abaff2944499911a14ffb2846b48", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc69c5267-24", "ovs_interfaceid": "c69c5267-24fa-40f1-8f76-b4f2ec1ced93", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 03:05:34 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Releasing lock "refresh_cache-56b80e5d-b7f4-4780-987e-5def6d299f3d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 03:05:34 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Updated the network info_cache for instance {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9913}}
Aug 09 03:05:35 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4997-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:05:35 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:05:35 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 03:05:35 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:05:35 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:05:35 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:05:35 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:05:36 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:05:37 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:05:38 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:05:39 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:05:39 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 03:05:39 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 03:05:39 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 03:05:39 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Auditing locally available compute resources for user (node: user) {{(pid=72351) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Aug 09 03:05:40 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56b80e5d-b7f4-4780-987e-5def6d299f3d/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:05:40 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4997-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:05:40 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56b80e5d-b7f4-4780-987e-5def6d299f3d/disk --force-share --output=json" returned: 0 in 0.140s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:05:40 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56b80e5d-b7f4-4780-987e-5def6d299f3d/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:05:40 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56b80e5d-b7f4-4780-987e-5def6d299f3d/disk --force-share --output=json" returned: 0 in 0.158s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:05:40 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 03:05:40 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 03:05:40 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Hypervisor/Node resource view: name=user free_ram=8778MB free_disk=25.155689239501953GB free_vcpus=11 pci_devices=[{"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72351) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Aug 09 03:05:40 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 03:05:40 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 03:05:40 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 56b80e5d-b7f4-4780-987e-5def6d299f3d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 03:05:40 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Total usable vcpus: 12, total allocated vcpus: 1 {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Aug 09 03:05:40 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Final resource view: name=user phys_ram=16011MB used_ram=1024MB phys_disk=40GB used_disk=1GB total_vcpus=12 used_vcpus=1 pci_stats=[] {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Aug 09 03:05:40 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 03:05:40 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 03:05:40 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Compute_service record updated for user:user {{(pid=72351) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Aug 09 03:05:40 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.167s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 03:05:43 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:05:45 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:05:50 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:05:50 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:05:50 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 03:05:50 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:05:50 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:05:50 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:05:54 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:05:55 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:06:00 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:06:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:06:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:06:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5003 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 03:06:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:06:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:06:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:06:10 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4997-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:06:10 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:06:10 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5003 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 03:06:10 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:06:10 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:06:10 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:06:15 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:06:15 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:06:15 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 03:06:15 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:06:15 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:06:15 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:06:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-27083fa4-b310-4cba-bd99-a559ca03fc64 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Acquiring lock "56b80e5d-b7f4-4780-987e-5def6d299f3d" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 03:06:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-27083fa4-b310-4cba-bd99-a559ca03fc64 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "56b80e5d-b7f4-4780-987e-5def6d299f3d" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 03:06:17 user nova-compute[72351]: DEBUG nova.objects.instance [None req-27083fa4-b310-4cba-bd99-a559ca03fc64 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lazy-loading 'flavor' on Instance uuid 56b80e5d-b7f4-4780-987e-5def6d299f3d {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 03:06:17 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-27083fa4-b310-4cba-bd99-a559ca03fc64 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "56b80e5d-b7f4-4780-987e-5def6d299f3d" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.051s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 03:06:18 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-27083fa4-b310-4cba-bd99-a559ca03fc64 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Acquiring lock "56b80e5d-b7f4-4780-987e-5def6d299f3d" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 03:06:18 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-27083fa4-b310-4cba-bd99-a559ca03fc64 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "56b80e5d-b7f4-4780-987e-5def6d299f3d" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 03:06:18 user nova-compute[72351]: INFO nova.compute.manager [None req-27083fa4-b310-4cba-bd99-a559ca03fc64 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Attaching volume d14e301f-3f95-460b-b2d5-c863284d0e97 to /dev/vdb
Aug 09 03:06:18 user nova-compute[72351]: DEBUG os_brick.utils [None req-27083fa4-b310-4cba-bd99-a559ca03fc64 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 03:06:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:06:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:06:18 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[8d14dadc-0003-4564-af4d-8a988606baae]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:a52269602ff4\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:06:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:06:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.016s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:06:18 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[f9351d15-1a56-47cf-b8f3-c640df42411f]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:06:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:06:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.015s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:06:18 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[089e25a0-d23b-471c-a949-f88b56e52a12]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:06:18 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[fcf4b8bc-c6a7-45ed-97f7-e3dad4f160e5]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:06:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-27083fa4-b310-4cba-bd99-a559ca03fc64 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Running cmd (subprocess): nvme version {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:06:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-27083fa4-b310-4cba-bd99-a559ca03fc64 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] 'nvme version' failed. Not Retrying. {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 03:06:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.nvmeof [None req-27083fa4-b310-4cba-bd99-a559ca03fc64 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] nvme not present on system {{(pid=72351) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Aug 09 03:06:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:06:18 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Aug 09 03:06:18 user nova-compute[72351]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Aug 09 03:06:18 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[e8c2b407-63b7-429c-b34c-c112a8bf8809]: (4, '') {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:06:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-27083fa4-b310-4cba-bd99-a559ca03fc64 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=72351) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Aug 09 03:06:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-27083fa4-b310-4cba-bd99-a559ca03fc64 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Aug 09 03:06:18 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.lightos [None req-27083fa4-b310-4cba-bd99-a559ca03fc64 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] LIGHTOS: no hostnqn found. {{(pid=72351) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Aug 09 03:06:18 user nova-compute[72351]: DEBUG os_brick.utils [None req-27083fa4-b310-4cba-bd99-a559ca03fc64 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] <== get_connector_properties: return (127ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:a52269602ff4', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 03:06:18 user nova-compute[72351]: DEBUG nova.virt.block_device [None req-27083fa4-b310-4cba-bd99-a559ca03fc64 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Updating existing volume attachment record: 7eb67dfc-acf8-4bb6-a7d9-c2ada0148115 {{(pid=72351) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Aug 09 03:06:19 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:06:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:06:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:06:29 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:06:30 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:06:30 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:06:30 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5001 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 03:06:30 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:06:30 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:06:30 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:06:31 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-27083fa4-b310-4cba-bd99-a559ca03fc64 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Calling os-brick to attach iSCSI Volume {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Aug 09 03:06:31 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-27083fa4-b310-4cba-bd99-a559ca03fc64 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, 'connection_properties': {'target_iqns': ['iqn.2010-10.org.openstack:d14e301f-3f95-460b-b2d5-c863284d0e97'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'KQTiWVhv', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 03:06:31 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-27083fa4-b310-4cba-bd99-a559ca03fc64 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 03:06:31 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-27083fa4-b310-4cba-bd99-a559ca03fc64 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 03:06:31 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-27083fa4-b310-4cba-bd99-a559ca03fc64 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 03:06:31 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-27083fa4-b310-4cba-bd99-a559ca03fc64 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 03:06:31 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-27083fa4-b310-4cba-bd99-a559ca03fc64 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:d14e301f-3f95-460b-b2d5-c863284d0e97" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 03:06:31 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-27083fa4-b310-4cba-bd99-a559ca03fc64 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:d14e301f-3f95-460b-b2d5-c863284d0e97" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 03:06:31 user nova-compute[72351]: INFO os_brick.initiator.connectors.iscsi [None req-27083fa4-b310-4cba-bd99-a559ca03fc64 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Aug 09 03:06:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d14e301f-3f95-460b-b2d5-c863284d0e97 -p 172.16.0.220:3260 {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:06:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d14e301f-3f95-460b-b2d5-c863284d0e97 -p 172.16.0.220:3260" returned: 21 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:06:31 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[22392abf-93ee-4409-937c-eb496b02dafe]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:06:31 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-27083fa4-b310-4cba-bd99-a559ca03fc64 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Aug 09 03:06:31 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 03:06:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d14e301f-3f95-460b-b2d5-c863284d0e97 -p 172.16.0.220:3260 --interface default --op new {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:06:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d14e301f-3f95-460b-b2d5-c863284d0e97 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.022s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:06:31 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[5d2d3a31-ab4e-4c11-977b-fbfcebe8aeb4]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:d14e301f-3f95-460b-b2d5-c863284d0e97] added\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:06:31 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-27083fa4-b310-4cba-bd99-a559ca03fc64 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:d14e301f-3f95-460b-b2d5-c863284d0e97] added
Aug 09 03:06:31 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 03:06:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d14e301f-3f95-460b-b2d5-c863284d0e97 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:06:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d14e301f-3f95-460b-b2d5-c863284d0e97 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:06:31 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[cb3cf40b-1613-4123-8260-95926b50b351]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:06:31 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-27083fa4-b310-4cba-bd99-a559ca03fc64 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 03:06:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d14e301f-3f95-460b-b2d5-c863284d0e97 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:06:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d14e301f-3f95-460b-b2d5-c863284d0e97 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:06:31 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[93190c86-39f1-4fe7-999b-1241806de2c5]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:06:31 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-27083fa4-b310-4cba-bd99-a559ca03fc64 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 03:06:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d14e301f-3f95-460b-b2d5-c863284d0e97 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v KQTiWVhv {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:06:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d14e301f-3f95-460b-b2d5-c863284d0e97 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v KQTiWVhv" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:06:31 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[67367884-cade-47cc-aeeb-a56771e924eb]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:06:31 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-27083fa4-b310-4cba-bd99-a559ca03fc64 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'KQTiWVhv'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 03:06:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d14e301f-3f95-460b-b2d5-c863284d0e97 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:06:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d14e301f-3f95-460b-b2d5-c863284d0e97 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.065s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:06:31 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[6624b74c-b714-4496-b63e-83eab3cd6980]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:06:31 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-27083fa4-b310-4cba-bd99-a559ca03fc64 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 03:06:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:06:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 21 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:06:31 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[1f24738b-6933-4126-90ff-3ba3f4b97f1e]: (4, ('', 'iscsiadm: No active sessions.\n')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:06:31 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-27083fa4-b310-4cba-bd99-a559ca03fc64 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsiadm ('-m', 'session'): stdout= stderr=iscsiadm: No active sessions.
Aug 09 03:06:31 user nova-compute[72351]:  {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 03:06:31 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-27083fa4-b310-4cba-bd99-a559ca03fc64 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsi session list stdout= stderr=iscsiadm: No active sessions.
Aug 09 03:06:31 user nova-compute[72351]:  {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 03:06:31 user nova-compute[72351]: WARNING os_brick.initiator.connectors.iscsi [None req-27083fa4-b310-4cba-bd99-a559ca03fc64 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsiadm stderr output when getting sessions: iscsiadm: No active sessions.
Aug 09 03:06:31 user nova-compute[72351]: 
Aug 09 03:06:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d14e301f-3f95-460b-b2d5-c863284d0e97 -p 172.16.0.220:3260 --login {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:06:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d14e301f-3f95-460b-b2d5-c863284d0e97 -p 172.16.0.220:3260 --login" returned: 0 in 0.053s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:06:31 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[fd25dd97-b141-40f1-96e8-b8270b4900b2]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:d14e301f-3f95-460b-b2d5-c863284d0e97, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:d14e301f-3f95-460b-b2d5-c863284d0e97, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:06:31 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-27083fa4-b310-4cba-bd99-a559ca03fc64 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:d14e301f-3f95-460b-b2d5-c863284d0e97, portal: 172.16.0.220,3260]
Aug 09 03:06:31 user nova-compute[72351]: Login to [iface: default, target: iqn.2010-10.org.openstack:d14e301f-3f95-460b-b2d5-c863284d0e97, portal: 172.16.0.220,3260] successful.
Aug 09 03:06:31 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 03:06:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d14e301f-3f95-460b-b2d5-c863284d0e97 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:06:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d14e301f-3f95-460b-b2d5-c863284d0e97 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:06:31 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[d935d1a6-0819-44c2-a324-ba2c341f5caf]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:06:31 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-27083fa4-b310-4cba-bd99-a559ca03fc64 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 03:06:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:06:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:06:31 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[6e5ca8e2-3066-49fd-bce4-3cbb511cb0d2]: (4, ('tcp: [70] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d14e301f-3f95-460b-b2d5-c863284d0e97 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:06:31 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-27083fa4-b310-4cba-bd99-a559ca03fc64 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [70] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d14e301f-3f95-460b-b2d5-c863284d0e97 (non-flash)
Aug 09 03:06:31 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 03:06:31 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-27083fa4-b310-4cba-bd99-a559ca03fc64 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsi session list stdout=tcp: [70] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d14e301f-3f95-460b-b2d5-c863284d0e97 (non-flash)
Aug 09 03:06:31 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 03:06:31 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-27083fa4-b310-4cba-bd99-a559ca03fc64 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:d14e301f-3f95-460b-b2d5-c863284d0e97" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.314s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 03:06:31 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-27083fa4-b310-4cba-bd99-a559ca03fc64 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Connected to 172.16.0.220:3260 {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Aug 09 03:06:31 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-27083fa4-b310-4cba-bd99-a559ca03fc64 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] HCTL ('33', '-', '-', 0) found on session 70 with lun 0 {{(pid=72351) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Aug 09 03:06:31 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-27083fa4-b310-4cba-bd99-a559ca03fc64 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=72351) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Aug 09 03:06:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:06:31 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:06:31 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[7ed6bd31-e0d3-41fb-b4df-bdb4661ff5c4]: (4, ('- - 0', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:06:31 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-27083fa4-b310-4cba-bd99-a559ca03fc64 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Searching for a device in session 70 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=72351) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Aug 09 03:06:31 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-27083fa4-b310-4cba-bd99-a559ca03fc64 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Connected to sdb using {'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'KQTiWVhv', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'target_portal': '172.16.0.220:3260', 'target_iqn'} {{(pid=72351) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Aug 09 03:06:32 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:06:32 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72351) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10461}}
Aug 09 03:06:32 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-27083fa4-b310-4cba-bd99-a559ca03fc64 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.346s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 03:06:32 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-27083fa4-b310-4cba-bd99-a559ca03fc64 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] <== connect_volume: return (1347ms) {'type': 'block', 'scsi_wwn': '23664316633373331', 'path': '/dev/sdb'} {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 03:06:32 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-27083fa4-b310-4cba-bd99-a559ca03fc64 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23664316633373331', 'path': '/dev/sdb'} {{(pid=72351) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Aug 09 03:06:32 user nova-compute[72351]: DEBUG nova.objects.instance [None req-27083fa4-b310-4cba-bd99-a559ca03fc64 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lazy-loading 'flavor' on Instance uuid 56b80e5d-b7f4-4780-987e-5def6d299f3d {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 03:06:33 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-27083fa4-b310-4cba-bd99-a559ca03fc64 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] attach device xml: <disk type="block" device="disk">
Aug 09 03:06:33 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 03:06:33 user nova-compute[72351]:   <source dev="/dev/sdb"/>
Aug 09 03:06:33 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 03:06:33 user nova-compute[72351]:   <serial>d14e301f-3f95-460b-b2d5-c863284d0e97</serial>
Aug 09 03:06:33 user nova-compute[72351]: </disk>
Aug 09 03:06:33 user nova-compute[72351]:  {{(pid=72351) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Aug 09 03:06:33 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-27083fa4-b310-4cba-bd99-a559ca03fc64 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] No BDM found with device name vda, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 03:06:33 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-27083fa4-b310-4cba-bd99-a559ca03fc64 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] No BDM found with device name vdb, not building metadata. {{(pid=72351) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12092}}
Aug 09 03:06:33 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-27083fa4-b310-4cba-bd99-a559ca03fc64 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] No VIF found with MAC fa:16:3e:ea:18:dd, not building metadata {{(pid=72351) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12068}}
Aug 09 03:06:33 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-27083fa4-b310-4cba-bd99-a559ca03fc64 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "56b80e5d-b7f4-4780-987e-5def6d299f3d" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 15.513s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 03:06:34 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:06:35 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:06:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1270ea66-c122-4e5c-a80d-dfd44e326112 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Acquiring lock "56b80e5d-b7f4-4780-987e-5def6d299f3d" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 03:06:35 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1270ea66-c122-4e5c-a80d-dfd44e326112 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "56b80e5d-b7f4-4780-987e-5def6d299f3d" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 03:06:35 user nova-compute[72351]: INFO nova.compute.manager [None req-1270ea66-c122-4e5c-a80d-dfd44e326112 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Detaching volume d14e301f-3f95-460b-b2d5-c863284d0e97
Aug 09 03:06:35 user nova-compute[72351]: INFO nova.virt.block_device [None req-1270ea66-c122-4e5c-a80d-dfd44e326112 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Attempting to driver detach volume d14e301f-3f95-460b-b2d5-c863284d0e97 from mountpoint /dev/vdb
Aug 09 03:06:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-1270ea66-c122-4e5c-a80d-dfd44e326112 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Attempting to detach device vdb from instance 56b80e5d-b7f4-4780-987e-5def6d299f3d from the persistent domain config. {{(pid=72351) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2477}}
Aug 09 03:06:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-1270ea66-c122-4e5c-a80d-dfd44e326112 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 03:06:35 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 03:06:35 user nova-compute[72351]:   <source dev="/dev/sdb"/>
Aug 09 03:06:35 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 03:06:35 user nova-compute[72351]:   <serial>d14e301f-3f95-460b-b2d5-c863284d0e97</serial>
Aug 09 03:06:35 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Aug 09 03:06:35 user nova-compute[72351]: </disk>
Aug 09 03:06:35 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 03:06:35 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-1270ea66-c122-4e5c-a80d-dfd44e326112 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Successfully detached device vdb from instance 56b80e5d-b7f4-4780-987e-5def6d299f3d from the persistent domain config.
Aug 09 03:06:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-1270ea66-c122-4e5c-a80d-dfd44e326112 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 56b80e5d-b7f4-4780-987e-5def6d299f3d from the live domain config. {{(pid=72351) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2513}}
Aug 09 03:06:35 user nova-compute[72351]: DEBUG nova.virt.libvirt.guest [None req-1270ea66-c122-4e5c-a80d-dfd44e326112 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] detach device xml: <disk type="block" device="disk">
Aug 09 03:06:35 user nova-compute[72351]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Aug 09 03:06:35 user nova-compute[72351]:   <source dev="/dev/sdb"/>
Aug 09 03:06:35 user nova-compute[72351]:   <target dev="vdb" bus="virtio"/>
Aug 09 03:06:35 user nova-compute[72351]:   <serial>d14e301f-3f95-460b-b2d5-c863284d0e97</serial>
Aug 09 03:06:35 user nova-compute[72351]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Aug 09 03:06:35 user nova-compute[72351]: </disk>
Aug 09 03:06:35 user nova-compute[72351]:  {{(pid=72351) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Aug 09 03:06:35 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:06:35 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Starting heal instance info cache {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9842}}
Aug 09 03:06:35 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Rebuilding the list of instances to heal {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9846}}
Aug 09 03:06:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "refresh_cache-56b80e5d-b7f4-4780-987e-5def6d299f3d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 03:06:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquired lock "refresh_cache-56b80e5d-b7f4-4780-987e-5def6d299f3d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 03:06:36 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Forcefully refreshing network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1996}}
Aug 09 03:06:36 user nova-compute[72351]: DEBUG nova.objects.instance [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lazy-loading 'info_cache' on Instance uuid 56b80e5d-b7f4-4780-987e-5def6d299f3d {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 03:06:36 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-06002103-6195-45a7-8204-d1cd0f40f96c None None] Received event <DeviceRemovedEvent: 1691550396.2426448, 56b80e5d-b7f4-4780-987e-5def6d299f3d => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=72351) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2360}}
Aug 09 03:06:36 user nova-compute[72351]: DEBUG nova.virt.libvirt.driver [None req-1270ea66-c122-4e5c-a80d-dfd44e326112 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 56b80e5d-b7f4-4780-987e-5def6d299f3d {{(pid=72351) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2589}}
Aug 09 03:06:36 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-1270ea66-c122-4e5c-a80d-dfd44e326112 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Successfully detached device vdb from instance 56b80e5d-b7f4-4780-987e-5def6d299f3d from the live domain config.
Aug 09 03:06:36 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-1270ea66-c122-4e5c-a80d-dfd44e326112 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] calling os-brick to detach iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Aug 09 03:06:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1270ea66-c122-4e5c-a80d-dfd44e326112 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7fcfdc56d960>, {'target_iqns': ['iqn.2010-10.org.openstack:d14e301f-3f95-460b-b2d5-c863284d0e97'], 'target_portals': ['172.16.0.220:3260'], 'target_luns': [0], 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'KQTiWVhv', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb'}, None), 'kwargs': False}}" {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Aug 09 03:06:36 user nova-compute[72351]: WARNING os_brick.initiator.connectors.base [None req-1270ea66-c122-4e5c-a80d-dfd44e326112 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Aug 09 03:06:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-1270ea66-c122-4e5c-a80d-dfd44e326112 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Aug 09 03:06:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-1270ea66-c122-4e5c-a80d-dfd44e326112 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Aug 09 03:06:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1270ea66-c122-4e5c-a80d-dfd44e326112 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:d14e301f-3f95-460b-b2d5-c863284d0e97', 0)] {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Aug 09 03:06:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:06:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.017s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:06:36 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[706aa450-d308-4f2b-8e01-e354ac3dcabf]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:d14e301f-3f95-460b-b2d5-c863284d0e97\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:06:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:06:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.020s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:06:36 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[489d1638-5ead-4027-9156-d237c32c9cc4]: (4, ('tcp: [70] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d14e301f-3f95-460b-b2d5-c863284d0e97 (non-flash)\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:06:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1270ea66-c122-4e5c-a80d-dfd44e326112 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [70] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d14e301f-3f95-460b-b2d5-c863284d0e97 (non-flash)
Aug 09 03:06:36 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Aug 09 03:06:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1270ea66-c122-4e5c-a80d-dfd44e326112 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsi session list stdout=tcp: [70] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d14e301f-3f95-460b-b2d5-c863284d0e97 (non-flash)
Aug 09 03:06:36 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Aug 09 03:06:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1270ea66-c122-4e5c-a80d-dfd44e326112 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7fcfdc189360>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:d14e301f-3f95-460b-b2d5-c863284d0e97'): ({'sdb'}, set())}) {{(pid=72351) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Aug 09 03:06:36 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-1270ea66-c122-4e5c-a80d-dfd44e326112 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Removing single pathed devices sdb {{(pid=72351) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Aug 09 03:06:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:06:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.018s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:06:36 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[b2b8abf8-5ded-4dfc-bc6e-c48ab361ac8e]: (4, ('path checker states:\nup                  1\n\npaths: 0\nbusy: False\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:06:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:06:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:06:36 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[88020f08-de0a-4f71-9e10-1870f3d74a65]: (4, ('ok\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:06:36 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-1270ea66-c122-4e5c-a80d-dfd44e326112 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Flushing IO for device /dev/sdb {{(pid=72351) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Aug 09 03:06:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:06:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:06:36 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[b081afff-6d7a-499a-8f8e-8bf9fed7cec5]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:06:36 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-1270ea66-c122-4e5c-a80d-dfd44e326112 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=72351) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Aug 09 03:06:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:06:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.035s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:06:36 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[6c49ebd7-c181-429a-916b-8e7b5fa94799]: (4, ('1', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:06:36 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-1270ea66-c122-4e5c-a80d-dfd44e326112 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Aug 09 03:06:36 user nova-compute[72351]: DEBUG os_brick.initiator.linuxscsi [None req-1270ea66-c122-4e5c-a80d-dfd44e326112 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] SCSI volumes sdb have been removed. {{(pid=72351) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Aug 09 03:06:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1270ea66-c122-4e5c-a80d-dfd44e326112 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:d14e301f-3f95-460b-b2d5-c863284d0e97')] {{(pid=72351) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Aug 09 03:06:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d14e301f-3f95-460b-b2d5-c863284d0e97 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:06:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d14e301f-3f95-460b-b2d5-c863284d0e97 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:06:36 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[b523f12a-6ee5-47eb-a35c-cd4354bc3619]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:06:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1270ea66-c122-4e5c-a80d-dfd44e326112 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 03:06:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d14e301f-3f95-460b-b2d5-c863284d0e97 -p 172.16.0.220:3260 --logout {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:06:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d14e301f-3f95-460b-b2d5-c863284d0e97 -p 172.16.0.220:3260 --logout" returned: 0 in 0.044s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:06:36 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[12e848ac-7512-42e4-a842-984e9e0dd84c]: (4, ('Logging out of session [sid: 70, target: iqn.2010-10.org.openstack:d14e301f-3f95-460b-b2d5-c863284d0e97, portal: 172.16.0.220,3260]\nLogout of [sid: 70, target: iqn.2010-10.org.openstack:d14e301f-3f95-460b-b2d5-c863284d0e97, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:06:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1270ea66-c122-4e5c-a80d-dfd44e326112 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 70, target: iqn.2010-10.org.openstack:d14e301f-3f95-460b-b2d5-c863284d0e97, portal: 172.16.0.220,3260]
Aug 09 03:06:36 user nova-compute[72351]: Logout of [sid: 70, target: iqn.2010-10.org.openstack:d14e301f-3f95-460b-b2d5-c863284d0e97, portal: 172.16.0.220,3260] successful.
Aug 09 03:06:36 user nova-compute[72351]:  stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 03:06:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d14e301f-3f95-460b-b2d5-c863284d0e97 -p 172.16.0.220:3260 --op delete {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:06:36 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d14e301f-3f95-460b-b2d5-c863284d0e97 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.019s {{(pid=83366) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:06:36 user nova-compute[72351]: DEBUG oslo.privsep.daemon [-] privsep: reply[b3363ef9-ce44-4707-a287-0be5038df1fe]: (4, ('', '')) {{(pid=83366) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Aug 09 03:06:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1270ea66-c122-4e5c-a80d-dfd44e326112 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=72351) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Aug 09 03:06:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.base [None req-1270ea66-c122-4e5c-a80d-dfd44e326112 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.246s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Aug 09 03:06:36 user nova-compute[72351]: DEBUG os_brick.initiator.connectors.iscsi [None req-1270ea66-c122-4e5c-a80d-dfd44e326112 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] <== disconnect_volume: return (247ms) None {{(pid=72351) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Aug 09 03:06:36 user nova-compute[72351]: DEBUG nova.virt.libvirt.volume.iscsi [None req-1270ea66-c122-4e5c-a80d-dfd44e326112 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Disconnected iSCSI Volume {{(pid=72351) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Aug 09 03:06:36 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Updating instance_info_cache with network_info: [{"id": "c69c5267-24fa-40f1-8f76-b4f2ec1ced93", "address": "fa:16:3e:ea:18:dd", "network": {"id": "45ee1620-bd54-4ca8-ab3a-9222782aea6f", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-173232413-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.86", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a636abaff2944499911a14ffb2846b48", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc69c5267-24", "ovs_interfaceid": "c69c5267-24fa-40f1-8f76-b4f2ec1ced93", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 03:06:36 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Releasing lock "refresh_cache-56b80e5d-b7f4-4780-987e-5def6d299f3d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 03:06:36 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Updated the network info_cache for instance {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9913}}
Aug 09 03:06:37 user nova-compute[72351]: DEBUG nova.objects.instance [None req-1270ea66-c122-4e5c-a80d-dfd44e326112 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lazy-loading 'flavor' on Instance uuid 56b80e5d-b7f4-4780-987e-5def6d299f3d {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 03:06:37 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-1270ea66-c122-4e5c-a80d-dfd44e326112 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "56b80e5d-b7f4-4780-987e-5def6d299f3d" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.816s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 03:06:37 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:06:37 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:06:38 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:06:38 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:06:39 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:06:39 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:06:39 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 03:06:39 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 03:06:39 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 03:06:39 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Auditing locally available compute resources for user (node: user) {{(pid=72351) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Aug 09 03:06:40 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56b80e5d-b7f4-4780-987e-5def6d299f3d/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:06:40 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:06:40 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56b80e5d-b7f4-4780-987e-5def6d299f3d/disk --force-share --output=json" returned: 0 in 0.147s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:06:40 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56b80e5d-b7f4-4780-987e-5def6d299f3d/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:06:40 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56b80e5d-b7f4-4780-987e-5def6d299f3d/disk --force-share --output=json" returned: 0 in 0.157s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:06:40 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 03:06:40 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 03:06:40 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Hypervisor/Node resource view: name=user free_ram=8767MB free_disk=25.15463638305664GB free_vcpus=11 pci_devices=[{"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72351) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Aug 09 03:06:40 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 03:06:40 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 03:06:40 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 56b80e5d-b7f4-4780-987e-5def6d299f3d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 03:06:40 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Total usable vcpus: 12, total allocated vcpus: 1 {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Aug 09 03:06:40 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Final resource view: name=user phys_ram=16011MB used_ram=1024MB phys_disk=40GB used_disk=1GB total_vcpus=12 used_vcpus=1 pci_stats=[] {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Aug 09 03:06:40 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 03:06:40 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 03:06:40 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Compute_service record updated for user:user {{(pid=72351) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Aug 09 03:06:40 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.178s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 03:06:45 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:06:45 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:06:45 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5003 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 03:06:45 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:06:45 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:06:45 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:06:45 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:06:50 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:06:55 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:07:00 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:07:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:07:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:07:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5003 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 03:07:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:07:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:07:05 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:07:10 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:07:15 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:07:20 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:07:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:07:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:07:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5003 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 03:07:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:07:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:07:25 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:07:30 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:07:30 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:07:30 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 03:07:30 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:07:30 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:07:30 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:07:30 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:07:34 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:07:34 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72351) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10461}}
Aug 09 03:07:35 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:07:35 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:07:35 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 03:07:35 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:07:35 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:07:35 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:07:36 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:07:36 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Starting heal instance info cache {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9842}}
Aug 09 03:07:36 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Rebuilding the list of instances to heal {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9846}}
Aug 09 03:07:37 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "refresh_cache-56b80e5d-b7f4-4780-987e-5def6d299f3d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Aug 09 03:07:37 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquired lock "refresh_cache-56b80e5d-b7f4-4780-987e-5def6d299f3d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Aug 09 03:07:37 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Forcefully refreshing network info cache for instance {{(pid=72351) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1996}}
Aug 09 03:07:37 user nova-compute[72351]: DEBUG nova.objects.instance [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lazy-loading 'info_cache' on Instance uuid 56b80e5d-b7f4-4780-987e-5def6d299f3d {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 03:07:37 user nova-compute[72351]: DEBUG nova.network.neutron [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Updating instance_info_cache with network_info: [{"id": "c69c5267-24fa-40f1-8f76-b4f2ec1ced93", "address": "fa:16:3e:ea:18:dd", "network": {"id": "45ee1620-bd54-4ca8-ab3a-9222782aea6f", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-173232413-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.86", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a636abaff2944499911a14ffb2846b48", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc69c5267-24", "ovs_interfaceid": "c69c5267-24fa-40f1-8f76-b4f2ec1ced93", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 03:07:37 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Releasing lock "refresh_cache-56b80e5d-b7f4-4780-987e-5def6d299f3d" {{(pid=72351) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Aug 09 03:07:37 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Updated the network info_cache for instance {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9913}}
Aug 09 03:07:38 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:07:39 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:07:40 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:07:40 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:07:40 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:07:41 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:07:41 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 03:07:41 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 03:07:41 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 03:07:41 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Auditing locally available compute resources for user (node: user) {{(pid=72351) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Aug 09 03:07:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56b80e5d-b7f4-4780-987e-5def6d299f3d/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:07:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56b80e5d-b7f4-4780-987e-5def6d299f3d/disk --force-share --output=json" returned: 0 in 0.132s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:07:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56b80e5d-b7f4-4780-987e-5def6d299f3d/disk --force-share --output=json {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Aug 09 03:07:42 user nova-compute[72351]: DEBUG oslo_concurrency.processutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/56b80e5d-b7f4-4780-987e-5def6d299f3d/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=72351) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Aug 09 03:07:42 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 03:07:42 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 03:07:42 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Hypervisor/Node resource view: name=user free_ram=8784MB free_disk=25.153888702392578GB free_vcpus=11 pci_devices=[{"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72351) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Aug 09 03:07:42 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 03:07:42 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 03:07:42 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Instance 56b80e5d-b7f4-4780-987e-5def6d299f3d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=72351) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Aug 09 03:07:42 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Total usable vcpus: 12, total allocated vcpus: 1 {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Aug 09 03:07:42 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Final resource view: name=user phys_ram=16011MB used_ram=1024MB phys_disk=40GB used_disk=1GB total_vcpus=12 used_vcpus=1 pci_stats=[] {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Aug 09 03:07:42 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 03:07:42 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 03:07:42 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Compute_service record updated for user:user {{(pid=72351) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Aug 09 03:07:42 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.186s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 03:07:45 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:07:47 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:07:50 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:07:55 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:07:58 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:08:00 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:08:01 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e3f9f95a-3d8b-481a-a9ed-3afedbb54f11 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Acquiring lock "56b80e5d-b7f4-4780-987e-5def6d299f3d" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 03:08:01 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e3f9f95a-3d8b-481a-a9ed-3afedbb54f11 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "56b80e5d-b7f4-4780-987e-5def6d299f3d" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 03:08:01 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e3f9f95a-3d8b-481a-a9ed-3afedbb54f11 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Acquiring lock "56b80e5d-b7f4-4780-987e-5def6d299f3d-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 03:08:01 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e3f9f95a-3d8b-481a-a9ed-3afedbb54f11 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "56b80e5d-b7f4-4780-987e-5def6d299f3d-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 03:08:01 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e3f9f95a-3d8b-481a-a9ed-3afedbb54f11 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "56b80e5d-b7f4-4780-987e-5def6d299f3d-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 03:08:01 user nova-compute[72351]: INFO nova.compute.manager [None req-e3f9f95a-3d8b-481a-a9ed-3afedbb54f11 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Terminating instance
Aug 09 03:08:01 user nova-compute[72351]: DEBUG nova.compute.manager [None req-e3f9f95a-3d8b-481a-a9ed-3afedbb54f11 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Start destroying the instance on the hypervisor. {{(pid=72351) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3124}}
Aug 09 03:08:01 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:08:01 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:08:01 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:08:01 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:08:02 user nova-compute[72351]: DEBUG nova.compute.manager [req-6b802fb3-770e-4f5c-8d10-c89a40ce247e req-ecb761a4-1236-441e-a9b0-665c2a88aa4a service nova] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Received event network-vif-unplugged-c69c5267-24fa-40f1-8f76-b4f2ec1ced93 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 03:08:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-6b802fb3-770e-4f5c-8d10-c89a40ce247e req-ecb761a4-1236-441e-a9b0-665c2a88aa4a service nova] Acquiring lock "56b80e5d-b7f4-4780-987e-5def6d299f3d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 03:08:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-6b802fb3-770e-4f5c-8d10-c89a40ce247e req-ecb761a4-1236-441e-a9b0-665c2a88aa4a service nova] Lock "56b80e5d-b7f4-4780-987e-5def6d299f3d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 03:08:02 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-6b802fb3-770e-4f5c-8d10-c89a40ce247e req-ecb761a4-1236-441e-a9b0-665c2a88aa4a service nova] Lock "56b80e5d-b7f4-4780-987e-5def6d299f3d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 03:08:02 user nova-compute[72351]: DEBUG nova.compute.manager [req-6b802fb3-770e-4f5c-8d10-c89a40ce247e req-ecb761a4-1236-441e-a9b0-665c2a88aa4a service nova] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] No waiting events found dispatching network-vif-unplugged-c69c5267-24fa-40f1-8f76-b4f2ec1ced93 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 03:08:02 user nova-compute[72351]: DEBUG nova.compute.manager [req-6b802fb3-770e-4f5c-8d10-c89a40ce247e req-ecb761a4-1236-441e-a9b0-665c2a88aa4a service nova] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Received event network-vif-unplugged-c69c5267-24fa-40f1-8f76-b4f2ec1ced93 for instance with task_state deleting. {{(pid=72351) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10810}}
Aug 09 03:08:02 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:08:02 user nova-compute[72351]: INFO nova.virt.libvirt.driver [-] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Instance destroyed successfully.
Aug 09 03:08:02 user nova-compute[72351]: DEBUG nova.objects.instance [None req-e3f9f95a-3d8b-481a-a9ed-3afedbb54f11 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lazy-loading 'resources' on Instance uuid 56b80e5d-b7f4-4780-987e-5def6d299f3d {{(pid=72351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1140}}
Aug 09 03:08:02 user nova-compute[72351]: DEBUG nova.virt.libvirt.vif [None req-e3f9f95a-3d8b-481a-a9ed-3afedbb54f11 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-08-09T03:00:41Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-SnapshotDataIntegrityTests-server-987829355',display_name='tempest-SnapshotDataIntegrityTests-server-987829355',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-snapshotdataintegritytests-server-987829355',id=52,image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNotwPL2HAojEvWq4iWFaRF6K2ttAaXMNCv5PqTAvmXQtQ2oZUb0Y5aQuvrV3ERKKWu0WcMHTZ53M0AXKuTVceZAdA/FjnYInOjduGzbBvX/kPOV2Ht34221e9wQ+LuGig==',key_name='tempest-keypair-756037336',keypairs=<?>,launch_index=0,launched_at=2023-08-09T03:00:49Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='a636abaff2944499911a14ffb2846b48',ramdisk_id='',reservation_id='r-v2t8deh2',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='18ccda7f-31d3-4978-ab9d-4ed2f74eff94',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-SnapshotDataIntegrityTests-248167435',owner_user_name='tempest-SnapshotDataIntegrityTests-248167435-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-08-09T03:00:49Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='44f56f4289eb4eb79f4751ef02264af8',uuid=56b80e5d-b7f4-4780-987e-5def6d299f3d,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "c69c5267-24fa-40f1-8f76-b4f2ec1ced93", "address": "fa:16:3e:ea:18:dd", "network": {"id": "45ee1620-bd54-4ca8-ab3a-9222782aea6f", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-173232413-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.86", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a636abaff2944499911a14ffb2846b48", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc69c5267-24", "ovs_interfaceid": "c69c5267-24fa-40f1-8f76-b4f2ec1ced93", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Aug 09 03:08:02 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-e3f9f95a-3d8b-481a-a9ed-3afedbb54f11 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Converting VIF {"id": "c69c5267-24fa-40f1-8f76-b4f2ec1ced93", "address": "fa:16:3e:ea:18:dd", "network": {"id": "45ee1620-bd54-4ca8-ab3a-9222782aea6f", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-173232413-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.86", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a636abaff2944499911a14ffb2846b48", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc69c5267-24", "ovs_interfaceid": "c69c5267-24fa-40f1-8f76-b4f2ec1ced93", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Aug 09 03:08:02 user nova-compute[72351]: DEBUG nova.network.os_vif_util [None req-e3f9f95a-3d8b-481a-a9ed-3afedbb54f11 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:ea:18:dd,bridge_name='br-int',has_traffic_filtering=True,id=c69c5267-24fa-40f1-8f76-b4f2ec1ced93,network=Network(45ee1620-bd54-4ca8-ab3a-9222782aea6f),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc69c5267-24') {{(pid=72351) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Aug 09 03:08:02 user nova-compute[72351]: DEBUG os_vif [None req-e3f9f95a-3d8b-481a-a9ed-3afedbb54f11 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:ea:18:dd,bridge_name='br-int',has_traffic_filtering=True,id=c69c5267-24fa-40f1-8f76-b4f2ec1ced93,network=Network(45ee1620-bd54-4ca8-ab3a-9222782aea6f),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc69c5267-24') {{(pid=72351) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Aug 09 03:08:02 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 17 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:08:02 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapc69c5267-24, bridge=br-int, if_exists=True) {{(pid=72351) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Aug 09 03:08:02 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:08:02 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:08:02 user nova-compute[72351]: INFO os_vif [None req-e3f9f95a-3d8b-481a-a9ed-3afedbb54f11 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:ea:18:dd,bridge_name='br-int',has_traffic_filtering=True,id=c69c5267-24fa-40f1-8f76-b4f2ec1ced93,network=Network(45ee1620-bd54-4ca8-ab3a-9222782aea6f),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc69c5267-24')
Aug 09 03:08:02 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-e3f9f95a-3d8b-481a-a9ed-3afedbb54f11 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Deleting instance files /opt/stack/data/nova/instances/56b80e5d-b7f4-4780-987e-5def6d299f3d_del
Aug 09 03:08:02 user nova-compute[72351]: INFO nova.virt.libvirt.driver [None req-e3f9f95a-3d8b-481a-a9ed-3afedbb54f11 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Deletion of /opt/stack/data/nova/instances/56b80e5d-b7f4-4780-987e-5def6d299f3d_del complete
Aug 09 03:08:02 user nova-compute[72351]: INFO nova.compute.manager [None req-e3f9f95a-3d8b-481a-a9ed-3afedbb54f11 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Took 0.88 seconds to destroy the instance on the hypervisor.
Aug 09 03:08:02 user nova-compute[72351]: DEBUG oslo.service.loopingcall [None req-e3f9f95a-3d8b-481a-a9ed-3afedbb54f11 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=72351) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Aug 09 03:08:02 user nova-compute[72351]: DEBUG nova.compute.manager [-] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Deallocating network for instance {{(pid=72351) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2263}}
Aug 09 03:08:02 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] deallocate_for_instance() {{(pid=72351) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1795}}
Aug 09 03:08:04 user nova-compute[72351]: DEBUG nova.network.neutron [-] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Updating instance_info_cache with network_info: [] {{(pid=72351) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Aug 09 03:08:04 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Took 1.41 seconds to deallocate network for instance.
Aug 09 03:08:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e3f9f95a-3d8b-481a-a9ed-3afedbb54f11 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 03:08:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e3f9f95a-3d8b-481a-a9ed-3afedbb54f11 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 03:08:04 user nova-compute[72351]: DEBUG nova.compute.manager [req-a37eaf6d-22fd-498c-af7f-bc23321d069a req-3c95c370-bce4-470b-9212-0e27085e01d6 service nova] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Received event network-vif-deleted-c69c5267-24fa-40f1-8f76-b4f2ec1ced93 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 03:08:04 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-e3f9f95a-3d8b-481a-a9ed-3afedbb54f11 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 03:08:04 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-e3f9f95a-3d8b-481a-a9ed-3afedbb54f11 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 03:08:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e3f9f95a-3d8b-481a-a9ed-3afedbb54f11 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.108s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 03:08:04 user nova-compute[72351]: INFO nova.scheduler.client.report [None req-e3f9f95a-3d8b-481a-a9ed-3afedbb54f11 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Deleted allocations for instance 56b80e5d-b7f4-4780-987e-5def6d299f3d
Aug 09 03:08:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-e3f9f95a-3d8b-481a-a9ed-3afedbb54f11 tempest-SnapshotDataIntegrityTests-248167435 tempest-SnapshotDataIntegrityTests-248167435-project-member] Lock "56b80e5d-b7f4-4780-987e-5def6d299f3d" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.590s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 03:08:04 user nova-compute[72351]: DEBUG nova.compute.manager [req-87cd5e6b-29cd-437e-8948-27425c498e62 req-938f278b-831e-4f28-bd17-92431e753531 service nova] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Received event network-vif-plugged-c69c5267-24fa-40f1-8f76-b4f2ec1ced93 {{(pid=72351) external_instance_event /opt/stack/nova/nova/compute/manager.py:11032}}
Aug 09 03:08:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-87cd5e6b-29cd-437e-8948-27425c498e62 req-938f278b-831e-4f28-bd17-92431e753531 service nova] Acquiring lock "56b80e5d-b7f4-4780-987e-5def6d299f3d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 03:08:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-87cd5e6b-29cd-437e-8948-27425c498e62 req-938f278b-831e-4f28-bd17-92431e753531 service nova] Lock "56b80e5d-b7f4-4780-987e-5def6d299f3d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 03:08:04 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [req-87cd5e6b-29cd-437e-8948-27425c498e62 req-938f278b-831e-4f28-bd17-92431e753531 service nova] Lock "56b80e5d-b7f4-4780-987e-5def6d299f3d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 03:08:04 user nova-compute[72351]: DEBUG nova.compute.manager [req-87cd5e6b-29cd-437e-8948-27425c498e62 req-938f278b-831e-4f28-bd17-92431e753531 service nova] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] No waiting events found dispatching network-vif-plugged-c69c5267-24fa-40f1-8f76-b4f2ec1ced93 {{(pid=72351) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Aug 09 03:08:04 user nova-compute[72351]: WARNING nova.compute.manager [req-87cd5e6b-29cd-437e-8948-27425c498e62 req-938f278b-831e-4f28-bd17-92431e753531 service nova] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Received unexpected event network-vif-plugged-c69c5267-24fa-40f1-8f76-b4f2ec1ced93 for instance with vm_state deleted and task_state None.
Aug 09 03:08:07 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:08:12 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:08:17 user nova-compute[72351]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1691550482.585101, 56b80e5d-b7f4-4780-987e-5def6d299f3d => Stopped> {{(pid=72351) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Aug 09 03:08:17 user nova-compute[72351]: INFO nova.compute.manager [-] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] VM Stopped (Lifecycle Event)
Aug 09 03:08:17 user nova-compute[72351]: DEBUG nova.compute.manager [None req-1787cb4f-bcc8-4cd8-9915-e5d5d57c7f64 None None] [instance: 56b80e5d-b7f4-4780-987e-5def6d299f3d] Checking state {{(pid=72351) _get_power_state /opt/stack/nova/nova/compute/manager.py:1766}}
Aug 09 03:08:17 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:08:17 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:08:17 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5001 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 03:08:17 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:08:17 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:08:17 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:08:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:08:27 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:08:32 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:08:32 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:08:34 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:08:34 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=72351) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10461}}
Aug 09 03:08:37 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:08:37 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:08:37 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Starting heal instance info cache {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9842}}
Aug 09 03:08:37 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Rebuilding the list of instances to heal {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9846}}
Aug 09 03:08:37 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Didn't find any instances for network info cache update. {{(pid=72351) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9928}}
Aug 09 03:08:39 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:08:39 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:08:40 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:08:42 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:08:42 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:08:42 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager.update_available_resource {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:08:42 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 03:08:42 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 03:08:42 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 03:08:42 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Auditing locally available compute resources for user (node: user) {{(pid=72351) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Aug 09 03:08:43 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 03:08:43 user nova-compute[72351]: WARNING nova.virt.libvirt.driver [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Aug 09 03:08:43 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Hypervisor/Node resource view: name=user free_ram=8919MB free_disk=25.181182861328125GB free_vcpus=12 pci_devices=[{"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=72351) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Aug 09 03:08:43 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Aug 09 03:08:43 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Aug 09 03:08:43 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Total usable vcpus: 12, total allocated vcpus: 0 {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Aug 09 03:08:43 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Final resource view: name=user phys_ram=16011MB used_ram=512MB phys_disk=40GB used_disk=0GB total_vcpus=12 used_vcpus=0 pci_stats=[] {{(pid=72351) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Aug 09 03:08:43 user nova-compute[72351]: DEBUG nova.compute.provider_tree [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed in ProviderTree for provider: 8eae9b37-1236-4e6d-91a5-b650de27b009 {{(pid=72351) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Aug 09 03:08:43 user nova-compute[72351]: DEBUG nova.scheduler.client.report [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Inventory has not changed for provider 8eae9b37-1236-4e6d-91a5-b650de27b009 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=72351) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Aug 09 03:08:43 user nova-compute[72351]: DEBUG nova.compute.resource_tracker [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Compute_service record updated for user:user {{(pid=72351) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Aug 09 03:08:43 user nova-compute[72351]: DEBUG oslo_concurrency.lockutils [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.137s {{(pid=72351) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Aug 09 03:08:47 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:08:47 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:08:47 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:08:47 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5001 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 03:08:47 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:08:47 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:08:47 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:08:50 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._cleanup_expired_console_auth_tokens {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:08:52 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:08:53 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:08:56 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:08:57 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:08:59 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:09:02 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:09:05 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._run_pending_deletes {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:09:05 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Cleaning up deleted instances {{(pid=72351) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11129}}
Aug 09 03:09:05 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] There are 0 instances to clean {{(pid=72351) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11138}}
Aug 09 03:09:07 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:09:09 user nova-compute[72351]: DEBUG oslo_service.periodic_task [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Running periodic task ComputeManager._cleanup_incomplete_migrations {{(pid=72351) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Aug 09 03:09:09 user nova-compute[72351]: DEBUG nova.compute.manager [None req-72f3c4c8-c855-41f2-9557-ceaf702e1255 None None] Cleaning up deleted instances with incomplete migration  {{(pid=72351) _cleanup_incomplete_migrations /opt/stack/nova/nova/compute/manager.py:11167}}
Aug 09 03:09:12 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:09:17 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Aug 09 03:09:17 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:09:17 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5001 ms, sending inactivity probe {{(pid=72351) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Aug 09 03:09:17 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:09:17 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=72351) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Aug 09 03:09:17 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 21 {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Aug 09 03:09:22 user nova-compute[72351]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=72351) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}